Fork me on GitHub

cuttlefish wiki

发表于 | 分类于

[TOC]
本文档为cuttlefish的官方wiki，所有内容都是cuttlefish github wiki上誊抄下来的，便于转为pdf，用于线下上下班路上无聊时学习的。如有任何侵权，请联系我：243818817@qq.com

#Home

##Welcome to the Cuttlefish wiki!

this cuttlefish is putting her tentacle up in solidarity with sea creatures the world over

Cuttlefish is a library for Erlang applications that wish to walk the fine line between erlang app.configs and a sysctl-like syntax. The name is a pun on the pronunciation of ‘sysctl’ and babblefish. Also, jokes are better explained.

This wiki is divided into sections based on what type of cuttlefish user you are:

1. Application User
    1.1 Cuttlefish for Application Users
2. Erlang Developer
    2.1 Cuttlefish for Erlang Developers
    2.2 Datatypes
    2.3 Unit Testing A Schema
    2.4 Cuttlefish Developer's API
3. Builder/Packager of Erlang Applications
    3.1 Cuttlefish for node_package users
    3.2 Cuttlefish for non node_package users

##Cuttlefish for Application Users
If you’re a user of an erlang application that supports cuttlefish conf files, welcome!

Cuttlefish is a pun on the pronunciation of ‘sysctl’. As someone configuring an application, you might be familiar with ‘sysctl’ and it’s with that in mind that we designed Cuttlefish.

Here are the simple rules of the syntax:

  • Everything you need to know about a single setting is on one line
  • Lines are structured Key = Value
  • Any line starting with # is a comment, and will be ignored.

The author of your particular application will have included a Cuttlefish schema as part of their application. That schema defines what keys can be defined and what the datatypes for those values can be. They also define how they map to the traditional app.config of an Erlang application. Honestly, Cuttlefish exists so you don’t have to worry about app.configs anymore. Leave that to Erlang developers who actually like that syntax!

With that said, you probably know about app.configs from previous versions of the Erlang application you’re dealing with. In that case, note that until you port your app.config to a .conf file, Cuttlefish will see your app.config sitting where it’s supposed to, and use that instead.

There may be settings that an Erlang developer chooses not to expose via Cuttlefish, and in that scenario, you have the ability to add an advanced.config file to the same directory your .conf file lives in. This will allow you to override anything Cuttlefish does with a specific Erlang term, if you wind up missing that kind of thing.

##Cuttlefish for Erlang Developers

Note: If you are looking for information in how to “wire” Cuttlefish to your application, consult Cuttlefish-for-node_package-users or Cuttlefish for non-node_package users.

As an Erlang developer, you’re probably used to application:set_env and app.config files. The good news for you is that you can keep on coding that way! The… additional news is that people who are using your application may not understand that syntax so easily. So how can you help? I’m glad you asked!

Write Cuttlefish Schemas!

You have a new job. You get to choose which knobs you expose to users! You can choose to name these things anything you want, so where you previously might have been confined to including a dependency’s application name, you are now not.

You can define datatypes for these settings, and you can explain to Cuttlefish how a simple name-value pair becomes part of a complex hierarchy of Erlang terms!

###Want to see how?
As the Erlang developer, you are the person responsible for being the ambassador of your setting to the world. Here’s how it looks in ASCII art:

1
2
3
4
5
6
7
┌--------------------┐                                                                      ┌--------------------┐
| <app>.conf         |                                                                      | app.config         |
|--------------------|       {mapping,                       {translation,                  | -------------------|
| my.setting = value | --->   "my.setting",            --->   "<dependency>.setting",  ---> | [{<dependency>, [  |
| ...                |        "<dependency>.setting",         fun(Conf) ->                  |    {setting, value}|
|                    |        []}.                              %% erlang fun               |  ]}, ... ].        |
└--------------------┘                                        end}.                         └--------------------┘

###The Schema
There are three types of Schema elements in Cuttlefish: mapping, translation and validator. It’s easy to tell the difference! They’re all tuples, and the first element is an atom: mapping, translation, or validator. You’re welcome.

####Mappings
@ - Annotations
Mappings are the only schema element type that support annotations and there are two available: @doc and @see.

@doc: If you write a multiline @doc it will be included in your generated .conf file. These docs will be available programatically. We chose to make it an annotation because as Erlangers, you already know and love @doc AND we didn’t want you to worry about multiline strings and an array of strings as a member of a proplist. This just seemed cleaner.

@see: If you already wrote an @doc for a setting, but you were describing how multiple settings work together, you can just specify an @see to that mappings name. It’s just a pointer to additional documentation.

1
2
3
4
5
%% @doc 'a.x' and 'a.y' set the start coordinate
{mapping, "a.x", "app.a.x", []}.

%% @see a.x
{mapping, "a.y", "app.a.y", []}.

Note: Cuttlefish has a bit of fun with these when generating your default .conf file. It will include the documentation inline if you have a @doc. If you also have a @see, it will include references to those other settings. If it only has an @see, it will include the @doc of the mapping you specified in the @see.

Elements of the mapping tuple
Aside from documentation, there is plenty going on with mappings, but the basic form is as follows:

1
2
%% {mapping, string(), string(), proplist()}
Mapping = {mapping, ConfKey, ErlangConfMapping, Attributes}.
  • element(1, Mapping) = mapping
  • element(2, Mapping) = ConfKey - the string key that you want this setting to have in the .conf file
  • element(3, Mapping) = ErlangConfMapping - the nested location of the thing in the app.config that this field represents
  • element(4, Mapping) = Attributes - other helpful things we’ll go into right… about… now!

Attributes is a proplist, and let’s assume you know how those work. Here are the keys in that proplist that we work with:

  • default - This is the default value for the setting. When a .conf file is generated, a line will be added to set the setting to this value. Additionally, if no value is specified in the .conf file, the value specified in the default attribute will be used when generating the app.config file.
  • new_conf_value - If this is defined, then when you generate a .conf file, this value will be used for the default setting. This causes the default attribute to be overridden when generating a .conf file, but has no effect on what default value is used when generating app.config.
  • commented - If this is defined, then when you generate a .conf file the documentation for this setting appears, along with the setting, but the setting is commented out and that comment includes this value.
  • datatype - This is the datatype for the field. For a list of those, check Datatypes.
  • hidden - If this atom is present or set to true, this value will be in the generated app.config, but not the generated .conf file. It still can be overridden in the .conf file, you just have to know about it. It’s a way of adding “undocumented knobs”.
  • include_default - If there is a substitutable value in the ConfKey, in the generated .conf file, this value is substituted. (don’t worry if that last one didn’t make so much sense now, I’ll explain more below)
  • validators - the list of names for validators for this mapping.
  • merge - is the only thing included just as an atom. It specifies that this is supplemental to an existing mapping, not a replacement mapping. The default behavior is to replace an existing mapping with this newer one

The best way to get it, is to take a look at some examples. Let’s start with Riak’s ring_size.

1
2
3
4
5
6
7
%% example of super basic mapping
%% @doc Default ring creation size.  Make sure it is a power of 2,
%% e.g. 16, 32, 64, 128, 256, 512 etc
{mapping, "ring_size", "riak_core.ring_creation_size", [
  {datatype, integer},
  {commented, 64}
]}.

First of all, comments before the @doc annotation are ignored, so feel free to put Schema specific comments in here as you see fit. Everything after the @doc in the comments, is part of the documentation. Cuttlefish will treat this documentation as:

1
2
3
4
[
  "Default ring creation size.  Make sure it is a power of 2,",
  "e.g. 16, 32, 64, 128, 256, 512 etc"
].

Then, we can also see from element(1) that this is a mapping. element(2) says that it’s represented by “ring_size” in the riak.conf file. element(3) says that it’s “riak_core.ring_creation_size” in the app.config. We also know from the attibutes that it is an integer, and that it will appear in the generated riak.conf file with a value of 64. It just so happens that the default is also 64, but that’s specified in riak_core’s app.src.

Let’s talk about element(3) here for a minute. What that means is that there’s an app.config out there that looks like this:

1
2
3
4
5
[
  {riak_core, [
    {ring_creation_size, X}
  ]}
].

and that we’re concerned with X.

An important note on Cuttlefish’s Philosophy
Here at cuttlefish, one thing we’re really disturbed by is this idea of a “magic number”. A magic number is a setting that we’re not actually sure where the default value comes from. In the above example it comes from riak_core’s app.src file, but they can come from anywhere, even the dreaded application:get_env/3.

We encourage every mapping in a cuttlefish schema to set an explicit default. In that case your application has a single location for default values, and it generates a complete app.config.

Now, if life were as simple as 1:1 mappings like this, we’d be done. But it’s not, and so we need to introduce translations.

####Lost in Translations
Actually, they’re pretty easy.
A translation looks like this:

1
2
%% {translation, string(), fun((proplist()) -> term())}
Translation = {translation, ErlangConfMapping, TranslationFunction}.

Let’s break it down:

  • element(1, Translation) = translation
  • element(2, Translation) = ErlangConfMapping this is the same as the corresponding ErlangConfMapping in the mapping above. This is how we tie back to a mapping
  • element(2, Translation) = TranslationFunction this is a fun() that takes in a proplist representing the .conf file and returns an erlang term. This erlang term will be the value of the ErlangConfMapping.

Ok, that does sound more confusing than it should. Let’s take a look at one, you’ll like it better in practice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
%% @doc How Riak will repair out-of-sync keys. Some features require
%% this to be set to 'active', including search.
%%
%% * active: out-of-sync keys will be repaired in the background
%% * passive: out-of-sync keys are only repaired on read
%% * active-debug: like active, but outputs verbose debugging
%%   information
{mapping, "anti_entropy", "riak_kv.anti_entropy", [
  {datatype, {enum, [active, passive, 'active-debug']}},
  {default, active}
]}.

{translation,
 "riak_kv.anti_entropy",
 fun(Conf) ->
    Setting = cuttlefish:conf_get("anti_entropy", Conf),
    case Setting of
      active -> {on, []};
      'active-debug' -> {on, [debug]};
      passive -> {off, []};
      _Default -> {on, []}
    end
  end
}.

See what’s happening? First of all, you need a mapping. If you don’t have one, don’t bother writing a translation for it. It will not get run. It will also not get run if you have a mapping for it, but you don’t have a value for it. That value can be a default in the mapping or set in your .conf file. The mapping we defined for “anti_entropy” says that it’s an enum with values “active”, “passive”, and “active-debug”. The configuration in the app.config is more complicated. Basically, it works like this:

  • active - {on, []}
  • passive - {off, []}
  • active-debug - {on, [debug]}

It’s a relatively simple translation, but we want to spare non-Erlangers from this very Erlangy syntax. So, we give them the values “active”, “passive”, and “active-debug” and the translation “translates” (not just a clever nickname!) them into the erlang value we expect.

Translations and the Conf proplist

You may have noticed that the translation fun takes an argument Conf. Conf is a cool proplist and needs to get its “props”. Here are some fun facts about Conf:

By the time Conf gets to this function, a lot has happened since it was a .conf file. First of all, if you omitted a variable in your .conf file, but it had a default, that default value is in Conf.

Conf is a proplist, but its keys are no longer in the form “a.b.c”. For “easier” erlang processing (pattern matching, etc), the keys come in the form [“a”, “b”, “c”]. The values are also different. As you might imagine, in the .conf file, they’re all just strings. If you specified a datatype in the mapping, it will be transformed into that erlang type by the time it gets here.

There are convenience functions in the cuttlefish module for accessing these. They allow you to pass variables in with either “a.b.c” or [“a”, “b”, “c”] notation. For more on that, see the cuttlefish_variable module.

See the Cuttlefish Developer’s API page for more.

The Curious Case of Lager Config

There are other cases when multiple values turn into a single app.config complex data structure. Take lager as an example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

%% complex lager example
%% @doc location of the console log
{mapping, "log.console.file", "lager.handlers", [
  {default, "./log/console.log"}
]}.

%% *gasp* notice the same @mapping!
%% @doc location of the error log
{mapping, "log.error.file", "lager.handlers", [
  {default, "./log/error.log"}
]}.

%% *gasp* notice the same @mapping!
%% @doc turn on syslog
{mapping, "log.syslog", "lager.handlers", [
  {default, off},
  {datatype, enum},
  {enum, [on, off]}
]}.

{ translation,
  "lager.handlers",
  fun(Conf) ->
    SyslogHandler = case cuttlefish:conf_get("log.syslog", Conf) of
      on ->  {lager_syslog_backend, ["riak", daemon, info]};
      _ -> undefined
    end,
    ErrorHandler = case cuttlefish:conf_get("log.error.file", Conf) of
      undefined -> undefined;
      ErrorFilename -> {lager_file_backend, [{file, ErrorFilename}, {level, error}]}
    end,
        ConsoleHandler = case cuttlefish:conf_get("log.console.file", Conf) of
          undefined -> undefined;
          ConsoleFilename -> {lager_file_backend, [{file, ConsoleFilename}, {level, info}]}
        end,
        lists:filter(fun(X) -> X =/= undefined end, [SyslogHandler, ErrorHandler, ConsoleHandler])
  end
}.

We define three mappings here, that have different values in the riak.conf file, but represent a complex list of lager handlers in the app.config. The solution is to have them all map to the same ErlangConfMapping, which references lager.handlers. When we create a translation for that, we’re basically saying that “The return value of this function will be the value of {lager, [{handers, X}]}”. that was a weird way of saying it, but the generated app.config looks like this:

1
2
3
4
5
6
7
{lager,
     [
      {handlers,
          [{lager_syslog_backend,["riak",daemon,info]},
           {lager_file_backend,[{file,"/var/log/error.log"},{level,error}]},
           {lager_file_backend,[{file,"/var/log/console.log"},{level,info}]}]},
          ]},

Lists and Proplists and $names, oh my!

Sometimes you’ll find yourself needing to map elements of a list or proplist. Consider the way we configure HTTP listeners for Riak.

1
2
3
4
5
6
7
8
9
10
{riak_core,
    [
      {http,
        [
          {"127.0.0.1",8098},
          {"10.0.0.1",80}
        ]
      }
    ]
  }

We got really aggressive with the line breaks here, to illustrate that riak_core.http is a list of {IP, Port} tuples. Now, say for some reason, you wanted 10 of these listeners. We’re not here to judge you, we’re here to help. What we didn’t want to do was introduce some kind of list data structure on the right hand side of our .conf file. Instead we took a “list element per line” approach. We wanted to give you a syntax that was something like this:

1
2
listener.http.internal = 127.0.0.1:8098
listener.http.external = 10.0.0.1:80

But wait, what’s the deal with this “internal”/“external” business? Well, the mapping is defined with a wildcard. Think of it like a match group in a regex.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

%% HTTP Listeners
%% @doc listener.http.<name> is an IP address and TCP port that the Riak
%% HTTP interface will bind.
{mapping, "listener.http.$name", "riak_api.http", [
  {default, {"127.0.0.1", 8098}},
  {datatype, ip},
  {include_default, "internal"}
]}.

{translation,
 "riak_api.http",
  fun(Conf) ->
      HTTP = cuttlefish_variable:filter_by_prefix("listener.http", Conf),
      [ IP || {_, IP} <- HTTP]
  end
}.

See the $name? it can be anything! Then the translation is “smart” enough to parse all the listner.http.* config keys and create the list of {IP, Port}s for the “riak_core.http” section. (TODO: in the future, we’ll add the ability to refer back to $name as a variable, but for now, we didn’t need to because in this case, name was a throwaway).

Also, notice the {datatype, ip}, that is smart enough to turn “IP:Port” into {IP, Port}. Don’t worry, it works for IPv6 too. More on Datatypes.

More about include_default

This is the perfect place to talk about ‘include_default’. If there’s a wildcard in the ConfKey, we don’t want to include that wildcard in the default generated .conf file, so we need an example. The value from include_default provides that sample. So, the generated .conf looks like this:

1
2
3
## listener.http.<name> is an IP address and TCP port that the Riak
## HTTP interface will bind.
listener.http.internal = 127.0.0.1:8098

So, $names don’t matter?

What’s in a $name? That which we call internal by any other name would still bind internally; so HTTP would, were it not HTTP called.

Not so fast, Billy! Sometimes it does matter. Let’s look at the userlist in Riak Control:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
%% @doc If auth is set to 'userlist' then this is the
%% list of usernames and passwords for access to the
%% admin panel.
{mapping, "riak_control.user.$username.password", "riak_control.userlist", [
  {default, "pass"},
  {include_default, "user"}
]}.

{translation,
"riak_control.userlist",
fun(Conf) ->
  UserList = lists:filter(
    fun({K, _V}) ->
       cuttlefish_variable:is_fuzzy_match(K, string:tokens("riak_control.user.$username.password", "."))
    end,
    Conf),
  Users = [ {Username, Password} || {[_, _, Username, _], Password} <- UserList ],
  case Users of
      [] ->
          throw(unset);
      _ -> Users
  end

end}.

Right now, we’re leaving it up to the translation fun to tokenize the string and extract the username, but it shouldn’t have to. We should provide helpers for this, and we will.

####Validators

You can write validator functions to perform advanced validation for mappings. Let’s take the ring size in riak as an example:

1
2
3
4
{validator, "ring_size", "not a power of 2 greater than 1",
 fun(Size) ->
  Size > 1 andalso (Size band (Size-1) =:= 0)
 end}.

What we’re saying here is that ring_size has to be an integer that’s a power of 2 and greater than 1.

So how does this get triggered?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
%% @doc Number of partitions in the cluster (only valid when first
%% creating the cluster). Must be a power of 2, minimum 8 and maximum
%% 1024.
{mapping, "ring_size", "riak_core.ring_creation_size", [
  {datatype, integer},
  {default, 64},
  {validators, ["ring_size^2", "ring_size_max", "ring_size_min"]},
  {commented, 64}
]}.

%% ring_size validators
{validator, "ring_size_max", "2048 and larger are supported, but considered advanced config",
 fun(Size) ->
  Size =< 1024
 end}.

{validator, "ring_size^2", "not a power of 2",
 fun(Size) ->
  (Size band (Size-1) =:= 0)
 end}.

{validator, "ring_size_min", "must be at least 8",
 fun(Size) ->
  Size >= 8
 end}.

In the end, we went with three different validators. The validators property of a mapping specifies which validators get run for a field.They can be reused on multiple mappings, and you can add as many as you want.

####I like Eunit!
Me too. It’d be pretty crazy to just write a bunch of Erlang and expect it to just work in your application. So we decided to give you a module of unit test helper functions: cuttlefish_unit. Please see the Unit Testing A Schema page.

##Datatypes

###normal datatypes
As of January 2014, Cuttlefish supports the following datatypes:

integer
integer is exactly what it sounds like.

{enum, Enum = [atom()|string()]}

enum is great! You know the fixed set of options for a setting, so use this one. Those fixed options are provided in the second element of the enum tuple.

If you wish to map atoms or strings to some other value, use tuples instead:

{enum, Enum = [{atom()|string(), term()}]}

ip

The ip datatype exists currently as an IP/Port combo. You can specify it as a default as either a string (e.g. “127.0.0.1:8098”) or a tuple ({“127.0.0.1”, 8098})

string

string is the default datatype. Everything in a .conf file is a string anyway, so we’re guaranteed a successful conversion.

atom

it works alot like string, but outputs an erlang atom to your generated app.config.

flag

Flags are pretty cool. We got sick of writing the same boolean translation over and over again. Flags make this easier. They are for variables that have 2 possible values, and can be specified in three formats:

flag The trivial case: “on” in your conf file becomes true in your app.config. “off” becomes false

{flag, On, Off} Say you don’t like “on” and “off”. You can change those strings to anything you want here.

{flag, {On, OnVal}, {Off, OffVal}} Say the two values you’re mapping to aren’t true and/or false you can change those with OnVal/OffVal

{duration, BaseUnit = f | w | d | h | m | s | ms }

durations are fixed intervals of time, the largest unit of which will be a one week. Anything larger will have to be expressed in terms of weeks, since larger units (month, year) are of variable duration. durations manifest in .conf files as strings like this: 1w2d.

The following units are supported:

  • f - fortnight
  • w - week
  • d - day
  • h - hour
  • m - minute
  • s - second
  • ms - millisecond

You can use any combination of these. I’m not sure why you’d want to specify 1w13ms, but you can.

Did I mention you can use floats here? 0.5d? no probalo.

duration will convert automatically to BaseUnit in your app.config. If you want it to be seconds instead of milliseconds, just use the datatype {duration, s}. It will round up to the nearest second, so 1ms = 999ms = 1s. Confused? You won’t be, after this week’s episode of…Soap.

bytesize

In your app.config, these will end up as bytes in integers, so 1KB = 1024.

bytesize will work pretty much like duration but with three differences.

  • The units will are MB, KB, and GB
  • If no unit is specified, it’s just bytes
  • You will only be able to use ONE unit. e.g. no 4gb3kb <- that makes no sense!

Note: lowercase units (i.e. gb, mb, and kb) are ok, but mixed case are not. That’s to avoid confusion with Megabits

file and directory

file and directory are effectively strings. In the future they could allow for included validators like valid path and exists. Some files may need to exist, some may not. some may need to be writable.

###Extended Datatypes
Sometimes you need a composite of two datatypes. Take a look at this example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
%% @doc This option specifies how many of each type of fsm may exist
%% concurrently.  This is for overload protection and is a new
%% mechanism that obsoletes 1.3's health checks. Note that this number
%% represents two potential processes, so erlang.process_limit in
%% vm.args should be at least 3X more. Setting this value to infinite
%% disables fsm overload protection.
{mapping, "max_concurrent_requests", "riak_kv.fsm_limit", [
  {default, 50000},
  {datatype, [integer, {atom, infinite}]},
  {level, advanced}
]}.

{translation, "riak_kv.fsm_limit",
 fun(Conf) ->
  TheLimit = cuttlefish:conf_get("max_concurrent_requests", Conf),
  case TheLimit of
      infinite -> undefined;
      Int when is_integer(Int) -> Int;
      _ ->
          %% This would have been caught earlier in datatype validation
          cuttlefish:invalid("should be an integer")
  end
 end
}.

Look at that datatype! We’re saying this can be any integer OR the atom ‘infinite’. Under the hood, riak_kv treats ‘undefined’ as ‘infinite’, so we take care of that in the translation.

NOTE when providing a default, it must be satisfied by the head of the datatypes list.

##Unit Testing A Schema
It’s a great time to be an Engineer. You can test things before you build and ship them. Yay! Let’s talk about the cuttlefish_unit module.

Setup
To generate an app.config in an eunit test. Use the cuttlefish_unit:generated_template_config/3. The three arguments are:

List Of Schema Files
This is the list of schema files you’re testing. If you’ve followed our conventional approach, you’ve stored myappname.schema in your priv directory. to access it, set this argument to ["../priv/myappname.schema"]. If it gets too big and you need multiple files, just append to this list.

Expected Processed Conf
This is in the same syntax and the Conf proplist that translations expect. See Translations and the Conf Proplist

Mustache Context
If you’re using rebar’s mustache templates to substitute values into your schema, please include them in a proplist for this argument.

1
2
3
4
5
6
7
8
9
[
        {handoff_port, "8099"},
        {ring_state_dir , "./ring"},
        {platform_bin_dir , "./bin"},
        {platform_data_dir, "./data"},
        {platform_etc_dir , "./etc"},
        {platform_lib_dir , "./lib"},
        {platform_log_dir , "./log"}
].

If you have a mapping like this:

1
2
3
{mapping, "a.b", "app.b", [
	{default, {{handoff_port}} }
]}.

then would turn into “8099”.

NOTE: for some unidentifiable reason, if you don’t put a space between the second and third }s, the whole thing explodes. This is a mustache issue, deal with it.

OTHER NOTE: these substitutions are made with rebar’s mustache module. If you’re running eunit outside of rebar, these won’t work.

Free Assertions (who doesn’t want those?)

Config is the return from cuttlefish_unit:generated_template_config/3.

assert_config

1
cuttlefish_unit:assert_config(Config, "riak_core.default_bucket_props.n_val", 3),

Assert that Config (which is a valid app.config) that has the nested value 3, like so:

1
2
3
4
5
6
[{riak_core, [
		{default_bucket_props, [
			{n_val, 3}
			]}
	]}
]

assert_not_configured

1
cuttlefish_unit:assert_not_configured(Config, "riak_core.ssl.certfile"),

Asserts that there is no value configured in the Config (app.config). This is different from the atom undefined.
assert_error

1
cuttlefish_unit:assert_error(Config),

Asserts that this config is an error state.
assert_error_in_phase

1
cuttlefish_unit:assert_error_in_phase(Config, Phase),

There are several phases to generating with cuttlefish: add_defaults, transform_datatypes, validation, apply_translations

assert_error_message

1
cuttlefish_unit:assert_error_message(Config, Message),

Asserts an error state, and that one of the errors is the message Message

assert_valid_config

1
cuttlefish_unit:assert_valid_config(Config)

It’s basically the opposite of assert_error/1.

##Cuttlefish Developer’s API

These are functions that are useful in writing translations

cuttlefish

cuttlefish:conf_get/2

Extract a specific value from Conf. If it’s not set, it throws a not_found, which cuttlefish knows how to wrap in an error. If you don’t like that behavior and want to use a default, choose the 3 arity version below. Example

1
2
3
4
5
6
Conf = [
	{["a", "b"], 4}
],

cuttlefish:conf_get("a.b", Conf), %% returns 4
cuttlefish:conf_get("a.c", Conf), %% throws 'not_found'

cuttlefish:conf_get/3

1
2
3
4
5
6
Conf = [
	{["a", "b"], 4}
],

cuttlefish:conf_get("a.b", Conf, undefined), %% returns 4
cuttlefish:conf_get("a.c", Conf, undefined), %% returns 'undefined'

cuttlefish:unset/0
When writing a translation and you get to a point where you don’t want a value written to the app.config, just call cuttlefish:unset() and we’ll take it from there.

cuttlefish:invalid/1

When you can’t generate a valid config value, and you know it, call this function with an error message and cuttlefish will report it properly.

cuttlefish_variable

cuttlefish_variable:tokenize/1

This is pretty much string:tokens(Key, ".") but it skips escaped dots.

1
2
string:tokens("a.b\\.c.d", "."), %% returns ["a", "b\\", "c", "d"]
cuttlefish_variable:tokenize("a.b\\.c.d", "."), %% returns ["a", "b.c", "d"]

cuttlefish_variable:fuzzy_matches/2

If you’ve used a $name in your mapping, you can get a list of all possible values of $name.

1
2
3
4
5
6
7
Conf = [
	{["a", "b1", "c"], 4},
	{["a", "b1", "d"], 4},
	{["a", "b2", "c"], 4},
],

cuttlefish_variable:fuzzy_matches(["a", "$name", "c"], Conf), %% returns [{"$name", "b1"}, {"$name", "b2"}].

Unfortunately, if you ran the same function on [“a”, “$name”, “d”] you’d only return [{“$name”, “b1”}]. [https://github.com/basho/cuttlefish/issues/99] is an open issue to fix this.

cuttlefish_variable:filter_by_prefix/2

Basically, this is “give me everything that starts with X”.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
%% HTTP Listeners
%% @doc listener.http.<name> is an IP address and TCP port that the Riak
%% HTTP interface will bind.
{mapping, "listener.http.$name", "riak_api.http", [
  {default, {"127.0.0.1", 8098}},
  {datatype, ip},
  {include_default, "internal"}
]}.

{translation,
 "riak_api.http",
  fun(Conf) ->
      HTTP = cuttlefish_variable:filter_by_prefix("listener.http", Conf),
      [ IP || {_, IP} <- HTTP]
  end
}.

##Builder/Packager of Erlang Applications

###Cuttlefish for node_package users

https://github.com/basho/node_package has built-in optional support for cuttlefish. To integrate cuttlefish into your node_package application, you will need to perform the following steps:

  • Add Cuttlefish to rebar.config
  • Enable the Cuttlefish rebar plugin
  • Inform reltool.config about schema files
  • Enable Cuttlefish in vars.config

rebar.config

You’ll need to add cuttlefish as a dependency, which will look something like this:

1
{cuttlefish, ".*", {git, "git://github.com/basho/cuttlefish", {branch, "master"}}}

and you’ll also need to teach it about the cuttlefish_rebar_plugin, like this:

1
2
{plugin_dir, "deps/cuttlefish/src"}.
{plugins, [cuttlefish_rebar_plugin]}.

reltool.config

You’ll need to tell reltool.config about any schema files you want included. You’ll also need to tell reltool.config the priority of those files. Add them to your {overlay, [...]} section, like this:

1
2
3
4
5
6
%% Cuttlefish Schema Files have a priority order. 
%% Anything in a file prefixed with 00- will override
%% anything in a file with a higher numbered prefix.
{template, "files/riak.schema", "lib/00-riak.schema"},
{template, "../deps/<dependency_name>/priv/<schema_name>.schema", "lib/02-<dependency>.schema"},
{template, "files/vm.args", "etc/vm.args"},

The priority is defined as “alphabetical order”. which means, if the file is named A.schema, everything in it overrides things in B.schema if there’s a conflict. As the reltool.config author, you can determine schema priority by renaming schema files in the template tuple.

Why would you do this? Say we provided a riak_core.schema (and we plan to!), say you like all our setting names for your riak_core application, except one. You could override that one in your own schema! Freedom! Choices! Options! See what I do for you?

vars.config

Everything you’ve done until now will be for naught if you don’t add cuttlefish to your vars.config. Here’s how!

1
2
{cuttlefish,         "on"}.
{cuttlefish_conf,    "<whatever_you_want_your_default_config_file_to_be_called>.conf"}.

That’s it! Truth be told, it could be {cuttlefish, “anything!”}, as all we’re checking is that this is not undefined.

Switching to Cuttlefish

Say you’re using app.config, like everyone is… when your first cuttlefish compliant version ships, it won’t blow away existing app.configs, and if an app.config is present, cuttlefish will short circuit and just use that.

###Cuttlefish for non node_package users
Basically you’ve to follow the steps outlined in the Cuttlefish for node_package users guide, but there are some additions since you don’t get the free changes to your startup scripts.

First of you need to make sure the cuttlefish binary (aka escript) gets copied over to your release, you can do this by adding this line to your reltool.config in overlay section:

1
{copy, "../deps/cuttlefish/cuttlefish", "bin/cuttlefish"},

If you want to have a custom named .conf file you also need to put a rebar.config into the rel/ folder containing the like:

1
{cuttlefish_filename, "sniffle.conf"}.

Then what you have to do is to adjust your startup scripts to honor the configuration options of cuttlefish and make the fish generate your configs. If you’re using rebar generated releases the file should be in rel/files/<your app>

So lets look at the parts that need to change. (extra from the node_package code for cuttlefish)

The code extracting the node name, since at this point cuttlefish has not generated a vm.args file the original code would fail. To handle it gracefully it checks first for the old stile vm.args then proceeds to use the cuttlefish config.

1
2
3
4
5
6
# Extract the target node name from node.args
NAME_ARG=`grep -e '-[s]*name' $RUNNER_ETC_DIR/vm.args`
if [ -z "$NAME_ARG" ]; then
    echo "vm.args needs to have either -name or -sname parameter."
    exit 1
fi

has to change to:

1
2
3
4
5
6
7
8
9
10
11
12
# Extract the target node name from node.args
NAME_ARG=`egrep '^\-name' $RUNNER_ETC_DIR/vm.args 2> /dev/null`
if [ -z "$NAME_ARG" ]; then
    NODENAME=`egrep '^[ \t]*nodename[ \t]*=[ \t]*' $RUNNER_ETC_DIR/{{cuttlefish_conf}} 2> /dev/null | tail -n 1 | cut -d = -f 2`
    if [ -z "$NODENAME" ]; then
        echo "vm.args needs to have a -name parameter."
        echo "  -sname is not supported."
        exit 1
    else
        NAME_ARG="-name ${NODENAME# *}"
    fi
fi

Same goes for the code that extracts the cookie:

1
2
3
4
5
6
# Extract the target cookie
COOKIE_ARG=`grep -e '-setcookie' $RUNNER_ETC_DIR/vm.args`
if [ -z "$COOKIE_ARG" ]; then
    echo "vm.args needs to have a -setcookie parameter."
    exit 1
fi

becomes

1
2
3
4
5
6
7
8
9
10
11
12
# Extract the target cookie
COOKIE_ARG=`grep -e '-setcookie' $RUNNER_ETC_DIR/vm.args`
if [ -z "$COOKIE_ARG" ]; then
#TODO: change {{cuttlefish_conf}} to something more senisble!
    COOKIE=`egrep '^[ \t]*distributed_cookie[ \t]*=[ \t]*' $RUNNER_ETC_DIR/{{cuttlefish_conf}} 2> /dev/null | cut -d = -f 2`
    if [ -z "$COOKIE" ]; then
        echo "vm.args needs to have a -setcookie parameter."
        exit 1
    else
        COOKIE_ARG="-setcookie $COOKIE"
    fi
fi

That’s it for the preconfiguration, now one thing is left: make cuttlefish generate your files and properly pass them to the VM when started. For this we look at the console section where you find something like this to create the erlexec command line.

1
CMD="$BINDIR/erlexec -boot $RUNNER_BASE_DIR/releases/$APP_VSN/$BOOTFILE -embedded -config $RUNNER_ETC_DIR/app.config -args_file $RUNNER_ETC_DIR/vm.args -- ${1+"$@"}"

Since we don’t have a app.conf and vm.args when using cuttlefish we need to change this a bit around and add some code to generate our files.

1
2
3
4
5
6
7
8
if CUTTLEFISH_CONFIG=$($RUNNER_BASE_DIR/bin/cuttlefish -e $RUNNER_ETC_DIR -d {{db_path}}/generated.configs -s $RUNNER_BASE_DIR/share/schema/ -c $RUNNER_ETC_DIR/{{cuttlefish_conf}})
        then
            CONFIG_FILES="$CUTTLEFISH_CONFIG"
        else
            echo "Cuttlefish failed! Oh no!: $CUTTLEFISH_CONFIG"
            CONFIG_FILES="-config $RUNNER_ETC_DIR/app.config -args_file $RUNNER_ETC_DIR/vm.args"
        fi
        CMD="$BINDIR/erlexec -boot $RUNNER_BASE_DIR/releases/$APP_VSN/$BOOTFILE -embedded $CONFIG_FILES -- ${1+"$@"}"

Please note that your schema folder might be different, so you might need to replace $RUNNER_BASE_DIR/share/schema/ with whatever place you specified in the reltool.conf as a destination for the .schema files.

That’s it, happy cuttling!

Note: distributed_cookie and nodename in the snippets above are defined as mappings in your schema files and should be adjusted for your desired naming scheme.

Generating a default conf file

You can manually generate a default .conf file from your schemas at any time with the following snippet.

1
2
{_, Mappings, _} = cuttlefish_schema:files(filelib:wildcard("path/to/*.schema")).
ok = cuttlefish_conf:generate_file(Mappings, "path/to/generated.conf").

This is usually handled by the cuttlefish_rebar_plugin, but if you’re not using that then this will help.

感谢您的赞赏,赚点辛苦奶粉钱