Resources

Cloud Workflow Resources Overview

Resources present the author with a consistent toolset for managing external state. Through resources one can enumerate collections, locate items and act on them. For example, resources allow managing RightScale abstractions like deployments, servers, server arrays, instances etc. Each resource corresponds to an underlying abstraction and is scoped to a namespace. For example, a RightScale Server resource might have:

• namespace: rs
• type: servers
• href: servers/123
• fields: cloud: aws-us-east, datacenter: us-east-1a, launched_at: 2011/8/1 16:56:87, ...
• links: current_instance, deployment, ...
• actions: launch(), terminate(), clone(), ...
• type actions: get(), create()

Namespaces

Resources are exposed to the cloud workflow engine in the form of HTTP APIs. A namespace exposes a given API to cloud workflows. The rs namespace encapsulates RightScale API 1.5 and gives access to all the resources it defines. It also implicitly scopes that access to the account running the process.

The rest of this document describes how to work with resources in a cloud workflow. Resources are managed in collections and expose fields, links, and actions. The Mapping RightScale Resources to the APIs section may help make things more concrete as it lays out how the various constructs map back to the RightScale API in the rs namespace.

Resource Collections

An important aspect of the language that may not be intuitive is that a cloud workflow always acts on a collection of resources. The resources in a collection are always of the same type (such as all servers or all instances). For example, executing resource actions is always done on a collection:

@servers.launch()

The expression above will execute the action launch() on all the resources in the @servers resource collection. Resource actions are further described below.

RCL includes built-in functions to manipulate collections. Functions like first(), last(), etc. also always return collections of resources (albeit made of a single resource in both these cases). Always working with collections allows for a much simpler language that does not need to differentiate between managing collections or unique items. The language supports constructs for checking the size of a collection and for extracting items (creating another collection with the extracted items). A collection may contain any number of resources including no resource. Executing an action on an empty resource collection is not an error, it just has no effect.

Note: An empty resource collection still has an associated resource type, attempting to execute an action not supported by the resource type on an empty collection raises an error.

Resource Fields

A cloud workflow primarily orchestrates how state propagates through various resources. Computation done on that state should be done primarily by systems exposing these resources (e.g. RightScale's APIs). Sometimes though it can be useful for a cloud workflow to read the value of a given resource field so it can trigger the proper activities. Fields can be read as follows:

@servers.name   # Retrieve name of first server in collection
@servers.name[] # Retrieve names of all servers in collection

The first line returns a string while the second line returns an array of strings, each element corresponding to the name of the resource in the @servers collection at the same index.

Note: Resource references always contain a collection of resources and extracting a field like in the expression @servers.name[] always returns an array, even if the collection consists of only one server. The notation sans brackets is meant to help in the case a collection only contains one element.

@servers.name[][0]

d"year/month/day [hours:minutes:seconds] [AM|PM]"

d"2012/07/01"d"2012/07/12 04:21:14 PM"

Resource Links

Resources also contain links that point to other collections. The main difference with respect to fields is that links yield resource collections rather than JSON values. For example, the RightScale deployments resource type has a link named servers that represents the collection of all servers in a deployment.

Retrieving the link on a collection will "flatten" the result. For example, if deployment A contains server S1 and S2 and deployment B contains server S3 accessing the link servers on a collection made of these two deployments will yield a collection composed of S1, S2, and S3.

Links on resource collections can be followed using the following syntax:

@servers = @deployments.servers()

The snippet above assumes that the @deployments reference contains a resource collection made of deployments.

Links may take arguments depending on the resource type and the nature of the link. Some links may return a single resource while others may return multiple resources. In both these cases arguments may be used to further specify what should be returned. Some arguments control which resource fields in the results should be initialized (views) while others affect what resources should be returned (filters). For example:

@servers = @deployments.servers(filter: ["state==operational"])

The example above only returns the operational servers in the given deployments. As with resource fields and actions, the complete list of available links and their arguments is documented in the API 1.5 reference.

Provisioning and Terminating Resources

At the end of the day Cloud Workflow is all about managing resources in the cloud. The most common activities executed by workflows consist of creating and deleting resources. Often times just creating the resource is not enough, for example just creating a RightScale server through the RightScale API is not that useful, instead it would be much more useful if a single operation could create the server, launch it and wait until it is operational. This is the intent behind the built-in provision() function. This function behaves differently depending on the resource being provisioned. The exact behavior for each RightScale resource is described in the Functions section.

Similarly, just deleting a RightScale server is not that useful; instead the delete() function terminates the server, waits until it is terminated and then deletes it.

In both cases (provision() and delete()) the intent is for the function to act atomically. In particular the provision() function will clean up if anything fails after the resource has been created. For example, if the server fails to go to operational (strands in RightScale) then it is terminated and deleted automatically. If you need more control on how the resource is created or how failure is handled then use the create() and destroy() actions described in the Resource Actions section below, these are a subset of the actions used internally by provision() and delete().

# Create the new deployment declaration, note that the value
# gets assigned to a variable whose name starts with a "@"
# Also note that "fields" values must be JSON encoded
@new_deployment = { "namespace": "rs",
                    "type":      "deployments",
                    "fields":    { "name": to_json("my_deployment") } }

# Now provision deployment
provision(@new_deployment)

# @new_deployment now contains a resource collection and can be
# used to execute actions (see Resource Actions below)
@new_deployment.update("deployment": { "name": "new_name" })

# The definition below takes a declaration as parameter
define create_deployment(@deployment) do
  # Retrieve JSON object backing declaration
  $json = to_object(@deployment)
  # Manipulate JSON directly to change declaration
  $json["fields"]["name"] = "some_other_name"
  # Assign the JSON objet back to a declaration
  @new_deployment = $json
  # Now provision deployment
  provision(@new_deployment)
end

Resource Actions

Resource actions allow a cloud workflow to act on resources: scripts can be run on running servers, instances can be rebooted etc. Actions may accept zero or more argument(s). For example, the terminate() action takes no argument, the run_executable() action requires an executable name and inputs etc.

Arguments values are written using the JSON syntax already covered in the Resource Fields section above and all arguments are specified by name. For example, the run_executable() action requires two arguments: recipe_name is the name of the recipe to run, and inputs is the list of inputs to be used for the recipe execution. The following two expressions are equivalent invocations of run_executable():

# Argument names must be explicitly given
@instances.run_executable(recipe_name: "lb::setup_lb", inputs: { "lb/applistener_name": "text:my_app" })

# But order does not matter
@instances.run_executable(inputs: { "lb/applistener_name": "text:my_app" }, recipe_name: "lb::setup_lb") 

As a convenience, it is also possible to pass multiple arguments as a single hash value where each key corresponds to the argument name and each value to the corresponding argument value. So the request to execute the run_executable() action above is also equivalent to:

# Arguments can be given as a single hash value
@instances.run_executable({ "recipe_name": "lb::setup_lb", "inputs": { "lb/applistener_name": "text:my_app" } })

This makes it convenient to call actions passing in variables that were retrieved programmatically.

Built-in Actions

All resource collections expose the following actions on top of the actions supported by the resource type:

• get(): Refresh resource fields
• update(): updates fields of the underlying resources as supported by the resource type
• destroy(): destroys all resources in the collection
• exist?(): checks whether each resource in the collection still exists or was deleted
• request(): calls an action by name, useful to call update() and destroy() as implemented by the underlying resource type if it exposes such action. This is only required if there is a clash between the resource type action name and the built-in action names and is not used for resources in the RightScale namespace (where there is no such clash).

@deployment.update(deployment: { "name": "A new name" })

@resource.request(action: "update", arg_1: arg1_value, ...)

Resource Type Actions

So far we have seen the various ways that a cloud workflow may interact with resource collections:

• Actions may be called on collections using action_name(argument: "argument_value", ...)
• Fields may be retrieved using :field_name
• Links may be followed using link_name(argument: "argument_value", ...)
   

Cloud workflows may also interact with resource types to locate resources, create new resources, and execute actions that apply to a resource type rather than specific resources. For example, the RightScale instances resource type exposes actions to terminate or run scripts on multiple instances at once. The syntax to designate a resource type is:

<namespace>.<resource type>

For example:

rs.clouds.get(href: "/api/clouds/1234").instances

The expression above designate the instances resource type of the rs namespace. Resource types support the following built-in actions:

• get() is the resource type action used to locate resources. See Locating Resources below.
• create() is used to create new resources.
• empty() returns an empty resource collection of the given resource type.
• request() is used to call a custom action whose name is get, create, empty, or request. The first argument is the name of the action and the following arguments match the arguments expected by the custom action. This action is merely to work around name clashes where a resource type would expose an action that has the same name as one of the built-in actions.

Note: There is no resource type custom action in the rs namespace named 'get', 'create' , 'empty' or 'request' and 'request' is thus never needed for resource types in that namespace.

Resource types may expose custom actions on top of these built-in actions. For example, running a recipe on multiple instances can be done using: 

rs.clouds.get(href: "/api/clouds/1234").instances.multi_run_executable(recipe_name: "lb::setup_load_balancer", filter: ["name==front_end"])

While using multi_run_executable() is functionally equivalent to using something like @instances.run_executable(), where @instances was initialized with the same set of resources, the two expressions are executed differently by the engine. In the former case, the engine makes a single call to the RightScale API which dispatches the script or recipe on all instances concurrently. In the latter case, the engine iterates through all the instances in the collection and makes one API call for each to dispatch the script or recipe.

The Mapping RightScale Resources to the APIs section describes how to find the available RightScale resource type actions and arguments using the RightScale API documentation.

Creating Resources

The create() resource type action allows creating new resources. For example, creating a new RightScale deployment can be done with:

rs.deployments.create(deployment: { "name": "New deployment" })

The arguments used by the create() action are resource type specific. The convention used by RightScale resources is to use a single hash argument whose name is the singular resource type name, the keys of the hash are the field names and the values are the field values. Note that contrary to the provision() function the create() action makes a single API call to create the resource (i.e. a CREATE API request). There is no logic around launching a server and waiting for it to become operational for example.

Locating Resources

There are a few ways that resources can be located: by type using the get() action on the resource type, by resource href using the get() action on the namespace, or by following links.

Note: The tags resource type in the rs namespace exposes a by_tag() action which can be used to locate RightScale resources using their tags as well.

rs.servers.get()

rs.servers.get(filter: ["name==front_end"])

rs.get(href: "/api/instances/123", view: "full")
rs.get(href: ["/api/instances/123", "/api/instances/124"], view: "full")

@security_groups = rs.security_groups(filter: ["name==default"])

@template = rs.clouds.get(href: "/api/clouds/1234").instances.get(filter: ["name==frontend"]).server_template()

@clone = rs.clouds.get(href: "/api/clouds/1234").instances.get(filter: ["name==frontend"]).server_template()
                           .clone(server_template: { "name": "New name" })

@images = rs.clouds.get(href: "/api/clouds/1234").instances.get(filter: ["name==frontend"]).server_template()
                           .clone(server_template: { "name": "New name" }).multi_cloud_images()

Resource Collection Management

Operators and functions allow managing resource collections in various ways. Collections can be merged, extracted, or tested for inclusion.

Collections can be concatenated using the "+" operator:

@all_servers = @servers1 + @servers2

Elements of a collection can be removed from another collection using the "-" operator:

@servers1 = @all_servers - @servers2

Note: All instances of all resources in @servers2 are removed from @all_servers.

Finally, it is possible to test whether all elements of a collection are contained in another using the "<" and ">" operators:

@all_servers = @servers1 + @servers2
@all_servers > @servers1 # true
@servers1 < @all_servers # true

Cloud workflows can identify specific resources from resource collections in various ways. Items can be extracted by index or range using the bracket operator ([ ]). A range consists of two integers separated by two consecutive dots and is inclusive, like the following:

@servers = rs.get(href: "/api/deployments/123").servers()
@first_server = @servers[0]
@more_servers = @servers[1..3]

The second line evaluates to a collection consisting of a single element: the first server of deployment 123. The last line evaluates to a collection containing the second, third, and fourth servers in that deployment. The lower and upper bounds of the range are optional. If no lower bound is specified then the range starts at the first resource in the collection. If no upper bound is specified, then the range ends at the last resource of the collection.

Note: All indices are zero-based.

For example:

@all_other_servers = @servers[1..]

The code above initializes a collection made of all servers in the initial collection except the first.

No error is raised if the specified bounds exceed the number of resources in the collection. Instead, only the resources that are in the specified range are returned:

@servers[42..]

The example above returns the empty collection if the @servers collection has fewer than 43 resources.

While functions are covered in a different section, it is worth mentioning the size(), select() and detect() functions at this point:

• The size() function returns the number of resources in a collection.
• The select() function traverses a given collection and extracts (selects) all the resources whose fields values match the values given in the second argument (in the form of a hash).
• The detect() function stops as soon as one matching resource is found (and thus the resulting collection is made of a single resource).

The first argument for both these functions is the collection to be traversed while the second argument is a hash of field name and corresponding value. The value may be a regular expression in which case the field must be of type string and its value must match the given regular expression for the resource to be selected. The syntax used to write regular expressions is:

"/pattern/modifiers"

where pattern is the regular expression itself, and modifiers are a series of characters indicating various options. The modifiers part is optional. This syntax is borrowed from Ruby (which borrows it from Perl). The following modifiers are supported:

• /i makes the regex match case insensitive.
• /m makes the dot match newlines.
• /x ignores whitespace between regex tokens.

You can combine multiple modifiers by stringing them together as in /regex/im.

For example:

@servers = rs.get(href: "/api/deployments/123").servers()
@app_servers = select(@servers, { "name": "/^app_server.*/i" })

The above snippet initializes @app_servers with the collection of servers in the deployment with href deployments/123 whose name fields start with app_server using a non case sensitive comparison.

Passing Resource Collections to the APIs

Using the RightScale APIs to launch a cloud workflow requires giving the expected inputs. Inputs can be variables or resource collections. Variables are specified using the JSON notation. Resource collections are described using a JSON object that contains three fields:

• The field 'namespace' contains the namespace for the resources in the collection. For example, rs
• The field 'type' contains the resource type for all resources in the collection. For example, servers
• Finally the field 'hrefs' is an array of all the resources hrefs. For example, ["/api/servers/123", "/api/servers/234"]

 define launch(@servers, $timeout)
 ...
 end

{
  "@servers": { "namespace": "rs", "type": "servers", "hrefs": ["/api/servers/234"] },
  "$timeout": 10
}