|Table of Contents||Sections|
Some statements can be adorned with attributes that affects their behavior. Attributes appear after the statement on the same line right before the do keyword for expressions that have one (e.g. define, sub, concurrent, map, foreach). There can be multiple attributes specified on a single statement in which case they are separated with commas. Attributes allow specifying and handling timeouts and handling errors and cancelations.
An attribute has a name and a value. The syntax is:
The acceptable types for attribute values depend on the attribute: they may be numbers (e.g. wait_task: 1), strings (e.g. on_timeout: skip), arrays (e.g. wait_task: [ "this_task", "that_task" ]), or definition names with arguments (e.g. on_timeout: handle_timeout()).
Some of the attributes define behavior that apply to tasks, while others define behavior for the whole process. Processes and tasks are described in detail in the Cloud Workflow Processes section. For the purpose of understanding the attributes behavior described below, it is enough to know that a single process consists of one or more tasks. A task is a single thread of execution.
Some attributes attach behavior to the expression they adorn and all their sub-expressions. The sub-expressions of an expression are expressions that belong to the block defined by the parent expression. Not all expressions define blocks so not all expressions have sub-expressions. Expressions that may have sub-expressions include define, sub, concurrent and the looping expressions (foreach, concurrent foreach etc.).
The exhaustive list of all attributes supported by the language are listed below in alphabetical order:
|Attribute||Applies to||Possible Values||Description|
define, sub, call
name of the definition with arguments, skip or retry
Behavior to trigger when an error occurs in an expression or a sub-expression.
define, sub, call, concurrent,
name of the definition with arguments
Name of the definition called when an expression causes a rollback (due to an error or the task being canceled).
sub, concurrent, call
name of the definition with arguments, skip or retry
Behavior to trigger when a timeout occurs in an expression or a sub-expression.
string representing the name of a task
Change current task name to given value.
concurrent foreach, concurrent map
string representing the prefix of task names
Specifies the prefix of names of tasks created by concurrent loop (suffix is iteration index).
string representing a duration
Value defines the maximum time allowed for expressions in a statement and any sub-expression to execute.
concurrent, concurrent foreach, concurrent map
number of tasks to be waited on or name(s) of task(s) to be waited on
Pause the execution of a task until the condition defined by a value is met
|task_label||sub, call, define||string representing label||Labels allow processes to return progress information to clients. They are associated with an arbitrary name that gets returned by the cloud workflow APIs.|
The task_name, task_prefix and wait_task attributes are described in Cloud Workflow Processes. This section describes in detail the other attributes dealing with errors and timeouts.
Defining the steps involved in handling error cases is an integral part of all cloud workflows. This is another area where workflows and traditional programs differ: a workflow needs to describe the steps taken when errors occur the same way it describes the steps taken in the normal flow. Error handlers are thus first class citizen in RCL and are implemented as definitions themselves. Handling an error could mean alerting someone, cleaning up resources, triggering another workflow, etc. RCL makes it possible to do any of these things through the on_error attribute. Only the define, sub and concurrent expressions may be adorned with that attribute.
The associated value is a string that can be any of the following:
retry: retries the execution of the statement and any sub-expression.
To illustrate the behavior associated with the different values consider the following snippet:
The engine generates an exception when the raise expression executes. This exception causes the parent expression on_error attribute to execute. The associated value is skip which means ignore the error and proceed to run the first expression after the block. The engine then proceeds to the next expression after the block (the log_info expression). If the attribute value associated with the on_error handler had been retry instead, then the engine would have proceeded to re-run the block (which in this case would result in an infinite loop).
As mentioned in the introduction, a cloud workflow may need to define additional steps that need to be executed in case of errors. The on_error attribute allows specifying a definition that gets run when an error occurs. The syntax allows for passing arguments to the definition so that the error handler can be provided with contextual information upon invocation. On top of arguments being passed explicitly, the error handler also has access to all the variables and references that were defined in the scope of the expression that raised the error.
The error handler can stipulate how the caller should behave once it completes by assigning one of the string values listed above (skip or retry) to the special $_error_behavior local variable. If the error definition does not define $_error_behavior, then the caller uses the default behavior (raise) after the error definition completes. This default behavior causes the error to be re-raised so that any error handler defined on a parent scope may handle it. If no error handler is defined or all error handlers end-up re-raising then the task terminates and its final status is failed.
The following example shows how to implement a limited number of retries using an error handler:
Errors can originate from evaluating expressions (e.g. division by 0) or from executing resource actions (e.g. trying to launch an already running server). A variation on the former are errors generated intentionally using the raise keyword. In all these cases the most inner error handler defined using the on_error attribute gets executed.
The raise keyword optionally followed with a message causes an error which can be caught by an error handler. All error handlers have access to a special variable that contains information about the error being raised. $_error is a hash that contains three keys:
Resource actions always operate atomically on resource collections, in other words the expression @servers.launch() is semantically equivalent to making concurrent launch API calls to all resources in the @servers array. This means that multiple errors may happen concurrently if multiple resources in the collection fail to run the action. When that happens an error handler needs to have access to the set of resources that failed as well as the set that succeeded and the initial collection to take the appropriate actions. We have already seen the special $_error variable made available to error handlers in case of an error resulting from calling an action on a resource collection. RCL also makes available the following variables to the error handler:
The $_errors variable contains an array of hashes. Each element includes the following values:
In case of resource action errors the $_error variable is initialized with the type "resource_action" and includes the detailed error message with the problem, summary, and resolution fields as a string.
Given the above, the following definition implements a retry:
define handle_terminate_error do foreach $error in $_errors do @instance = rs.get($error["resource_href"]) # Retrieve the instance that failed to terminate if @instance.state != "stopped" # Make sure it is still running log_error("Instance " + @instance.name + " failed to terminate, retrying...") sub on_error: skip do @instance.terminate() # If so try again to terminate but this time ignore any error end end end $_error_behavior = "skip" # Proceed with the next statement in caller end sub on_error: handle_terminate_error() do @instances.terminate() end
In the definition above the error handler sets the special $_error_behavior local variable to "skip" which means that the process will proceed to the next statement after the call to terminate() even if there is an error terminating some of the instances. Note how the handler itself uses on_error to catch errors and ignore them (using skip).
Actions may return nothing, collection of resources, or array of values. In the case an action has a return value (collection or array), the error handler needs to be able to modify that value before it gets returned as a result of the action expression. For example, an error handler may retry certain actions and as a result may need to add to the returned value which would initially only contain values for the resources that ran the action successfully. An error handler can achieve this by updating the @_partial collection or the $_partial array, the content of this reference or variable at the end of the execution of the handler is what gets returned by the action call.
To take a concrete example let's consider the RightScale servers resource launch() action. This action returns a collection of launched instances. The following handler retries any failure to launch and updates the @_partial collection with instances that successfully launched on retry:
The definition above adds any instance that is successfully launched in the retry to the @_partial collection thereby making sure that they get assigned to @instances as result of the launch() action.
We've seen before that definitions executed via call only have access to the references and variables passed as argument (and global references and variables). Definitions executed through handlers, on the other hand, inherit from all the local variables and references defined at the time the handler is invoked (so at the time an exception is thrown, a timeout occurs or a cancelation is triggered).
In the snippet above, the error handler has access to @servers even though that collection is defined in the main scope (the various log_xxx() functions allow for appending messages to process logs and the inspect() function produces a human friendly string representation of the object it is given).
The timeout and on_timeout attributes allow setting time limits on the execution of expressions and specifying the behavior when a time limit is reached respectively:
The block in the snippet above must execute in less than 30 minutes otherwise its execution is canceled and the handle_launch_timeout definition is executed. Timeout values can be suffixed with d, h, m or s (respectively day, hour, minute or second).
Note that there does not need to be a on_timeout associated with all timeout attributes. Instead the most inner expression that includes a on_timeout attribute gets triggered when a timeout occurs:
In the snippet above, inner_handler gets executed if the sleep_until function takes more than 10 minutes to return.
Similar to the on_error attribute, the on_timeout attribute can be followed by a definition name or one of the behaviors values (skip or retry).
Note: Using the raise behavior in an on_timeout attribute will cause the next on_timeout handler to be executed. Timeouts never cause error handlers to be executed and vice-versa.
On top of specifying the behavior directly in the on_timeout attribute as in:
It's also possible for a definition handling the timeout to specify what the behavior should be by setting the $_timeout_behavior local variable:
Finally, the timeout handler may accept arguments that can be specified with the on_timeout attribute. The values of the references and variables at the point when the timeout occurs are given to the handler:
The snippet above will cause the handle_timeout definition to execute three times. The third times $retries is equal to 3, the timeout handler definition sets $_timeout_handler to skip and the block is canceled.
The task_label attribute is used to report progress information to clients. It does not affect the execution of the process and is simply a way to report what it is currently doing. The label attribute can be used on sub and call:
As shown in the snippet above RCL has built-in support for logging which helps troubleshoot and develop cloud workflows. Each process is associated with a unique log that is automatically created on launch. Logging is done using the log_title(), log_info() and log_error() functions:
Logs for a process can be retrieved using the RightScale API or through the RightScale dashboard by looking at the process audit entries.
We have seen how a cloud workflow may use attributes to annotate statements and defining additional behaviors. Attributes apply to the statement they adorned and some also apply to its sub-expressions. Definitions can be written to handle errors, timeouts and cancelation. Definitions handling errors that occur during resource action execution have access to all the underlying low level errors and can modify the return value of the action.
|RCL||Resources||Cloud Workflows & Definitions||Variables||► Attributes & Error Handling||Branching & Looping||Processes||Functions||Operators||Mapping|
© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.