This page describes all of the scripts, inputs, and operational procedures required to run the solution.
Table of Contents
Description: Creates a Deployment which is set up for use as a Univa Grid Engine (UGE) cluster. Each independent Deployment created by this Macro can be operated as an independent cluster.
How to Use:
To begin, subscribe to the Macro (Design -> MultiCloud Marketplace -> Macros). Once you've got a copy of the Macro in your local collection you can run it.
IMPORTANT NOTE! When asked if you wish to re-use a Security Group and/or SSH Key, we normally recommend leaving them blank and proceeding without specifying any existing group or key! The macro will proceed to prompt you to create a New security group and SSH key with appropriate values.
It is often NOT a good idea to re-use these items in order to maintain independent clusters. If you are going to be re-using an existing key or security group, please select the proper values and ENSURE that they are correctly defined, otherwise you may run into problems.
Click on the ‘Run’ button to begin executing the Macro. You will be prompted for several items which define your cluster. You may accept the default values provided, or change them if you wish. You will be asked to specify:
Instance types affect both the processing power and cost associated with your cluster. In general, your queue master can be a less-powerful instance type, and your execution hosts should be either the same or a more powerful instance type. If you are creating very large clusters (more than 20 nodes), consider using a more powerful instance type for your queue master to compensate.
When the macro completes, your new deployment will consist of a server called “Queue Master” along with individually-numbered “Execution Host” servers. Congratulations! Your new UGE cluster is ready to go.
Once the Macro completes, you will have a new UGE cluster Deployment.
To start your new cluster, go to the Deployment in your RightScale dashboard (Manage -> Deployments).
All inputs needed by the scripts are pre-populated with the proper credentials and information. This information is automatically configured for your Deployment. You can simply accept these values as-is and click “Launch” on the Launch Request screen.
To increase the number of Servers in your cluster, simply clone an existing Execution Host server. To reduce the number of servers in your cluster, either terminate and delete an Execution Host server, or start less than all of the Execution Hosts.
Note: You should not clone the Queue Master server or change its nickname.
The following RightScripts are used to bring servers into an operational state. These scripts are not meant to be run manually, as they perform software installation and configuration.
Description: Installs the S3FS filesystem software on a server. The FUSE software distributed with CentOS is replaced with a newer version (required to support S3FS). S3FS allows you to mount an S3 bucket as a filesystem, which in turn means you don’t have to change applications to store output data on S3 storage.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Description: Installs core Tortuga software on a server. The Tortuga software is part of the UniCloud management software and is required to manage a UGE cluster.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Description: Installs and configures the Univa Grid Engine software on a cluster.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Description: Configures a compute node server for use by the UGE software.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Description: Installs and configures the Univa Grid Engine software on cluster.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Description: Installs and configures the UniCloud management software on the cluster.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Currently none. TBD.
The following RightScripts are used when servers are decommissioned. They are not designed to be run manually.
Description: Gracefully removes a cluster resource so it is no longer used by UGE.
Required Inputs and Default Settings:
AWS credentials are only used to read files from S3 and are not stored after the script exits.
Once you have created your HPC cluster, you are ready to submit basic jobs immediately, or alternatively, you can install custom applications and load data into the cloud.
UGE comes with several 'sample' jobs you can use to test your cluster's operation. This example uses the simple.sh job, which simply sleeps for a brief period before exiting, and requires no additional software or configuration.
UGE is capable of running any application or script you have created. There is no 'standard' application installed with UGE because it is a framework. You submit a job to run your application the same way you would run it if starting it at a command-line prompt. You are able to run scripts, including shell scripts, or binary executables.
If your application is available in a single-file archive, you may be able to use a RightScript which has been provided by RightScale to install it automatically.
Note that your application may not be distributed as a single compressed file, or may have a non-standard format which cannot be installed by the standard RightScript. In this case, you will most likely want to write a custom RightScript to install your application so that it is available to you each time you start a new cluster.
© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.