Note: Please go to docs.rightscale.com to access the current RightScale documentation set. Also, feel free to Chat with us!
Home > ServerTemplates > Infinity > ST > Storage Toolbox (v13 Infinity) > Storage Toolbox (v13 Infinity) - Runbook

Storage Toolbox (v13 Infinity) - Runbook

 

Table of Contents    

Infinity

Leading edge features

   ►   Runbook

Common Operational Tasks

These tasks can be fully utilized using clouds that have complete volume support (dynamic snapshots, multiple attachable volumes, dynamically attachable and detachable). Currently, Rackspace, SoftLayer and Google do not offer full volume support. 

Mount a Single Device

It's important that the related 'block_device' scripts are executed in the correct order to ensure that any cookbook dependencies are properly resolved. The 'block_device' cookbook's default recipe (block_device::default) must be executed before any of the other 'block_device' recipes are executed because it sets parameters that are used by other recipes in the cookbook.

Use recipes from the 'repo' cookbook to populate the mounted device with data from a software repository or ROS location.

Provide values for the inputs that are specific to the device. Use the device suffix to locate the corresponding device's inputs. (e.g. Backup Lineage (1))

Required to Set up a Device

Input Name Description Example Value
Block Device Mount Directory (1) The name of the directory where the filesystem will be mounted. (e.g. /mnt/data_storage1)  text: /mnt/data_storage1
Number of Volumes in the Stripe (1) The number of physical volumes that make up the logical volume. text:  1
Total Volume Size (1) The total size of the logical volume, which is a summation of the size of all physical volumes in the stripe. (e.g. 10) text:  10
Percentage of the LVM used for data (1) The percent of the available disk space that will be used for storing data. The remaining percent will be used for temporarily storing modified (dirty) blocks of data for LVM. text: 90

 

Recommended to Set up a Device (Optional)

Input Name Description Example Value
Nickname (1)
(Optional)
The nickname that will be used to name the attached volumes. The backup snapshots will also use the nickname as a prefix along with a timestamp (e.g. data_storage_201206291950)*  text: my_data_1

*TIP it is recommended that you replace the default nickname (data_storage1) with a more descriptive nickname, which will make it easier to find your respective volumes in a RightScale account that contains numerous volumes.*

Required for Primary Backups

If you want to take a primary backup or enable continuous primary backups, see the Take a Primary Backup section for details.

Required for Setting Backup Policy

If you want to adjust primary backup frequency, see the Modify Device Backup Policy section for details.

Mount Multiple Devices

By default, the "Storage Toolbox" ServerTemplate is configured to support up to two devices. If you want to mount more than two devices, you will need to fork the repository and update the 'block_device' cookbook to support additional devices. See Override Chef Cookbooks.

In the 'block_device' cookbook's metadata, change the code to support the number of devices to support.

# Multiple Block Devices
device_count = 2devices = 1.upto(device_count).map {|number| "device#{number}"}

See Increase the Number of Block Devices for more detailed instructions.

Take a Primary Backup

If your cloud does not have full volume support, run the following script to take a primary backup snapshot of the specified device to store in ROS. 

  • block_device::do_primary_backup - Create a backup snapshot of the device. If the device is comprised of multiple volumes, a snapshot will be created for each attached volume and tag them appropriately. So if a device consisted of two volumes, the snapshot of the first volume would have the following tags: rs_backup:count=2, rs_backup:position=1


Required Inputs

If the following inputs did not inherit appropriate values at launch time, go to the current server's Inputs tab and set the following inputs before running the script.

Input Name Description Example Value
Primary Backup Secret (default)

Valid cloud credentials may be required to take a primary backup depending on the specified cloud ('Primary Backup Storage Cloud').

  • AWS EBS Snapshots - No value/ignore
  • AWS S3 - AWS Access Secret Key
  • CloudFiles (US) - Rackspace API Key
  • CloudFiles (UK) - Rackspace API Key
  • Swift - No value/ignore
  • SoftLayer - No value/ignore
text: No value/ignore
Primary Backup User (default)

If you are saving a backup to a CloudFiles container from a Rackspace Cloud Server, it's recommended that you set this input to 'true'.

For Rackspace, specify the Rackspace username and API key to use for Rackspace Cloud Files authentication. For Amazon and other cloud providers, set these to "ignore" since authentication is not required for volume storage.

text: No value/ignore
Primary Backup Storage Cloud (default) The primary backup storage cloud. This is only used if the server's cloud does not have volume support. text: No value/ignore
Primary Backup Storage Cloud Endpoint URL (default) The endpoint URL for the primary backup storage cloud. This is used to override the default endpoint or for generic storage clouds such as Swift. text: No value/ignore
Rackspace SNET Enabled for Backup

If you are saving a backup to a CloudFiles container from a Rackspace Cloud Server, it's recommended that you set this input to 'true'.

When 'true', Rackspace internal private networking (preferred) is used for communications between servers and Rackspace Cloud Files. Ignored for all other clouds.

text:  true

Backup Lineage (1)

The lineage name that will be associated with this device. The lineage name will be used to tag backup snapshots that are created of the device. (e.g. rs_backup:lineage=dev1-lineage) The lineage tag will be used to select the correct snapshots for restoring a device.

text:  dev1-lineage

Enable Continuous Primary Backups

Before you enable continuous backups, make sure that any inputs that are required to take a primary backup have already been set and passed to the instance. Typically, these values are set before the server is launched. However, if you changed any of the inputs related to taking a primary backup or entered new values, you must first run the 'default' recipe to pass the updated input values to the instance before you run the script to enable continuous backups script.

  1. (Optional) If you want to change the values for any of the required inputs for taking a primary backup or modify the backup policy, edit the inputs under the current server's Inputs tab. For example, you may want to adjust the frequency of backups.
  2. Under the current server's Inputs tab, specify which device you are going to enable primary backups for by setting the Block Device(s) to Operate On input accordingly. List a single value (e.g. device1), a comma-separated list of devices (e.g. device1,device2) or use an asterisk (*) to run the script on all devices.
  3. If you made changes to any of the primary backup or backup policy related inputs, run the block_device::default script, otherwise go to the next step.
  4. Run the block_device::do_primary_backup_schedule_enable script to create and enable a cronjob that takes periodic (primary) backups of the device.

Disable Continuous Primary Backups

  1. Under the current server's Inputs tab, specify which device you are going to disable primary backups for by setting the Block Device(s) to Operate On input accordingly. List a single value (e.g. device1), a comma-separated list of devices (e.g. device1,device2) or use an asterisk (*) to run the script on all devices.
  2. If you made changes to any of the primary backup inputs, run the block_device::default script, otherwise go to the next step.
  3. Run the block_device::do_primary_backup_schedule_disable script to disable (stop) continuous backups for the specified device(s).

Take a Secondary Backup

Run the following script to take a secondary backup of the specified device. 

  • block_device::do_secondary_backup - Create a backup of the device to a ROS location. You will need to specify a bucket or container name and valid cloud credentials.


Required Inputs

If the following inputs did not inherit appropriate values at launch time, go to the current server's Inputs tab and set the following inputs before running the script.

Input Name Description Example Value
Secondary Backup Storage Cloud (default)

Select the cloud provider or service where the secondary backup will be stored.

  • s3 - Amazon S3
  • cloudfiles - RackspaceCloudFiles (US)
  • cloudfilesuk - RackspaceCloudFiles (UK)
  • swift - OpenStack
  • SoftLayer_Dallas - SoftLayer Dallas 
  • SoftLayer_Singapore - SoftLayer Singapore
  • SoftLayer_Amsterdam - SoftLayer Amsterdam
text:  s3
Secondary Backup Secret (default) 

Valid cloud credentials are required to save a secondary backup in an ROS location.

  • s3 - Amazon Secret Access Key
  • cloudfiles - Rackspace API Key (US)
  • cloudfilesuk - Rackspace API Key (UK)
  • swift - OpenStack Password
  • SoftLayer_Dallas - SoftLayer Secret Access Key
  • SoftLayer_Singapore - SoftLayer Secret Access Key
  • SoftLayer_Amsterdam - SoftLayer Secret Access Key
cred:  AWS_SECRET_ACCESS_KEY
Secondary Backup User (default)

Valid cloud credentials are required to save a secondary backup in an ROS location.

  • s3 - Amazon Access Key ID
  • cloudfiles - Rackspace Username (US)
  • cloudfilesuk - Rackspace Username (UK)
  • swift - OpenStack Username
  • SoftLayer_Dallas - SoftLayer Access Key ID
  • SoftLayer_Singapore - SoftLayer Access Key ID
  • SoftLayer_Amsterdam - SoftLayer Access Key ID
cred:  AWS_ACCESS_KEY_ID
Secondary Backup Storage Container (1)

Enter the name of the ROS container where the backup will be stored. For Amazon S3, enter the bucket name. For Rackspace CloudFiles, enter the container name.

text:  my-container
Secondary Backup Storage Cloud Endpoint URL (default) If you are using a generic cloud storage option like Swift, you must manually specify the cloud provider's endpoint URL. This input is only required if you select 'swift' for the 'Secondary Backup Storage Cloud (default)' input. Rackspace and SoftLayer both offer Swift-based cloud storage services.To use Rackspace CloudFiles, specify the cloud's API endpoint. (e.g. https://lon.auth.api.rackspacecloud.com/v1.0) text: https://lon.auth.api.rackspacecloud.com/v1.0
Rackspace SNET Enabled for Backup

If you are saving a backup to a CloudFiles container from a Rackspace Cloud Server, it's recommended that you set this input to 'true'.

When 'true', Rackspace internal private networking (preferred) is used for communications between servers and Rackspace Cloud Files. Ignored for all other clouds.

text:  true
Block Device(s) to Operate On

Specify for which device the secondary backup will be created.

  • device1
  • device2
text:  device1

Restore a Device from a Primary Backup

If there is an existing primary backup available, you can restore a device from the backup. By default, the most recently completed primary backup will be used for a device restoration. However, if you want to use a different backup that has a different lineage or an older backup, use the timestamp and lineage overrides.

  1. Under the current server's Inputs tab, set the inputs below accordingly.
  2. Run the block_device::do_primary_restore script.

 

Input Name Description Example Value
Backup Lineage (1) The name associated with your primary and secondary database backups. It's used to associate them with your database environment for maintenance, restore, and replication purposes. Backup snapshots will automatically be tagged with this value (e.g. rs_backup:lineage=mysqlbackup). Backups are identified by their lineage name. Note: For servers running on Rackspace, this value also indicates the Cloud Files container to use for storing primary backups. If a Cloud Files container with this name does not already exist, one will automatically be created. text:  mysqlbackup
Rackspace SNET Enabled for Backup

If you are retrieving a backup from a CloudFiles container to a Rackspace Cloud Server, it's recommended that you set this input to 'true' to retrieve the backup faster.

When 'true', Rackspace internal private networking (preferred) is used for communications between servers and Rackspace Cloud Files. Ignored for all other clouds.

text:  true

Backup Lineage Override

Specify an EBS nickname as the lineage override. If defined, this will override the input defined for 'Backup Lineage' (block_device/devices/device2/backup/lineage) so that you can restore the volume from another backup that has as a different lineage name. The most recently completed snapshots will be used unless a specific timestamp value is specified for 'Restore Timestamp Override' (block_device/devices/device2/backup/timestamp_override).

text:  otherlineage
Backup Restore Timestamp Override (1) Another optional variable to restore from a specific timestamp. Specify a string matching the timestamp tags on the volume snapshot set. You will need to specify the timestamp that's defined by the snapshot's tag (not name). This is only used with AWS. For example, if the snapshot's tag is 'rs_backup:timestamp=1303613371' you would specify '1303613371' for this input. text:  1303613371
Block Device(s) to Operate On

Specify which device to restore.

  • device1
  • device2
text:  device1

 

Restore a Device from a Secondary Backup

If there is an existing secondary backup available, you can restore a device from the backup. By default, the most recently completed secondary backup will be used for a device restoration. However, if you want to use a different backup that has a different lineage or an older backup, use the timestamp and lineage overrides.

  1. Under the current server's Inputs tab, set the inputs below accordingly.
  2. Run the block_device::do_secondary_restore script.

 

Input Name Description Example Value
Secondary Backup Storage Cloud (default)

Select the cloud provider or service where the secondary backup will be restored from.

  • s3 - Amazon S3
  • cloudfiles - Rackspace CloudFiles (US)
  • cloudfilesuk - Rackspace CloudFiles (UK)
  • swift - OpenStack
  • SoftLayer_Dallas - SoftLayer Dallas 
  • SoftLayer_Singapore - SoftLayer Singapore
  • SoftLayer_Amsterdam - SoftLayer Amsterdam
text:  s3
Secondary Backup Secret (default)

Valid cloud credentials are required to retrieve a secondary backup from an ROS location.

  • s3 - Amazon Secret Access Key
  • cloudfiles - Rackspace API Key (US)
  • cloudfilesuk - Rackspace API Key (UK)
  • swift - OpenStack Password
  • SoftLayer_Dallas - SoftLayer Secret Access Key
  • SoftLayer_Singapore - SoftLayer Secret Access Key
  • SoftLayer_Amsterdam - SoftLayer Secret Access Key
cred:  AWS_SECRET_ACCESS_KEY
Secondary Backup User (default)

Valid cloud credentials are required to retrieve a secondary backup from an ROS location.

  • s3 - Amazon Access Key ID
  • cloudfiles - Rackspace Username (US)
  • cloudfilesuk - Rackspace Username (UK)
  • swift - OpenStack Username
  • SoftLayer_Dallas - SoftLayer Access Key ID
  • SoftLayer_Singapore - SoftLayer Access Key ID
  • SoftLayer_Amsterdam - SoftLayer Access Key ID
cred:  AWS_ACCESS_KEY_ID
Backup Lineage (1) The name associated with your primary and secondary database backups. It's used to associate them with your database environment for maintenance, restore, and replication purposes. Backup snapshots will automatically be tagged with this value (e.g. rs_backup:lineage=mysqlbackup). Backups are identified by their lineage name. Note: For servers running on Rackspace, this value also indicates the Cloud Files container to use for storing primary backups. If a Cloud Files container with this name does not already exist, one will automatically be created. text:  mysqlbackup
Secondary Backup Storage Container (1) Enter the name of the ROS container where the backup will be stored. For Amazon S3, enter the bucket name. For Rackspace CloudFiles, enter the container name. text:  my-container
Rackspace SNET Enabled for Backup

If you are retrieving a backup from a CloudFiles container to a Rackspace Cloud Server, it's recommended that you set this input to 'true' to retrieve the backup faster.

When 'true', Rackspace internal private networking (preferred) is used for communications between servers and Rackspace Cloud Files. Ignored for all other clouds.

text:  true
Secondary Backup Storage Cloud Endpoint URL (default) If you are using a generic cloud storage option like Swift, you must manually specify the cloud provider's endpoint URL. This input is only required if you select 'swift' for the 'Secondary Backup Storage Cloud (default)' input. Rackspace and SoftLayer both offer Swift-based cloud storage services.To use Rackspace CloudFiles, specify the cloud's API endpoint. (e.g. https://lon.auth.api.rackspacecloud.com/v1.0) text: https://lon.auth.api.rackspacecloud.com/v1.0

Backup Lineage Override

If defined, this will override the input defined for 'Backup Lineage' (block_device/devices/device2/backup/lineage) so that you can restore the volume from another backup that has as a different lineage name. The most recently completed snapshots will be used unless a specific timestamp value is specified for 'Restore Timestamp Override' (block_device/devices/device2/backup/timestamp_override). text:  otherlineage
Backup Restore Timestamp Override (1) Another optional variable to restore from a specific timestamp. Specify a string matching the timestamp tags on the volume snapshot set. You will need to specify the timestamp that's defined by the snapshot's tag (not name). This is only used with AWS. For example, if the snapshot's tag is 'rs_backup:timestamp=1303613371' you would specify '1303613371' for this input. text:  1303613371
Block Device(s) to Operate On

Specify which device to restore.

  • device1
  • device2
text:  device1

Terminate Server and Delete a Device

If you want to terminate a server that contains an attached device, you should use the 'block_device::do_delete_volumes_and_terminate_server' script to perform that operation instead of using the standard 'Terminate' action button in the Dashboard so that any attached devices will be properly deleted. All attached volumes of the specified device will be detached and deleted. If you terminate a server by using the standard 'Terminate' action button, the instance will be terminated, but any previously attached volumes will persist and not be deleted.

Before you can successfully run the script, you must first remove the default safety setting that prevents the script from being inappropriately executed.  Set the following inputs under the "current" server's Inputs tab.

Input Name Description Example Value
Block Device(s) to Operate On The block device(s) to operate unmount from the device. You can specify a comma-separated list of device names or use a wildcard (*) to indicate all devices. Example: device1 text: device1
Terminate Safety Prevents the accidental running of the 'block_device::do_delete_volumes_and_terminate_server' script. Click the "Override" checkbox and change the input to 'text:off' to allow the execution of the script. text: off


Run the following script to unmount the specified device.

  • block_device::do_delete_volumes_and_terminate_server - Unmount and delete the specified block device (device 1 or 2). All attached volumes of the specified device will be detached and deleted. Terminate the current running server. WARNING! Execution of this script will delete any data on your block device! Any data not saved in a backup will be lost.

 

If you're using more than one block device, and you want to operate unmount all of your block devices, you must use * for your input value

Reset a Device

Unmount and delete the attached block device(s) for the defined lineage. Designed for test and development purposes only.

Warning! Execution of this script will delete any data on your block device!"

Before you can successfully run the script, you must first remove the default safety setting that prevents the script from being inappropriately executed.  Set the following inputs under the "current" server's Inputs tab.

Input Name Description Example Value
Block Device(s) to Operate On The block device(s) to operate unmount from the device. You can specify a comma-separated list of device names or use a wildcard (*) to indicate all devices. Example: device1 text: device1
Terminate Safety Prevents the accidental running of the 'block_device::do_delete_volumes_and_terminate_server' script. Click the "Override" checkbox and change the input to 'text:off' to allow the execution of the script. text: off

 

Run the following operational script to unmount the specified device.

  • block_device::do_force_reset - Unmount and delete the specified block device (device 1 or 2). All attached volumes of the specified device will be detached and deleted. Designed for test and development purposes only. WARNING! Execution of this script will delete any data on your block device! Any data not saved in a backup will be lost.

 

Modify Device Backup Policy

Use the following inputs to control the backup policy for a device. The inputs that apply to 'device1' are listed below. A duplicate set of inputs are available for 'device2' which are highlighted with a (2) suffix in the namespace. Typically, you will set the values for the backup policy inputs before servers are launched. 

To change the backup policy on a current running server, you must perform the following steps:

  1. Modify the inputs under the "current" server's Inputs tab.
  2. Run the block_device::do_primary_backup_schedule_disable script.
  3. Run the block_device::default script.
  4. Run the block_device::do_primary_backup_schedule_enable script.
Input Name Description Example Value
Backup Cron Hour (1) Defines the hour when the backup will be taken (in crontab format). Use a value of 1-24, or set to 'Ignore' to create a backup every hour. To take multiple backups during the day, use a comma-separated list. For example, specify 11,23 to create backups at 11:00 AM and 11:00 PM. text:11, 23
Backup Cron Minute (1) Defines the minute of the hour when the backup will be taken (in crontab format). Use a value of 1-59, or set to 'Ignore' and a random minute will be calculated. text:15
Keep Daily Backups (1) The number of daily backups to keep (i.e. rotation size). (Default is 14.) A daily backup is the last completed backup snapshot of the day with a timestamp closest to the end of the day (23:59:59). text:14
Backup Max Snapshots (1) The maximum number of backups to keep in addition to those being rotated. (Default is 60.) text:60
Keep Monthly Backups (1) The number of monthly backups to keep. (Default is 12.) A monthly backup is the last completed backup snapshot for a given month, with a date and timestamp closest to 23:59:59 on the last day of the month. text:12
Keep Weekly Backups (1) The number of monthly backups to keep. (Default is 6.) A weekly backup is the last completed backup snapshot for a given week, with a date and timestamp closest to 23:59:59 on Saturday. text:6
Keep Yearly Backups (1) The number of monthly backups to keep. (Default is 2.) A yearly backup is the last completed backup snapshot for a given year, with a date and timestamp closest to 23:59:59 on December 31. text:2
You must to post a comment.
Last modified
11:11, 26 Sep 2014

Tags

Classifications

This page has no classifications.

Announcements

None


© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.