Note: If you cannot find a topic, go to docs.rightscale.com where all RightScale documentation will soon be located. Also, feel free to Chat with us!
Home > Partners > Couchbase > ST and Runbooks > Couchbase Enterprise Edition ServerTemplate

Couchbase Enterprise Edition ServerTemplate

The ServerTemplate has numerous inputs that are used to configure the instance when it's launched. Many of these inputs have default values that can be modified, if desired. Note that the same input may be used by more than one script and in some cases will take on a slightly different meaning for that script. Also, some features are accessible only via the Couchbase UI which can be accessed by pointing your browser at one of your Couchbase nodes at http://<host>:8091 (make sure you have the security groups setup properly to allow access to this port)</host>

Table of Contents

  1. Description and Inputs
    1. Boot Scripts
      1. RightScript: SYS Timezone set v1
      2. RightScript: SYS Monitoring install v9
      3. RightScript: SYS lvm on /mnt v9
      4. RightScript: SYS Add Swap Partition
      5. RightScript:  RB rubygems 1.3.1 + quickinstall v4
      6. RightScript:  EBS stripe Rightscale tools install v2
      7. RightScript:  Couchbase Server - EBS stripe volume create v2
      8. RightScript:  Couchbase Server - EBS stripe volume restore v2
      9. RightScript:  Couchbase Server - EBS create backup scripts v2
      10. RightScript:  Couchbase Server - EBS continuous backups v2
      11. RightScript:  Couchbase - Enterprise Server Install
      12. RightScript:  Couchbase Server - Enterprise Edition EBS Setup
      13. RightScript:  Couchbase Server - DNS Setup
      14. RightScript:  Couchbase Server - Join Cluster
      15. RightScript:  Couchbase Server - Delay Decommission
    2. Operational Scripts
      1. RightScript:  EBS unfreeze volume backups
      2. RightScript: EBS freeze volume backups
      3. RightScript:  EBS stripe volume backup v2
    3. Decommission Scripts
  2. Common Runbook Operations
    1. Security Group Settings
    2. Setting up Your First Node
    3. Adding Nodes to a Cluster
    4. Performing Backups
    5. Disaster Recovery

Description and Inputs

Boot Scripts

The following Boot scripts are listed in order.

RightScript: SYS Timezone set v1

Description: A RightScale provided script to set the timezone of this instance.

Required Inputs and Default Settings

  • SYS_TZINFO | UTC
RightScript: SYS Monitoring install v9

Description: A RightScale provided script to install the integrated monitoring capabilities.

Required Inputs and Default Settings 

  • MON_PROCESSES | memcached 
  • SERVER_UUID | RS_INSTANCE_UUID
  • SKETCHY | RS_SKETCHY
RightScript: SYS lvm on /mnt v9

Description:  A RightScale provided script to create a large LVM under /mnt.

Required Inputs and Default Settings:

  • OPT_LVM_SIZE| max
RightScript: SYS Add Swap Partition

Description:  A RightScale provided script to configure an amount of swap on this instance.  While actively using a system in swap is not recommended, Couchbase's best-practice recommendation is to have at least some swap configured so as to prevent the Linux OOM killer from terminating the Membase processes.

Required Inputs and Default Settings: (a detailed description of this RightScript can be found here)

  • OPT_SYS_LVM_SWAP_SIZE_IN_G | 5
RightScript:  RB rubygems 1.3.1 + quickinstall v4

Description:  A RightScale provided script needed for EBS functionality.

Required Inputs and Default Settings:

  • none
RightScript:  EBS stripe Rightscale tools install v2

Description:  A RightScale provided script to install the tools needed for EBS functionality.

Required Inputs and Default Settings:

  • none
RightScript:  Couchbase Server - EBS stripe volume create v2

Description:  A RightScale-provided (and Couchbase-modified) script to create a new EBS stripe volume.

Required Inputs and Default Settings:

  • EBS_STRIPE_COUNT | 2
  • EBS_TOTAL_VOLUME_GROUP_SIZE_GB | 10
  • CB_DNS_NAME | $ignore ( In the context of this script, CB_DNS_NAME is used to rename the EBS stripe volume so that it can be identified at a later boot time.)
  • CB_INITIAL_LAUNCH | TRUE (In the context of this script, CB_INITIAL_LAUNCH controls whether or not to run this script. When recovering from an backup of this instance, it is not necessary to re-run the script.)
  • CB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.
RightScript:  Couchbase Server - EBS stripe volume restore v2

Description:  A RightScale-provided (and Couchbase-modified) script to restore from an existing EBS stripe volume snapshot.

Required Inputs and Default Settings:

  • EBS_LINEAGE_PREFIX_OVERRIDE | $ignore
  • EBS_SNAPSHOT_TIMESTAMP | $ignore
  • CB_DNS_NAME | $ignore (In the context of this script, the CB_DNS_NAME is used to identify the EBS stripe volume snapshot to be restored from.)
  • CB_INITIAL_LAUNCH | TRUE (In the context of this script, CB_INITIAL_LAUNCH controls whether or not to run this script. When recovering from an backup of this instance, it is necessary to run this script.)
  • CB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)
RightScript:  Couchbase Server - EBS create backup scripts v2

Description:  A RightScale-provided (and Couchbase-modified) script to create the appropriate EBS backup scripts.

Required Inputs and Default Settings:

  • CB_DNS_NAME | $ignore (In the context of this script, the CB_DNS_NAME is used to identify the EBS volume.)
  • CB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)
RightScript:  Couchbase Server - EBS continuous backups v2

Description:  A RightScale-provided (and Couchbase-modified) script to configure this instance for automatic and continuous EBS  backups.

Required Inputs and Default Settings:

  • EBS_BACKUP_FREQUENCY | $ignore (random)
  • EBS_BACKUP_KEEP_DAILY | $ignore (14)
  • EBS_BACKUP_KEEP_LAST | $ignore (60)
  • EBS_BACKUP_KEEP_MONTHLY | $ignore (12)
  • EBS_BACKUP_KEEP_WEEKLY | $ignore (6)
  • EBS_BACKUP_KEEP_YEARLY | $ignore (2)
  • CB_DNS_NAME | $ignore (In the context of this script, the CB_DNS_NAME is used to identify the EBS volume.)
  • CB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)
RightScript:  Couchbase - Enterprise Server Install

Description:  This script will install the Couchbase Server Enterprise Edition.

Required Inputs and Default Settings:

  • none
RightScript:  Couchbase Server - Enterprise Edition EBS Setup

Description:  This script will configure the Couchbase Server software to use the EBS stripe volume attached to the instance.

Required Inputs and Default Settings:

  • CB_INITIAL_LAUNCH | TRUE (In the context of this script, CB_INITIAL_LAUNCH controls whether or not to run this script. When recovering from an backup of this instance, it is not necessary to re-run the script.)
  • CB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)
RightScript:  Couchbase Server - DNS Setup

Description:  This script will edit an instance's configuration to identify itself using a hostname rather than it's internal IP address.

Required Inputs and Default Settings:

  • CB_DNS_NAME | $ignore (Used to denote the DNS name of this particular instance.  A value here will cause this instance to identify itself via a DNS in the Membase UI.  When combined with an EBS volume, this field is necessary for disaster recovery.  It is essential that this DNS name resolves to the IP address of this server from any other Membase servers.  We recommend using DNSMadeEasy (http://www.rightscale.com/library/ri...nal-id-de/7165)  If you are using some other DNS system, it will be necessary to ensure (either via a script or manually) that this DNS name gets updated with the appropriate value.)
  • CB_INITIAL_LAUNCH | TRUE ( In the context of this script, CB_INITIAL_LAUNCH controls whether or not to run this script. When recovering from an backup of this instance, it is not necessary to re-run the script. )
RightScript:  Couchbase Server - Join Cluster

Description:  This script will attempt to join this instance to a Couchbase cluster located at CB_CLUSTER_ADDRESS with username set to CB_USER and password set to CB_PASS.  Optionally, CB_AUTOREBALANCE can be set to TRUE to have this script kick off a rebalance after joining.

Required Inputs and Default Settings:

  • CB_AUTOREBALANCE | FALSE (When set to TRUE, the server will automatically rebalance the clusters upon joining the cluster denoted by CB_CLUSTER_ADDRESS.)
  • CB_CLUSTER_ADDRESS | $ignore (Used to denote the cluster to be joined. Populate CB_CLUSTER_ADDRESS with the IP address of one of the servers in an existing Couchbase cluster and the new server will automatically join that cluster. An IP address, or an ENV type input with EC2_PRIVATE_HOSTNAME set to the Name of the server may be used. This approach automatically fetches the IP address of the server indicated by EC2_PRIVATE_HOSTNAME instead of hardcoding a specific (but temporary) IP address into the input. IP addresses of servers in EC2 are dynamically assigned and will change upon shutdown and restart. Therefore, the use of EC2_PRIVATE_HOSTNAME results in less need for manual configuration. When launching the first Couchbase server of a cluster set to $ignore. Then return to give it the EC2_PRIVATE_HOSTNAME of one of the other machines in the Couchbase cluster.)
  • CB_INITIAL_LAUNCH | TRUE (In the context of this script, CB_INITIAL_LAUNCH controls whether or not to run this script. When recovering from an backup of this instance, it is not necessary to re-run the script.)
  • CB_JOIN | FALSE (Set to TRUE to activate the cluster joining script.  This is useful when you want to have the CB_CLUSTER_ADDRESS, CB_PASS and CB_USER fields defined, but still be able to toggle whether to run the join script or not. )
  • CB_PASS | $ignore (Used to denote the password for the server that this instance is attempting to join.  Note that this does not actually apply the password to the running instance, it is simply used for connecting to the remote cluster.)
  • CB_USER | $ignore (Used to denote the username for the server that this instance is attempting to join. Note that this does not actually apply the username to the running instance, it is simply used for connecting to the remote cluster. )
RightScript:  Couchbase Server - Delay Decommission

Description:  This script extends the decommissioning timeout in an EC2 environment. The purpose is to allow enough time for a node to be rebalanced out of the cluster before the instance is terminated (see "Couchbase - Leave Cluster" below).

Required Inputs and Default Settings:

  • CB_DELAY_SECONDS  | 3600 (The number of seconds to delay the hard-termination of this instance. Depending on the amount of data, rebalancing can take a significant amount of time so it is best to set this value quite high.)

Operational Scripts

RightScript:  EBS unfreeze volume backups

Description:  A RightScale provided script to manually resume the automatic backups of the EBS volume.

Required Inputs and Default Settings:

  • CB_USE_EBS| TRUE (Whether or not this instance should use the EBS functionality.)
RightScript: EBS freeze volume backups

Description:  A RightScale provided script to manually stop the automatic backups of the EBS volume

Required Inputs and Default Settings:

  • MB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)
RightScript:  EBS stripe volume backup v2

Description:  A RightScale provided script to manually perform a backup of the EBS volume.

Required Inputs and Default Settings:

  • EBS_BACKUP_KEEP_DAILY | $ignore (14)
  • EBS_BACKUP_KEEP_LAST | $ignore (60)
  • EBS_BACKUP_KEEP_MONTHLY | $ignore (12)
  • EBS_BACKUP_KEEP_WEEKLY | $ignore (6)
  • EBS_BACKUP_KEEP_YEARLY | $ignore (2)
  • MB_USE_EBS | TRUE (Whether or not this instance should use the EBS functionality.)

Decommission Scripts

While no decommission scripts are provided by default, the Operational Script "Couchbase Server - Leave Cluster" can be used as a decommission script.  Make sure that the "Couchbase Server - Delay Decommission" script is set to run on boot and that the decommission delay is long enough for the rebalancing event to take place (one hour by default should be plenty of time).

Common Runbook Operations

Note: This runbook is not meant to provide a substantial background of the underlying architecture of Couchbase.

Security Group Settings

  • The security groups for these servers need to allow these ports between nodes: 8091 (web port), 11211 (data port), 11210 (direct Couchbase port), 4369 (epmd) and the port range 21100 to 21199, inclusive (dynamic ns_server usage). 
  • You'll also want ports 11211 and 11210 open between your clients and the Couchbase servers.
  • For external access, port 8091 should be open to whatever system will be accessing the management UI (GUI or REST API).

Setting up Your First Node

  • A cluster of Couchbase servers is created by telling the member servers the network address of one of the servers, and the username and password that allows it to authenticate. When the 1st server is launched, there is no cluster to join yet.

  • Therefore, set the [CB|MC]_CLUSTER_ADDRESS, [CB|MC]_PASS, and [CB|MC]_USER to $ignore. Launch the first server and wait for its status to change to "operational". Once the server is operational, go to its "Info" tab and click on the "Public DNS Name" and change the port to 8091:  Example: http://ec2-184-72-182-181.compute- 1.amazonaws.com:8091
  • If the server is operational but it's refusing connections to port 8091 then it is likely that the security group for the server doesn't have port 8091 open to the IP address that it's attempting to connect to it. Allowing all IP addresses to connect to port 8091 temporarily is a way to test if this is the case.
  • Once you have connected to the Couchbase web console, configure the servers using the UI. (See http://www.couchbase.org/wiki for more documentation and support information.)

cluster_overview_blank.png

Adding Nodes to a Cluster

  • After configuring a new Couchbase server using its web console, set its username and password. When launching additional servers, they must be configured with the proper credentials to join a cluster.
  • Set the [CB|MC]_CLUSTER_ADDRESS to ENV[EC2_PRIVATE_HOSTNAME] or to an IP/DNS name of the nickname of one of the servers in the cluster.  See the inputs section here for more details.
  • When a server has joined a cluster it will show up in the web console as shown below:

 

server_added.png

Performing Backups

  • Performing backups with the Couchbase Enterprise Edition in RightScale is performed the same way as on any other deployment.  See the instructions provided here.

Disaster Recovery

Note: Applies only to the "Couchbase Enterprise Edition" ServerTemplate.

  • When using Couchbase in a production and mission critical environment, it is best to have a disaster recovery plan in the event that all servers within a cluster are lost.  Couchbase is designed to be highly available via a user-configureable number of replicas.  However, a total cluster outage will require the data to be reloaded from disk.  The only way to recover your data in EC2 is by using EBS volumes that will persist even after its instance has been terminated.

  • The two prerequisites to performing a recovery is that all Couchbase servers were originally configured to use a DNS name (via the DNS_ID input to the ServerTemplate) and that all the Couchbase servers were configured to use an EBS volume (via the CB_USE_EBS input).  All data and configuration information is saved on the EBS volume.  Additionally, the volume will have been automatically renamed to match the DNS_ID provided so that the administrator can properly signal the RightScale recovery process.

  • To recover an instance that had its configuration and data saved on an EBS volume:

    1. Create a new server from the same ServerTemplate as the original instance.

    2. Set the DNS_ID of the new instance to be the same value as the instance you are recovering.

    3. Set the CB_INITIAL_LAUNCH value to FALSE and make sure the CB_USE_EBS is still set to TRUE.

  •  That's it!  Just let the new instance come up and it will automatically attach an EBS volume based upon the last snapshot that was taken.

You must to post a comment.
Last modified
23:39, 16 May 2013

Tags

This page has no custom tags.

Classifications

This page has no classifications.

Announcements

None

Powered by MindTouch ®

© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.