Create a Deployment (if needed)

Note: Some may have already created a Deployment. If so, please skip to the next step.

Go to Manage -> Deployments and select the "New" action button to create a new Deployment. Provide a title, brief description that describes the project, and select a preferred Availability Zone. Later, when you add Servers to a Deployment you can define a specific Availability Zone. The Availability Zone that is defined at the Deployment level will only take effect if you add a Server with '-any-' as its Availability Zone setting. 

For this tutorial, ignore the Default VPC Subnet and Default Placement Group and keep the default 'None' settings.

Make a selection and click Save.

Add Servers to the Deployment and Define Inputs

The "Database Manager with PostgreSQL" ServerTemplates are based off of EBS Snapshots. Snapshots are highly efficient backups representing an EBS volume at a specific point in time. New EBS volumes are created from a snapshot and are attached to a master/slave instance. When you launch a new Server using this ServerTemplate, it will look for the most recent EBS snapshot (or snapshots if you're using a set of striped EBS volumes) that's available to begin the boot process. 

Since you are creating this EBS setup from scratch, you will not have a backup (i.e. a set of EBS Snapshots) of the database. Therefore, the first step is to create your first backup of the database. When dealing with EBS Volumes, you should always attach fresh volumes that were created from the most recent snapshots. 

Go to Design -> MultiCloud Marketplace and import the following components:

• ServerTemplate: "Database Manager with PostgreSQL 9.0" 

From the Deployment's Servers tab, and click the Add Server button.  Select the cloud region where you want to create your database setup (e.g. 'AWS US-East'). Select either the imported or cloned version of the "Database Manager with PostgreSQL 9.0" ServerTemplate. Provide a nickname (ex: my-db1) and select your SSH Key and Security Group.  For more information on the Add Server assistant, see Add Server Assistant.

For the Availability Zone, you can either launch the Master/Slave servers in the same or different zones. For production environments, you might want to have your database servers in separate zones, but for cost-efficient testing purposes.

Select a particular Availability Zone. (e.g. us-east-1a) Click Add. This first Server will become your "Master" database Server. 

Although a running Slave-DB is technically not required to have a functional site, it's obviously strongly recommended for replication and failover purposes. Fortunately, you can use the same ServerTemplate to add a second database server that will become a "slave" of the Master-DB. Although, you can repeat the steps again to create a second server, the easiest way is to simply clone it. Click on the nickname link of the first database server. Click the Clone button.  Rename the second server (e.g. my-db2). 

For high-availability purposes, it's also recommended that you launch the Slave-DB server in a different availability zone as the Master-DB. 

Under its Info tab, click the Edit button. Select a different availability zone as the Master-DB (e.g. us-east-1b).

Next, as a best practice we will define some common Input parameters at the Deployment level. Go to the Deployment's (not the Server's) Inputs tab. By defining Input parameters at the Deployment level, all Servers in the Deployment will inherit these Inputs and their values, so we will only have to define them once. Notice that most of the Input parameters can be inherited from the ServerTemplate. However, there are a few required inputs that need to be defined, as well as a few other Inputs whose values you might want to change. 

Select the Deployment's Inputs tab and click the Edit action button to configure the following Input parameters. 

Required

Hover over the info icon next to the Input name for a detailed description and sample value. The values that need to be provided below will depend on the DNS Provider that you're going to use to handle the dynamic DNS records for you master/slave database servers. 

When ready, select Save to save your Inputs.

Launch Database Server

Go back to the Deployment's Servers tab and launch the first database Server (e.g. mydb). This server will soon become your Master-DB Server. Click the launch action icon.

When the Input confirmation screen appears, provide the following parameters. Since you don't have a Master-DB for the Server to synchronize with at this time, we need to make sure that the Server does not try to initialize to "Slave" Server at boot time. You must set the value for the INIT_SLAVE_AT_BOOT Input parameter to "False" which is the default setting.

Click the Launch action button to continue launching your database Server.

In a few minutes, you will have a new Server instance with PostgreSQL installed, but it has an empty database or your imported database. Be sure to wait for the Server to boot-up and enter the "operational" state before continuing to the next step (~5-10 min). 

Troubleshooting

If you receive an error message about missing input parameters and no input is highlighted in red, it's most likely because some of the required RightScale Credentials do not exist. The ServerTemplates have been preconfigured to use certain credentials and assumes that you've already created them in an earlier prerequisite step. If you haven't completed the Create Credentials for Common Inputs step in the Deployment Setup section, you most likely haven't created the required credentials that the ServerTemplates are trying to use. Once you've created those credentials, you should be able to successfully launch the Server without receiving any missing input error message.

Set up PostgreSQL Database

The next step is to create and attach an EBS volume or stripe of EBS volumes to the running instance where your PostgreSQL data will be stored. Click on the running Server's Scripts tab. 

Next, execute another 'Operational Script' called "DB Create PostgreSQL EBS stripe volume" RightScript, which will create and attach an EBS stripe designed for a PostgreSQL Server. Note: If you do not see this RightScript in the list, you will need to import it from the Design -> MultiCloud Marketplace -> RightScripts.

By default, this RightScript will create a blank PostgreSQL database with no striping (EBS _STRIPE_COUNT = 1) that includes all the necessary privileges for replication, administration, and application purposes. However, if you have an existing PostgreSQL database that you want to use instead, you will need to define the "optional" parameters below if you haven't already imported (defined the inputs in the beginning).

Be sure to select the latest revision of the script that's available.

Register Server as the Master-DB with your DNS Provider

In order for the automatic backup RightScript to function properly, you first need to designate the running instance as the "master" database Server. The next step is to run a script that will make the server function as a "Master-DB" server.

Go the Scripts tab of the running Server. Under the Any Script section, select and run the "DNS Master DB register" RightScript to register this server as the "Master-DB" and update your DNS A records accordingly. Note: If you do not see this RightScript in the list, you will need to import it from the Design -> MultiCloud Marketplace -> RightScripts.

Select the script and click Run.

When the Inputs confirmation window appears, enter the missing information. Be sure to use the same values that you defined earlier at the Deployment level. Note: When you run an 'Any Script' the inputs are not pre-populated with the values that are defined at the Deployment level.

The following inputs will be used by the script. Since you set these values at the Deployment level earlier in the tutorial, you will not need to specify them again.

Required

When the script is run, the running server's Private IP Address will be reported to your DNS Provider so that the A Record that you previously defined for your master database server can be properly updated. For a quick verification check, you can log into your DNS provider and check your A Records. The IP address of the "Master-DB" A Record and the private (not public) IP address of the server you just defined as the "Master-DB" should match. Notice that since we don't have a slave up and running, the A Record for the Slave-DB still has its placeholder IP address (1.2.3.4). Later, when we launch the Slave-DB, this A Record will automatically be updated by one of the boot scripts.

The screenshot below shows an example for DNSMadeEasy.

Create a Backup of your Master-DB

The next step is to manually create a snapshot (backup) of the EBS volume(s), which is required before you launch the second (slave) Server. When you launch a Slave-DB Server, it will use the most recent backup to restore the database so that it will be able to come up and become in-sync with the Master-DB in a much shorter timeframe. Go to the Scripts tab of the running Server and run the "DB EBS PostgreSQL backup" operational RightScript. This script will take a backup of your PostgreSQL database by creating a snapshot for each attached volume in the stripe.

Important! 

You must wait for all of the snapshots to be 100% complete before continuing the tutorial. Go to Clouds -> AWS Region -> EBS Snapshots to check the status. If there are pages of Snapshots, use the Filter by Nickname option and search for the value that you set for the DB_LINEAGE_NAME input to quickly find your related snapshots (e.g. mystripe). You should find a snapshot for each volume in the stripe.

Note: The time required to complete the initial EBS snapshot will vary depending on the size of the EBS volume(s). For example, it can take a long time (over an hour) for large volumes, where smaller volumes might complete in just 5-10 minutes.

You now have a backup (set of EBS Snapshots) of your PostgreSQL database in the form of a set of EBS snapshots that you can use to launch a fresh, new database Server using the same ServerTemplate. Notice that each Snapshot has the same name and timestamp, but has a unique ID and a tag which denotes the stripe order.

Launch a Slave-DB Server

Once the master snapshots are 100% complete, you are ready to launch your Slave-DB Server (e.g. mydb2). 

Before you launch your Slave-DB, you might want to consider changing its availability zone. Since you created the slave server by cloning the Master-DB, it's configured to launch an instance into the same availability zone as the master. However, if you need added redundancy (and are willing to pay for the additional data transfer costs) it's in your best interest to launch a Slave-DB in a different availability zone as the Master-DB, which will allow you to have a running and in-sync backup server that you can use for failover and recovery scenarios if there's ever a major failure in the availability zone of your Master-DB server. You can edit the Server's availability zone under its Info tab.

Important! Set the INIT_SLAVE_AT_BOOT Input parameter to "True" since you have a Master-DB Server that's currently running. Setting this Input to "true" will instruct the instance to initialize PostgreSQL as a "Slave-DB" server during the boot process.Click the Launch action button of the second Server in your Deployment. This new Server will become a replicating slave Server of the existing master Server.

This time the ServerTemplate will find the most recent snapshots with the appropriate DB_LINEAGE_NAME (e.g. mystripe). The snapshot(s) will have a matching prefix (e.g. mystripe-master-yyyymmddttmm).

Note: For optimization purposes, the ServerTemplate will automatically use the most recent completed set of backup snapshots regardless of whether it's a "slave" or "master' backup. This way, the slave will be able to "get up to speed" faster and be in-sync with the Master-DB server in a shorter timeframe.

Congratulations! In a few minutes, when the Server enters the "operational" state, you will have a PostgreSQL master/slave database setup with data replication and automatic backups saved as EBS snapshots.

Since we defined this server to be a Slave-DB server at launch time, one of the scripts reported its Private IP Address back to your DNS provider. So, if you checked your A Records again you should see that both database A Records are properly pointing to the Private IP Addresses of your master and slave database servers.

The screenshot below shows an example for DNSMadeEasy.

Note: In this example that we purposely did not name the first database server "master" (in the Dashboard) and the other one "slave" because we do not want the Server names to become a source of confusion. Remember, over the lifecycle of your setup you'll most likely have to promote a "slave" to become the new "master" database Server for troubleshooting purposes or for performing various upgrades (e.g. vertical scaling from a 'small' to 'large' instance type). 

Important! If for any reason both Servers are shutdown and you want to recreate the master/slave setup, it technically doesn't matter which Server you launch first. Simply launch the first Server with INIT_SLAVE_AT_BOOT = False. Once the Server becomes operational, simply run "DB EBS PostgreSQL restore and become master" RightScript to make it become the "master" Server. Then you can launch the other Server as the "slave" with INIT_SLAVE_AT_BOOT = True.

Shutting down a Database Server

If you ever need to shut down a database server, be sure to terminate it by running the 'DB TERMINATE SERVER' operational script, which will automatically delete the detached EBS volumes. If you terminate an instance using the "Terminate" action button in the Dashboard, the detached EBS volumes will not be deleted. You will need to manually delete them.

Backup Frequency

The frequency of the backups are defined in the ServerTemplate. You can modify any of the Inputs to change the frequency and number of stored backup snapshots. 

• DB_BACKUP_KEEP_DAILY - Defines the number of daily snapshots to keep.
  
  • Default: 14
• DB_BACKUP_KEEP_LAST - Defines the number of snapshots that will be saved before the oldest snapshot is removed.  
  • Default: 60  (60 snapshots of the master, 60 snapshots of the slave)
• DB_BACKUP_KEEP_MONTHLY - Defines the number of monthly snapshots to keep.
  
  • Default: 12
• DB_BACKUP_KEEP_WEEKLY - Defines the number of weekly snapshots to keep.
  
  • Default: 6
• DB_BACKUP_KEEP_YEARLY - Defines the number of yearly snapshots to keep.
  • Default: 2

Subsequent Launches of the Database Servers

Since this lab only covers the initial launch of the server, here is how to produce subsequent launches:

For Master:

• Launch the Server that will become your Master-DB. Set INIT_SLAVE_AT_BOOT = false
• Once the Server becomes operational, run the "DB EBS PostgreSQL restore and become master" Operational Script

For Slave:

• Once the Master-DB has been configured, launch the Server that will become your Slave-DB. This time make sure to set INIT_SLAVE_AT_BOOT = true