Note: Please go to docs.rightscale.com to access the current RightScale documentation set. Also, feel free to Chat with us!
Home > Clouds > AWS > AWS FAQs > How do EBS backups of my MySQL instance work?

How do EBS backups of my MySQL instance work?

Table of Contents

Background

This is just an overview of how the "DB EBS Backup" script works when it takes a backup of your MySQL servers.

Answer

One of the first things done by the script, is verify which type of instance we are running against. We run a check to see if it's a master or a slave. To determine if it's a master, we will check the IP address, then compare that to the MasterDB DNS input, to make sure the IP addresses match. If the IP addresses match, then we will run the following line:

/usr/local/bin/mysql-binary-backup.rb --if-master $script_args && \

This will backup the current database to an EBS snapshot, and tag that snapshot with -master- in the nickname, as well a tag it as a master instance, with "rs_backup:from_master=true".

If the IP address of the master DNS input doesn't match the IP of the server, we will then check replication. You will see the following message in /var/log/messages or in the audit entries for the backup script:

This is not a Master instance...must be a slave.

From this point, we will do a logic check to insure that the slave is replicating properly. So we run the following:

  dbcon = Mysql.new("", "root" )
  slave_status = dbcon.query("SHOW SLAVE STATUS").fetch_hash
  if slave_status.nil? || slave_status["Seconds_Behind_Master"].nil? || slave_status["Seconds_Behind_Master"].strip.to_i > max_lag
    puts "Slave is not replicating correctly!!! Exiting..."
    exit -1
  end

If the results come back negative, the backup will cease, since the instance doesn't look like a master instance, from our viewpoint, nor does it look like a properly replicating slave, so we will stop the backup, to try and prevent any faulty snapshots being taken.

After all of those checks have been done, we will start the actual backup process. First, we will check for a lockfile, to see if a backup could already be in progress. So we run the following lines of code:

#check for lock file and if not found create it
if !File.exists?(options[:lock_file])
  f = File.new(options[:lock_file],File::CREAT|File::RDWR, 0600)
  f.write("0")
  f.close
end

Next, we check if MySQL is currently running, and we have a check to find with distribution of Linux we are on:

# 1- Check if the host has mysql running (abort if not)
if ENV['RS_DISTRO'] == 'ubuntu'
  mysql_service = "mysql"
  mysql_started_tag1 = "Threads"
  mysql_started_tag2 = "start/running"
else
  mysql_service = "mysqld"
  mysql_started_tag1 = "MySQL running"
  mysql_started_tag2 = "mysqld\s.+is running"
end
status_res = `service #{mysql_service} status`

If this check comes back without the service running, we will terminate the backup, since there could be potential health issues that are causing MySQL to crash. Following this check, we will then check all current snapshots using the lineage specified on that instance's inputs. Based on the amount of snapshots allowed from inputs, if we find any, we will check the type, and clean up any snapshots that will put us over the maximum allowed snapshots of that type.

Finally, we freeze the filesystem, to prevent writing to the system while we take the backup. We will also lock the database, but allow all existing transactions to finish. We attempt to keep the database locked for as little as possible (usually only a few seconds), but it can cause slowdown during the backup process. We will then take a snapshot of the current volume setup (creating multiple snapshots if the system is using a stripe) and then send it to EBS, making the name as "<lineagename>-<master/slave>-<timestamp>. These snapshots can then be used in promotions, restoration, or to bring up new instances.

 

You must to post a comment.
Last modified
21:29, 16 May 2013

Tags

EBS

Classifications

This page has no classifications.

Announcements

None


© 2006-2014 RightScale, Inc. All rights reserved.
RightScale is a registered trademark of RightScale, Inc. All other products and services may be trademarks or servicemarks of their respective owners.