CloudOptimizer Setup and Runbook Guide

Define Inputs

You will need to define any missing inputs for any boot/decomission scripts before you can successfully launch the server. Some inputs apply to operational scripts and can be left blank.

The following inputs must be specified for the successful execution of the CloudOpt scripts.  It's recommended that you define the inputs at the deployment level.  Go to the deployment's Inputs tab and define values for the following inputs:

• AWS_X509_CERT - Select the "cred:AWS_X509_CERT" option from the dropdown menu to use the credential with the matching name that you created in an earlier step.
• AWS_X509_KEY -  Select the "cred:AWS_X509_KEY" option from the dropdown menu.
• CACHE_SIZE_GB -  Enter the size (GB) of the EBS volume that CloudOptimizer will use to store its byte cache. Minimum size if 30GB. Required input for launching a gateway server. (e.g. 30)

(Optional) You can define values for the other missing inputs, but it's not required to successfully launch the server.

Click Save.

Endpoint Configuration

Note: Please refer to http://kb.cloudopt.com assistance with any of the steps below. You can click on “All Articles” to access all of the articles on the CloudOpt Knowledge Base.

In an Endpoint Configuration, CloudOptimizer runs on the same system that is used to initiate file transfers.

1. You will need to launch two CloudOptimizer Endpoint servers where one server will send files and another server will receive files.  Change the name of the server that was created using the "CloudOptimizer Gateway" ServerTemplate to denote that it will be the "sender" and the clone the server to create one that will be the "receiver."  Be sure to change the name of the receiving server. 
2. Launch both servers.
3. SSH into the CloudOptimizer Endpoint instance that will be used to send files.
4. Generate a test file, using the following command to produce a normal but somewhat random file that is 100 MB in size and is named 1-100.mb.

   • test_file_generator.pl --size 100

5. Transfer the file to the receiving CloudOptimizer system, using the following command.  Replace <key> with the SSH authentication key, <username> with a user, and <address> with the private IP address all from the receiving CloudOptimizer system.

   • scp -S cloudcopy -i <key> 1-100.mb <username>@<address>:~/.

6. CloudOptimizer will optimize the transfer of the file and report the amount of time that it takes to complete the transfer with the bandwidth savings CloudOptimizer achieves.
7. Check the transfer statistics from CloudOptimizer, with the following command:

   • cloudstats
     

   • This will show the bandwidth savings for the completed transfer. Here is an example of these statistics:

   • Component - cloudcopy
     ------------------------------------------------------------
           Number of connections:          1
           Original data size:        	100.00 MB
           Transferred data size:        	100.89 MB
           Bandwidth Saving:          	0 bytes (0.0%)
     

8. Use an editor to make a modification to the file on the client CloudOptimizer Endpoint:

   • nano 1-100.mb

9. Add random characters at different points in the file by adding characters at a random place or two in the file.

   • Save the file - <Cntl - W>

10. Exit the editor
11. Clear the cloudstats

    • cloudstats clear

12. Transfer the file again to the receiving CloudOptimizer system, using the following command. Replace <key> with the SSH authentication key, <username> with a user, and <address> with the private IP address all of the receiving CloudOptimizer system. CloudOptimizer will again optimize the transfer of the file using any redundant bytes that were recognized from the previous transfer and will report the amount of time that it takes to complete the transfer along with the bandwidth savings CloudOptimizer achieves.

    • scp -S cloudcopy -i <key> 1-100.mb <username>@<address>:~/.

13. Check the transfer statistics from CloudOptimizer, with the following command :

    • cloudstats

14. An example of the bandwidth savings would be:

    • Component - cloudcopy
      ------------------------------------------------------------
            Number of connections:     1
            Original data size:        100.00 MB
            Transferred data size:     1.61 MB
            Bandwidth Saving:          98.39 MB (98.4%)
      

15. CloudOptimizer in this mode works for more than just scp. Additional tests can be run using different applications and protocols. For example, using an http browser on the client and an http server on the receiving end, all http communication can be accelerated by either configuring CloudOptimizer peers/endpoints or by using CloudOptimizer Intelligent Mesh. CloudOptimizer will accelerate many protocols such as ftp, rsync, or others.
     

Gateway Configuration

Note: Please refer to http://kb.cloudopt.com assistance with any of the steps below. You can click on “All Articles” to access all of the articles on the CloudOpt Knowledge Base.

In a Gateway Configuration, files will be sent from the local computer to a CloudOptimizer running in Endpoint Mode via a CloudOptimizer running in Gateway Mode.

To keep the demonstration as simple as possible, you'll use two CloudOptimizer instances and one local computer. Use one CloudOptimizer server in Endpoint Mode and the other server in Gateway Mode.

1. You will need to launch two CloudOptimizer servers.  Use the server that you created with the "CloudOptimizer Gateway" ServerTemplate to launch a CloudOptimizer server in Gateway mode. Use the server that you created with the "CloudOptimizer Endpoint" ServerTemplate to launch a CloudOptimizer server in Endpoint mode.
2. SSH into the CloudOptimizer Gateway server and configure the Linux instance as an “endpoint” in the CloudOptimizer Gateway configuration /etc/cloudoptimizer.conf file.
3. Install a static route on the local computer with the Endpoint Mode CloudOptimizer as the host and the Gateway Mode CloudOptimizer as the gateway.
4. SSH into the Gateway Mode CloudOptimizer and run the cloudstats utility to monitor the transfer. Use the following command:

   • watch cloudstats

5. SSH into the Endpoint Mode CloudOptimizer server and configure or enable an http or ftp server, then locate or generate a file to use in a test transfer. For example, generate a 5 MB file called 1-5.

   • mb:dd if=/dev/urandom of=1-5.mb bs=5M count=1

6. Using curl or ftp, from the command line of the local computer, transfer the file from the address of the Endpoint Mode CloudOptimizer.
7. Observe the CloudOptimizer statistics in the cloudstats utility on the Gateway Mode CloudOptimizer. An example of the bandwidth savings would be:

   • Component - cloudcopy
     ------------------------------------------------------------
          Number of connections:     1
          Original data size:        5.00 MB
          Transferred data size:     5.04 MB
          Bandwidth Saving:          0 bytes (0.0%)
     

8. Use an editor to make a modification to the file on the local computer. Add random characters at different points in the file by adding characters at a random place or two in the file. Save the file using ‘Cntl - W then exit the editor.
9. Clear the cloudstats.

   • cloudstats clear

10. Transfer the file again to the address of the local computer from the Endpoint Mode CloudOptimizer. CloudOptimizer will again optimize the transfer of the file using any redundant bytes that were recognized from the previous transfer and will report the amount of time that it takes to complete the transfer along with the bandwidth savings CloudOptimizer achieves.
11. Observe the CloudOptimizer statistics in the cloudstats utility on the Gateway Mode CloudOptimizer. An example of the bandwidth savings would be:

    • Component - cloudcopy
      ------------------------------------------------------------
            Number of connections:        	1
            Original data size:          	5.00 MB
            Transferred data size:         	87.45 KB
            Bandwidth Saving:          	4.91 MB (98.3%)