Chef Developer Workflows

Objective

Show a recommended developer workflow for developing ServerTemplates using Chef cookbooks and recipes.

Overview

There are two basic types of workflows for Chef development.

• Workflow using refetch
• Workflow using rsync

The rsync program is available on Linux machines and can be used to synchronize file trees across local disks, directories, or a local network. In this example, rsync will be used to synchronize the cookbook repositories on your local development box with the cookbook repositories on the running server instance.

Workflow using push and refetch

Steps

Note: You must use this workflow if you are going to make any changes to the Chef metadata (e.g. Add a new recipe).

1. Create a GitHub account and set up a working environment where you can push/pull changes between your local development machine and your GitHub repositories. See Create a New Cookbook Repository on GitHub. 
2. Clone the ServerTemplate to create an editable version and name it accordingly. It's recommended that you commit the ServerTemplate before you make any changes, which will make it easier to track your changes against the original version.
3. Go to the Scripts tab of the HEAD version (of the cloned ServerTemplate) and click Modify.
4. Attach any new or modified cookbooks to the ServerTemplate. (Note: You can only attach one version of a cookbook to a ServerTemplate.)
5. Add or change the list of scripts by adding new recipes or replacing old recipes with ones from your repository. 
6. Create and launch a server with the HEAD version of the cloned ServerTemplate. It's important to use the HEAD version because you're likely going to change and test the ServerTemplate multiple times before you save your changes and commit the ServerTemplate to create a static revision. 
7. Make a local change to a Chef cookbook/recipe and push the changes from your local development machine to your repository (e.g. GitHub). 
8. The next step is to import the new and/or modified cookbooks from your repository (e.g. GitHub) into Repose (RightScale's mirror of your repositories) so that all of your changes can be used by ServerTemplates and Servers. See Refetch and Import a Repository.
9. Wait for the refetch and import to be 100% completed.  You can monitor the status of the refresh action in the Events pane.
10. In the RightScale dashboard, go to the running Server's Scripts tab and run the modified recipe.
11. Check the Server's Audit Entries tab to verify that the script was executed successfully. 
12. Repeat steps 3-10 each time you want to test a new script or code modification. 

Workflow using rsync

Note: You can only use this workflow if you are not going to make any changes to the Chef metadata (e.g. Add a new recipe).

Steps

1. Create a GitHub account and set up a working environment where you can push/pull changes to/from your local development machine to your GitHub repositories. See Create a New Cookbook Repository on GitHub. 
2. Create a server with a HEAD version of ServerTemplate that is using Chef cookbooks and recipes for server configuration.  The cloud server will be used for developing and testing your scripts. 
3. Under the Server's Info tab, add the following tag: rs_agent_dev:download_cookbooks_once=true
   The "download cookbooks once" tag tells the RightScale platform to only download the comma-separated list of cookbooks once from Repose (at launch time) instead of every time before a script is run (default with no tag). 
4. Launch the server.   
5. Modify a Chef cookbook or recipe. Instead of pushing the changes from your local development machine to your repository (e.g. GitHub) so that you can run and test the script on the running server, you can rsync the changes directly from your development machine to the running server instance in the cloud.
6. Before you can run the rsync command, you will need to determine the unique repository ID for each repository that you're going to modify by SSHing into the running server instance.  See the Find a Cookbook UID section below.
7. Use rsync to synchronize the cookbook directories on your local development machine and the cloud server instance.  You can either run the rsync command locally on either your development machine or instance, or create a script that will accomplish the same task.
   • Command Line - Development Machine
   • Ruby Script ('cbsync')
      
8. You can now run the modified script using 'rs_run_recipe' directly on the running server instance.  Open an SSH console session on the server instance and run the following command to view the logs in realtime as the script is run:

rs_run_recipe -n "<cookbook_name>::<recipe>" && tailf /var/log/messages

9. Test and verify that the script was executed successfully. 
10. Repeat steps 5-9 to test any further script modifications.  
11. Once you are satisfied with all of your changes, commit and push the changes from your local development machine to your repository (e.g. GitHub). 
12. Terminate the current development cloud server.
13. The next step is to import the new or modified cookbooks from your repository (e.g. GitHub) into Repose (RightScale's mirror of your repositories) so that all of your changes can be used by ServerTemplates and Servers. See Refetch and Import a Repository. 
14. Launch a new server instance using the HEAD ServerTemplate that reflects your latest code changes.
15. Test the cloud server to make sure the scripts were run successfully.  Check the Server's Audit Entries tab for details.
16. Commit the HEAD ServerTemplate to create a static and tested revision that you can use in a production environment.  Note: When you commit a HEAD ServerTemplate, the contents of the attached cookbooks becomes immutable. 

# cd /var/cache/rightscale/cookbooks/default/

# cd /var/cache/rightscale/cookbooks/default/

# ls
05fe0e9401da911fdb518fe37b3a8c5393e3883d
12fdb518fe37b3a8c5393e3283d05fe0e9401da9
b518fe37b3a8c5193e3883d05fe0e9401da912fd

# cd 05fe0e9401da911fdb518fe37b3a8c5393e3883d/cookbooks/

# rsync from Local Development Machine to Cloud Instance
#
# Assumes public key has been added to /root/.ssh/authorized_keys
#
# rsync -avz --exclude=.git <src_path> root@<instance>:/var/cache/rightscale/cookbooks/default/<HASH>
# where <scr_path> is the directory of where the cookbooks are locally stored on your development machine.
# <dst_path> is the directory of where the cookbooks are locally stored on the cloud server instance.
# <HASH> is the Cookbook UID.
# EXAMPLE:
rsync -avz --exclude=.git /src/cookbooks/ root@ec2-50-19-197-20.compute-1.amazonaws.com:/var/cache/rightscale/cookbooks/default/05fe0e9401da911fdb518fe37b3a8c5393e3883d/cookbooks