Support Search

    Table of Contents

    Objective

    Show a recommended developer workflow for developing ServerTemplates using Chef cookbooks and recipes.

    Overview

    There are two basic types of workflows for Chef development.

    • Workflow using refetch
    • Workflow using rsync

    The rsync program is available on Linux machines and can be used to synchronize file trees across local disks, directories, or a local network. In this example, rsync will be used to synchronize the cookbook repositories on your local development box with the cookbook repositories on the running server instance.

    Which workflow is recommended?

    Although you can use both workflows for Chef development purposes, some developers may find it more useful to use the rsync model. One of the key benefits of using the rsync workflow is that you can develop your code faster and more efficiently because you do not have to constantly push all of the changes to your repository and refresh them in the Dashboard before you can run and test the script on the server. By using the rsync model, you can author code on your local development machine and test the script immediately on the running server without having to perform the push and refetch steps every time. This way, you will only have to push the code changes after you have already tested them on a running server.

    Note: If you are using the rsync model, you will never be able to refetch changes from your cookbook repositories. As a best practice, once you are satisfied with your changes after testing your scripts using rsync, you should commit the changes to your cookbook repositories and relaunch a fresh new server without the "development" tag (which is mentioned in more detail below).

    Workflow using push and refetch

    diag-Chef_Developer_Workflow_push-v2.png

    Steps

    Note: You must use this workflow if you are going to make any changes to the Chef metadata (e.g. Add a new recipe).

    1. Create a GitHub account and set up a working environment where you can push/pull changes between your local development machine and your GitHub repositories. See Create a New Cookbook Repository on GitHub
    2. Create a RepoPath that points to the cookbook repositories that contain the scripts (Chef recipes) that you want to add to your ServerTemplate. See Create a New RepoPath
    3. Create or update the ServerTemplate (under the Repos tab) so that it's using the new RepoPath object that you just created. 
    4. Create and launch a server with the HEAD version of the ServerTemplate that's configured to use the Chef cookbooks and recipes for server configuration. It's important that you use a HEAD ServerTemplate because you can only manually refresh/refetch the cookbook repositories on HEAD ServerTemplates. The refresh action is not available if you're using a committed revision of a ServerTemplate. 
    5. Make a local change to a Chef cookbook/recipe and push the changes from your local development machine to your repository (e.g. GitHub). 
    6. In order for your change to become visible and usable in the RightScale platform, you must refetch the Chef metadata so that a copy of the modified repository will be updated in Repose (RightScale's mirror of your repositories). ServerTemplates and Servers will always reflect what is currently available in Repose.  But, if you want to see your changes immediately, you must manually click the refresh button on your RepoPath object.  See Refresh Cookbook Repositories.
    7. Wait for the refresh to be completed.  You can monitor the status of the refresh action in the Events pane.
    8. In the RightScale dashboard, go to the running Server's Scripts tab and run the modified recipe.
    9. Check the Server's Audit Entries tab to verify that the script was executed successfully. 
    10. Repeat steps 3-7 each time you want to test a new script or code modification. 

    Workflow using rsync

    diag-Chef_Developer_Workflow_rsync-v2.png

    Note: You can only use this workflow if you are not going to make any changes to the Chef metadata (e.g. Add a new recipe).

    Steps

    1. Create a GitHub account and set up a working environment where  you can push/pull changes to/from your local development machine to your GitHub repositories. See Create a New Cookbook Repository on GitHub
    2. Create a RepoPath that points to the cookbook repositories that contain the scripts (Chef recipes) that you want to add to your ServerTemplate. See Create a New RepoPath
    3. Create or update the ServerTemplate (under the Repos tab) so that it's using the new RepoPath object that you just created. 
    4. Create a server with a HEAD version of ServerTemplate that is using Chef cookbooks and recipes for server configuration.  The cloud server will be used for developing and testing your scripts. 
    5. Under the Server's Info tab, add the following tag: rs_agent_dev:download_cookbooks_once=true 
      The "download cookbooks once" tag tells the RightScale platform to only download the required cookbooks and recipes once from Repose (at launch time) instead of every time before a script is run (default with no tag).
    6. Launch the server.   
    7. Modify a Chef cookbook or recipe. Instead of pushing the changes from your local development machine to your repository (e.g. GitHub) so that you can run and test the script on the running server, you can rsync the changes directly from your development machine to the running server instance in the cloud.
    8. Before you can run the rsync command, you will need to determine the unique repository ID for each repository that you're going to modify by SSHing into the running server instance.  See the "Find a Repository UID" section below.
    9. Use rsync to synchronize the cookbook directories on your local development machine and the cloud server instance.  You can either run the rsync command locally on either your development machine or instance, or create a script that will accomplish the same task.
    10. You can now run the modified script using 'rs_run_recipe' directly on the running server instance.  Open an SSH console session on the server instance and run the following command to view the logs in realtime as the script is run:
    rs_run_recipe -n "<cookbook_name>::<recipe>" && tailf /var/log/messages
    
    1. Test and verify that the script was executed successfully. 
    2. Repeat steps 5-9 to test any further script modifications.  
    3. Once you are satisfied with all of your changes, commit and push the changes from your local development machine to your repository (e.g. GitHub). 
    4. Terminate the current development cloud server.
    5. In order for your changes to become visible and usable in the RightScale platform, you must refetch the Chef metadata so that a copy of the modified repository will be updated in Repose (RightScale's mirror of your repositories). ServerTemplates and Servers will always reflect what is currently available in Repose.  But, if you want to see your changes immediately, you must manually click the refresh button on your RepoPath object.  See Refresh Cookbook Repositories.
    6. Launch a new server instance using the HEAD ServerTemplate that reflects your latest code changes.
    7. Test the cloud server to make sure the scripts were run successfully.  Check the Server's Audit Entries tab for details.
    8. Commit the HEAD ServerTemplate to create a static and tested revision that you can use in a production environment.  Note: When you commit a HEAD ServerTemplate that's referencing a HEAD RepoPath object, you will be prompted to also commit the RepoPath object (default), which is strongly recommended. Be sure to manually lock the repositories to a specific SHA.

    Find a Repository UID

    When you launch a server that uses cookbooks for server configuration purposes, the required cookbook repositories are locally stored on the server instance itself. A unique identifier is assigned to each repository on a running server instance. You will need to know a repository's UID if you want to use the Chef Developer Workflow that uses rsync.  

    Steps

    Launch a server that uses cookbooks for server configuration purposes. 

    Once the server is operational, SSH into the server instance and navigate to the directory where cookbooks are locally stored.  If you are using a server that was launched with a RightImage the cookbooks will be stored in the following location: /var/cache/rightscale/cookbooks

    Run the following command to navigate to the default location:

    # cd /var/cache/rightscale/cookbooks

    Each cookbook repository that is used by the server will be assigned a unique identifer.  Therefore, you will most likely see a list of several cookbook directories.

    # cd /var/cache/rightscale/cookbooks
    
    # ls
    05fe0e9401da911fdb518fe37b3a8c5393e3883d
    12fdb518fe37b3a8c5393e3283d05fe0e9401da9
    b518fe37b3a8c5193e3883d05fe0e9401da912fd
    

    Inside each one you will find a 'cookbooks' directory with all of the repository's cookbooks and recipes.

    # cd 05fe0e9401da911fdb518fe37b3a8c5393e3883d/cookbooks/
    

     

    diag-CookbookStorage-v1.png

    To determine which Repository UID represents which repository, you might want to create a table for yourself:

    Repository Name Repository UID
    rs_premium 05fe0e9401da911fdb518fe37b3a8c5393e3883d
    rs_public 12fdb518fe37b3a8c5393e3283d05fe0e9401da9
    my_custom b518fe37b3a8c5193e3883d05fe0e9401da912fd

    References

    rsync Script Example

    Use this code example to create a RightScript or Chef Recipe.  Add it to the HEAD ServerTemplate as an operational script so that it will be easily accessible for script execution.  Later, when you're ready to commit the ServerTemplate for production use, you can remove it from the ServerTemplate.

    #!/usr/bin/env ruby
    
    # rsync's current directory to your VM
    #
    # You must set CLOUD_KEY in you environment to point to your
    # private SSH key file.
    #
    # Your VM must have the corresponding public SSH key in the
    # ~/.ssh/authorized_keys file.
    #
    # RightScale can place your public key on the VM at boot if you
    # set up your public key in the Dashboard with the 
    # "Use the credentials stored on my computer" option under
    # Settings > User Settings > SSH.
    #
    
    host = ARGV[0] 
    path = ARGV[1]
    raise "Usage: sync_to <host_ip> <dst_path>" unless host && path
    
    SSH_KEY_PATH = ENV["CLOUD_KEY"]
    
    puts "Syncing to #{host}..."
    
    opts = "-avz --exclude=.git -e 'ssh -i #{SSH_KEY_PATH}'"
    cmd = "rsync #{opts} . root@#{host}:#{path}"
    puts "Command: #{cmd}"
    puts `#{cmd}`

    Command Line - Development Machine

    If you want to run the command locally on your development machine, see code example below.

    <need code sample>
    
    rsync from Local Development Machine to Cloud Instance
    rsync -avz --exclude=.git <src_path> <dst_path>
    where <scr_path> is the directory of where the cookbooks are locally stored on your development machine.
          <dst_path> is the directory of where the cookbooks are locally stored on the cloud server instance.
          <HASH> is the Repository UID.
    
    EXAMPLE:
    rsync -avz --exclude=.git /src/cookbooks/ ec2-50-19-197-20.compute-1.amazonaws.com:/var/cache/rightscale/cookbooks/<HASH>/cookbooks
    
    # rsync -avz --exclude=.git /src/cookbooks/ ec2-50-19-197-20.compute-1.amazonaws.com:/var/cache/rightscale/cookbooks/05fe0e9401da911fdb518fe37b3a8c5393e3883d/cookbooks
    
     

    Command Line - Instance

    If you want to run the command on the test instance, SSH into the machine and run the following commands.  See the code example below.

    <need code sample>
    
     
    Powered by MindTouch