Intermediate 1 Chef Tutorial

Create a Deployment

Create a new deployment that you will use to launch a server later in this tutorial. (e.g. Intermediate 1 Chef Tutorial).  See Create a New Deployment.

Create a Chef Development Server

The next step is to create a Chef development environment for developing new Chef recipes. You can either install Chef on your local machine if you want to use it for development purposes or launch a server in the cloud for the same purpose. For this tutorial you will use the latter option, however you may want to eventually set up your local machine to act as your Chef development server once you become more comfortable with Chef. 

Fortunately, you can import a ServerTemplate from the MultiCloud Marketplace that's specifically designed to launch and configure a server in the cloud that you can use for Chef development purposes. The ServerTemplate contains scripts that install and configure Chef, Knife, and Git.

1. Go to the MultiCloudMarketplace (Design > MultiCloud Marketplace > ServerTemplates) and import the following ServerTemplate:
   • "Chef Development Environment (CDE)" published by "RightScale Open Source"
2. Add a server into the new deployment using the latest revision of the imported ServerTemplate. Select or create any required cloud resources. For this tutorial, please launch an EC2 instance. Therefore, you will need to select an SSH key and Security Group. Make sure the security group has TCP ports 22 and 80 open to any IP. (Tip: If you create a new Security Group from the wizard, it will automatically have the appropriate permissions to complete this tutorial.) Name the server accordingly.  (e.g. My Chef Dev) See the Add Server Assistant page if you need help adding a server to a deployment. 
3. Once the server has been added to the deployment, launch the server.
4. At the input confirmation page, provide the following values for the required inputs.
   • Git email address - The email address associated with your GitHub account. (e.g. text: user@example.com)
   • Git SSH Key used for read/write access - Select the same EC2 SSH key that you used to create the server. Select 'key' for the input type and select the appropriate key from the dropdown list. (e.g. key: my-key) 
   • Git user name - Enter your Git username. (e.g. text: username)
   • Tip:  Log into your Git account to find your associated email address under Account Settings.  (Upper right, wrench/screwdriver icon.)
5. (Optional) If you already have a Chef development repository that you would like to use, you can click the "Show Advanced" link for the Git category and provide values for the following inputs.
   • Git branch - Enter the name of the Git branch you want to pull code from. For this tutorial, point to the master branch. (e.g. master)
   • Git repository - The url to your Git repository where you have read + write access. (e.g. text: https://username@github.com/username/intermediate-tutorial.git)
6. Click Save and Launch button.

In a few minutes you will have a customized server in the cloud that you can use for developing new Chef cookbooks and recipes. Later in this tutorial you will create new scripts and push the changes up to your cookbook repository so that the cookbooks on your development server and your software repository will be in-sync.

Tip:  No need to wait until your server goes operational to continue with this tutorial.  You can proceed to the next step immediately.

Create a Chef Cookbook Repository

You are now ready to start your Chef development.

Although RightScale supports different types of source control management systems for hosting Chef repositories, most users will find it useful to initially use GitHub since it's commonly used in the Chef community. In fact, you'll find all of RightScale's own cookbooks on GitHub (https://github.com/rightscale), which will make it easier for you to view and fork our cookbook repositories. 

1. Log into your user account on GitHub. (http://github.com)
    
2. Go to the Dashboard view of your account by clicking your username in the top navigation bar.
    
3. Click the Create a new repo button in the upper right corner.
   
    
4. Provide details about the new repository.
   
   • Repository name - Enter a unique name for the repository. (e.g. my_repository) 
   • Description (optional) - Enter a short description for the repository and select the type of repository.
     • Public - Anyone can view the contents of the repository
     • Private - The contents of the repository are not publicly visible on GitHub.com. Requires a paid account.
   • Important! Leave the "Initialize this repository with a README" checkbox unchecked.
5. Click the Create repository button.
   Note: If you are using the "Chef Development Environment (CDE)" ServerTemplate in the context of the Intermediate 1 Chef Tutorial, ignore the suggested setup steps because you will set up your repository a different way. 
   

Create a Git SSH Key

In order to push code to a GitHub repository or pull code from a private repository, an SSH key is used for authentication purposes. For this tutorial, you will use the EC2 SSH key as your Git SSH key. Remember, you selected your EC2 SSH key when you launched the server. You will use the public key as your "Git SSH Key" so that you will be able to push changes from your Chef development server to your GitHub repository.

1. SSH into the running server and switch to the root user so that you will be allowed to navigate to the .ssh directory.

   • # sudo -i

2. Run the following command to retrieve the public SSH key of the EC2 SSH key that you used to launch the server.

   • # cat ~/.ssh/*.pub

3. Copy the contents of the public key.
4. Log into GitHub and go to your Account Settings > SSH Keys.
5. Click the Add New SSH Key button. Provide a name for the SSH Key and paste the contents of your public key into the "Key" field. (view screenshot)
6. Click Add Key. Note: You will be prompted to enter your GitHub password for authentication purposes.

Create a working directory for Chef development

You are now ready to set up a working directory on your Chef development server  For this tutorial, you are going to start by cloning a base repository that already contains the basic structure and core components of a plain repository. In the next section you will create a new cookbook and recipe, and then push your changes to your GitHub repository.

First, let's create a working directory by cloning an existing repository. When you clone a repository, all of its contents are copied into your local working directory.

1. SSH into the running "Development" server.
2. Create a local working directory where you will store the new cookbooks and recipes that you are about to create. Run the following command:

   • # mkdir -p /opt/development
     

3. The easiest way to create a new cookbook repository is to clone an existing one. Run the following command to clone a basic cookbook repository and save its contents into your working directory.

   • # git clone git://github.com/opscode/chef-repo.git /opt/development

4. Change into your Chef working directory.

   • # cd /opt/development
     

5. Run the following command to view the remote aliases. A remote is an alias to a remote repository. Since you created this repository by cloning an existing one, you inherit its remote, "origin" which points to the 'read-only' repository (git://github.com/opscode/chef-repo.git) where you pulled code from using the clone command.

   • # git remote
     origin

6. Since you don't have access to push changes to the sample repository from Opscode, you'll need to update the 'origin' remote to point to your own repository where you have 'read-write' access, so you'll be able to push changes that you make to your cookbooks on the Chef development server to your git repository. First, rename the current remote (to 'opscode').

   • # git remote rename origin opscode
     # git remote
     opscode 

7. Now you can create a new 'origin' remote that points to your own repository where you have 'read-write' access. (e.g. git@github.com:<username>/<repository_name>.git) To view where you can find the appropriate URL information so that you can construct the following command, go to your GitHub repository. You must use the SSH (not HTTP) URI since you can only push changes via SSH. (view screenshot) You should now have two remotes.

   • # git remote add origin git@github.com:<username>/<repository_name>.git
     # git remote
     opscode
     origin

8. Push the contents of your working directory into the "master" branch of the GitHub repository that the 'origin' remote is pointing to (git@github.com:<username>/<repository_name>.git).

   • # git push origin master

9. You can now go to GitHub and view the contents of your repository. (view screenshot)

Create a new Cookbook and Recipe

You can now create a new cookbook in your Chef development directory.

1. SSH into the running server.
2. Change into the cookbooks directory of your Chef development repository. If you look in the cookbooks directory, you'll notice that you only have a single readme file. At this point there are no cookbooks in your working directory.

   • # cd /opt/development/cookbooks
     # ls
     README.MD

3. Since the Knife command utility is already installed on the instance, you can run the following command to create a new cookbook with the name of your choice (e.g. intermediate) in your local working directory. Use a simple name for your cookbook. It's recommended that you use only lower-case letters and a single word. If you use multiple words, you must use underscore to separate multiple words. (e.g. intermediate_example) Do not use spaces inbetween words because git will assume the first word is your cookbook name.

   • # knife cookbook create <cookbook_name> -o /opt/development/cookbooks
     ** Creating cookbook <cookbook_name>
     ** Creating README for cookbook: <cookbook_name>
     ** Creating metadata for cookbook: <cookbook_name>

4. If you view the contents of the cookbooks directory, you should now see the cookbook that you just created. The path to your new cookbook should be /opt/development/cookbooks/<cookbook_name>

   • # ls
     README.MD   <cookbook_name>

5. The benefit of creating a cookbook using Knife is that it automatically creates the basic components of a Chef cookbook. Check out the contents of your new cookbook.

   • # cd <cookbook_name>
     # ls
     attributes   files      metadata.rb  README.md  resources
     definitions  libraries  providers    recipes    templates
     

6. Now you can create a new recipe in the 'recipes' directory. (/opt/development/cookbooks/<cookbook_name>/recipes) When you create a cookbook using Knife, a "default" recipe is automatically created for you in the 'recipes' directory.

   • # cd recipes
     # ls
     default.rb

7. For this tutorial, you're going to simply modify the existing default.rb file to be a simple "hello world" example.  You'll use a vi editor to change the contents of the file.

   • # vi default.rb

8. Add a simple log command. (Type 'Shift+i' to insert text.  Type 'Esc' and then 'Shift + q' to exit editing mode. Type 'wq' to save and exit the vi editor.)

   • #
     # Cookbook Name:: intermediate
     # Recipe:: default
     #
     # Copyright 2012, YOUR_COMPANY_NAME
     #
     # All rights reserved - Do Not Redistribute
     #
     
     log "Hello World! You should try Chef with RightScale!"

9. The next step is to update the Chef metadata so that RightScale will be able to properly discover and display its attributes in the dashboard. Each cookbook should have a metadata.rb file.

   • # cd /opt/development/cookbooks/<cookbook_name>
     # ls
     attributes   files      metadata.rb  README.md  resources
     definitions  libraries  providers    recipes    templates
     # vi metadata.rb
     

10. You must now declare the new recipe in your metadata.rb file. You must provide the location of the recipe and a description using the following syntax: recipe "<cookbook_name>::<recipe_name>","<recipe_description>" (Type Esc. Type Shift + q to exit editing mode. Type 'wq' to save and exit the vi editor.)

    • maintainer       "YOUR_COMPANY_NAME"
      maintainer_email "YOUR_EMAIL"
      license          "All rights reserved"
      description      "Installs/Configures intermediate"
      long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
      version          "0.0.1"
      
      recipe "<cookbook_name>::default","Prints hello world output"    

11. Next, you'll add all of the new files that you added to /opt/development/cookbooks. Use the git status command to view the list of new files that you're going to push to your repository.

    • # git add /opt/development/cookbooks
      # git status
      
      # On branch master
      # Changes to be committed:
      #   (use "git reset HEAD <file>..." to unstage)
      #
      #       new file:   CHANGELOG.md
      #       new file:   README.md
      #       new file:   metadata.rb
      #       new file:   recipes/default.rb
      #
      

12. You must now commit the changes that you added. Provide a basic description for the commit action.

    • # git commit -m "Added a hello world message to the default.rb recipe"
      [master 01e7613] Added a hello world message to the default.rb recipe
       4 files changed, 105 insertions(+), 0 deletions(-)
       create mode 100644 cookbooks/intermediate/CHANGELOG.md
       ​create mode 100644 cookbooks/intermediate/README.md
       create mode 100644 cookbooks/intermediate/metadata.rb
       create mode 100644 cookbooks/intermediate/recipes/default.rb
      

13. Before you push your changes to your GitHub repository, view your repository on GitHub first.  (https://github.com/<username>/<repository_name>/tree/master/cookbooks) Notice that there are currently zero cookbooks in the repository. Now run the following command to push your changes into the "master" branch of the repository:

14. # git push origin master
    Counting objects: 232, done.
    Compressing objects: 100% (116/116), done.
    Writing objects: 100% (232/232), 40.00 KiB, done.
    Total 232 (delta 81), reused 223 (delta 80)
    To git@github.com:<username>/<repository_name>.git
    * [new branch]      master -> master

15. Refresh your browser window of your GitHub repository and you should now see your new cookbook along with all of its contents.
    
    

Import Cookbooks

The first step is to import the new cookbook into your RightScale account.

1. Go to Design > Repositories > New.
2. Add the following sample repository that contains a few prebuilt cookbooks that you can use for this tutorial.
   
   • Name - Provide a user-friendly name. (e.g. Intermediate Tutorial)
   • Description - Provide a brief description about the repository. 
   • Type - Select 'Git' from the drop-down menu.
   • Tag/Branch - Enter 'master' for this value.
   • URL - Enter 'git://github.com/<username>/<repository_name>.git' 
   • Git SSH Key - Ignore this field because the cookbooks are in a publicly visible repository. Note: Git SSH Keys are only required for retrieving cookbooks from private repositories.
   • Cookbook Paths - Enter 'cookbooks' for this value and click Add.
3. Since you want to use the cookbooks in this repository, keep the checkbox checked (default) to automatically import all of the repository's cookbooks to the Primary Namespace. 
4. Click OK. You should now see your repository listed as well as its cookbooks. Note: You may need to wait a few minutes before the contents of a repository can be scraped and listed. 
   

Import and Clone the Base ServerTemplate

The next step is to create a new custom ServerTemplate that uses your own Chef cookbooks and recipes that are hosted in a software repository. You will use the custom ServerTemplate to create and launch a server for testing purposes.

In this tutorial, you will start with RightScale's "Base" ServerTemplate, which is recommended for developing ServerTemplates from scratch.

1. Go to the MultiCloudMarketplace (Design > MultiCloud Marketplace > ServerTemplates) and import the following ServerTemplate:
   • "Base ServerTemplate for Linux"  published by "RightScale"
2. Clone the ServerTemplate and rename it accordingly. (e.g. My Custom App)
3. Commit the ServerTemplate, so that the first committed revision of the ServerTemplate matches the original version that it was cloned from. Leave a simple commit message. (e.g. original version)

Attach a Cookbook to the ServerTemplate

1. Go the Scripts tab of the editable HEAD version of the cloned ServerTemplate.
2. Click Modify.
3. Click Attach Cookbooks.
4. Find and select the cookbook that you just created (e.g. 'intermediate'), which you imported into the RightScale account in a previous step. If more than one version of the cookbook is listed, select the version that is in the "primary" namespace and click Attach Selected.
5. Drag and drop the "default" Chef recipe (e.g. intermediate::default) from the attached cookbook into the Operational Scripts list.
   
   

Create a Chef Test Server

1. Go to the the HEAD version of the cloned ServerTemplate. (e.g. My Custom App) 
2. Use the HEAD version of the cloned ServerTemplate to add a server into the same deployment as the Chef development server. Name the server accordingly. (e.g. My Chef Test)  You can use the same cloud resources (e.g. SSH Key, Security Group) and settings that you used to launch the Chef development server.
   (Note: If you are actively developing and testing cookbooks and recipes, it's helpful to launch your test server with a HEAD version of a ServerTemplate so that you can easily add and test new scripts.
3. Launch the test server. Since all of the required inputs have acceptable default values, there are no missing inputs that you have to specify prior to launching the instance.

In a few minutes you will have a Chef development environment that consists of two servers running in the cloud.

• Chef Development Server - Used to create new Chef cookbooks and recipes.
• Chef Test Server - Used to test the Chef cookbooks and recipes. Once you are satisfied with your additions you can commit the ServerTemplate and have your quality assurance perform any final regression tests before publishing it to the MultiCloud Marketplace or using it in a production environment.

Test a Chef Recipe

1. Wait for the "Test" server to become operational.
2. Go to the "Test" server's Scripts tab and run the "default" Chef recipe that you previously attached as an operational script. 
3. Go to the "Test" server's Audit Entries tab to view the output. Wait for the script to be successfully executed (e.g. completed: intermediate::default) and then click on the audit entry. 
   

Modify a Chef Recipe

The next step in a Chef development cycle is to modify an existing recipe and re-test the script.

In the following steps, you will go through a basic Chef development workflow, which is highlighted in the diagram below.
 

1. SSH into your "Development" server.
2. Open up the "default.rb" file in your custom cookbook with the vi editor.

   • # cd /opt/development/cookbooks/<cookbook_name>
     # cd recipes
     # ls
     default.rb
     
     # vi default.rb
     

3. As a best practice, you should always use the "RightScale Marker" definition to mark the beginning of a script so that it can be more clearly displayed in an Audit Entry. The marker will also make it easier to troubleshoot a Chef run list that consists of a series of several scripts, such as a boot script sequence. In the vi editor, add the 'rightscale_marker' to the code.

   • rightscale_marker
     log "Hello World! You should try Chef with RightScale!"

4. The 'rightscale_marker' is a Chef resource developed by RightScale. You can find the code for this Chef definition in RightScale's 'cookbooks/rightscale_cookbooks' repository. (view script) Since the resource is located in a different cookbook, you will need to declare this cookbook dependency in the Chef metadata. You must now go up one directory and edit the metadata.rb file. (/opt/development/cookbooks/<cookbook_name>/metadata.rb)

   • # cd ..
     # ls
     attributes   files      metadata.json  providers  recipes    templates
     definitions  libraries  metadata.rb    README.md  resources
     # vi metadata.rb
     

5. In the vi editor, you must add the following line (highlighted in green) to make sure that the following cookbook ("rightscale") will also be retrieved as well as the cookbook where the current recipe is stored so that the code for the "rightscale_marker" definition is available when the default.rb recipe is run, otherwise the script will fail because it cannot find one of the referenced Chef resources. Save the changes to the metadata.rb file and exit.

   • maintainer       "YOUR_COMPANY_NAME"
     maintainer_email "YOUR_EMAIL"
     license          "All rights reserved"
     description      "Installs/Configures intermediate"
     long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
     version          "0.0.1"
     
     depends "rightscale"
     
     recipe "intermediate::default","Prints hello world output" 
     

6. You are now ready to push your changes from your local Chef development machine to your repository on GitHub. Run a 'git status' command to see what files were created or modified.

   • # git status
     # On branch master
     # Your branch is ahead of 'opscode/master' by 1 commit.
     #
     # Changed but not updated:
     #   (use "git add <file>..." to update what will be committed)
     #   (use "git checkout -- <file>..." to discard changes in working directory)
     #
     #       modified:   metadata.rb
     #       modified:   recipes/default.rb
     #

7. When you push your changes back to your GitHub repository, you will add the changes, create a commit of those changes, and finally push the changes. As a shortcut, you can perform the add and commit actions at the same time using the '-am' option. Then push the changes.

   1. # git commit -am "Added rightscale_marker to default.rb"
      [master 886374b] Added rightsale_marker to default.rb
       2 files changed, 4 insertions(+)
       
      # git push origin
      Counting objects: 13, done.
      Compressing objects: 100% (6/6), done.
      Writing objects: 100% (7/7), 668 bytes, done.
      Total 7 (delta 3), reused 0 (delta 0)
      To git@github.com:<username>/<repository_name>.git
         01e7613..886374b  master -> master
      

8. Now the code on your local Chef development machine matches the code in your GitHub repository. Go to GitHub and check your cookbook repository to verify that the files accurately reflect your changes.

Refetch and Import

Now that you pushed new code to GitHub, you need to update your RightScale account so that you have access to the latest version of your cookbook in your ServerTemplate.

1. Go to Design > Repositories.
2. Find and click on the repository that you created earlier. (e.g. Chef Tutorial)
3. Click the Refetch and Import button, which refetches the latest code from your GitHub repository and updates the cached version in RightScale's Repose service and imports all of the cookbooks from the repository into your RightScale account.
   
4. Wait for the refreshing of the repository to be 100% complete before proceeding to the next step. You want make sure that the cached copy of the cookbook in Repose reflects your most recent change and matches the version in your source repository (e.g. GitHub). You can track the progress of the refetch action under the Events pane. 
   

Re-test the Chef Recipe

You will now execute the modified script on your "Test" server.

1. Go to the "Test" server's Scripts tab and re-run the updated script. (e.g. <cookbook_name>::default)
2. Once the script is complete, go to the server's Audit Entry tab to see the output of the script. Notice how the script is more clearly highlighted in the audit entry. The 'rightscale_marker' is used to group the output related to a specific script in a more manageable format, which is especially useful in troubleshooting situations where you want to quickly navigate to the script that failed and see the related output.
    
   

Terminate the Servers

Congratulations! You have successfully completed this tutorial. You learned how to create and update a cookbook and recipe, update the metadata, push changes to GitHub and test the script on a "Test" server. You can now terminate the servers.