Using chef-solo-wrapper for development, testing and debugging
Overview
For users who develop Chef cookbooks, chef-solo-wrapper is a great tool to assist in testing and debugging. It is also handy when a chef run fails and the error needs to be diagnosed in more detail.
This software is new and there may be issues on certain platforms. If you do run into problems of any kind (including needing an enhancement so features work on a specific platform) please let us know on the chef-solo-wrapper issues page so a fix can be pushed.
Goal
After completing this tutorial, you'll see how easy it is to quickly set up a Ruby/Chef environment and do chef-solo runs in most UNIX-like systems.
As an example, we'll use RightScale OSS' RightScale Linux Server RL 5.7 ServerTemplate and run its recipes in both a local virtual machine or desktop as well as a RightScale Server in your favourite public cloud.
Steps
Prepare test machines
This tutorial runs through using chef-solo-wrapper on two different machines to demonstrate its potential, particularly in testing the same run_list in different environments at once. You can of course, choose to decide to do only one.
Launch a RightScale Server
Import RightScale Linux Server RL 5.7 from the MultiCloud Marketplace, add a server to a deployment (such as default), then launch.
Launch a local virtual machine
If you decide to also test locally, launch a virtual machine (e.g. VirtualBox) on your desktop.
This VM can (generally) be any UNIX-like OS/distribution that can support Ruby and Chef.
Of course, you can simply use your desktop computer itself if you don't mind Chef configuring the system (use with caution).
This tutorial should work for most Linux distributions, as well as OS X.
Install Ruby & RubyGems
If you don't have Ruby and RubyGems installed (system level), see the chef-solo-wrapper Quick Start for instructions per platform for each of your test machines, then move on to the express setup below.
For Amazon EC2 and many other platforms, these are commonly pre-installed, you can check quickly first: ruby -v; gem -v.
Note: Using Ruby 1.8.7 >= is strongly recommended. When using an older distro release such as CentOS 5.6, you can upgrade Ruby fairly easily.
Install chef-solo-wrapper
sudo gem install chef-solo-wrapper --no-rdoc --no-ri
Setup defaults
First, run a command to quickly setup Chef Solo and install RestConnection as well some intial configurations:
sudo cs --setup all --defaults
Example output:
chef-solo-wrapper 0.0.15 [Sun Jun 3 17:28:00 UTC 2012] Log level: INFO [Sun Jun 3 17:28:00 UTC 2012] INFO: Setting up Chef Solo. ii chef 0.10.10-2 A systems integration framework, built to bring the benefits of configuration management to your entire infrastructure. [Sun Jun 3 17:28:00 UTC 2012] INFO: [dpkg] Chef already installed, skipping. [Sun Jun 3 17:28:01 UTC 2012] INFO: Setting up rest_connection. [Sun Jun 3 17:28:01 UTC 2012] INFO: Installing rest_connection RubyGem... Building native extensions. This could take a while... Fetching: rest_connection-0.1.10.gem (100%) Successfully installed nokogiri-1.5.3 Successfully installed rest_connection-0.1.10 2 gems installed [Sun Jun 3 17:28:29 UTC 2012] INFO: Setting up /etc/chef/node.json. [Sun Jun 3 17:28:29 UTC 2012] INFO: Setting up /etc/chef/solo.rb. [Sun Jun 3 17:28:29 UTC 2012] INFO: Setup complete.
Test chef-solo-wrapper
First, lets just do a test to ensure that Chef can be successfully used by Ruby via chef-solo-wrapper:
sudo cs --test
In the output, you should see a successful test:
chef-solo-wrapper 0.0.15 [Sun Jun 3 17:29:54 UTC 2012] Log level: INFO [Sun Jun 3 17:29:54 UTC 2012] INFO: Testing require of chef. [Sun Jun 3 17:29:54 UTC 2012] INFO: Test passed.
Secondly, lets do an empty run. The --defaults option (used above) will setup Chef Solo with an 'empty' configuration i.e. no cookbooks or run_list. This means we can already test chef-solo using the wrapper:
sudo cs
Example output:
chef-solo-wrapper 0.0.15 [Sun Jun 3 17:30:19 UTC 2012] Log level: INFO [Sun Jun 3 17:30:20 UTC 2012] INFO: Starting Chef Solo. [Sun, 03 Jun 2012 17:30:20 +0000] INFO: *** Chef 0.10.10 *** [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Run List is [] [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Run List expands to [] [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Starting Chef Run for 01-30o89l8.localdomain [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Running start handlers [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Start handlers complete. [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Chef Run complete in 0.003785 seconds [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Running report handlers [Sun, 03 Jun 2012 17:30:21 +0000] INFO: Report handlers complete
You should see a run similar to the above where there are no recipes in the Run List.
Test a recipe: Install Git
Now lets install Git using Opscode's git cookbook. This is a good test and will also ensure you have git installed for checking out more cookbooks.
sudo cs --install git
Example output:
chef-solo-wrapper 0.0.15 [Sun Jun 3 17:31:08 UTC 2012] Log level: INFO [Sun Jun 3 17:31:08 UTC 2012] INFO: [wget] Fetching git cookbook from GitHub... [Sun, 03 Jun 2012 17:31:12 +0000] INFO: *** Chef 0.10.10 *** [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Setting the run_list to ["recipe[git::default]"] from JSON [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Run List is [recipe[git::default]] [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Run List expands to [git::default] [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Starting Chef Run for 01-30o89l8.localdomain [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Running start handlers [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Start handlers complete. [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Processing package[git-core] action install (git::default line 21) [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Chef Run complete in 0.083887 seconds [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Running report handlers [Sun, 03 Jun 2012 17:31:12 +0000] INFO: Report handlers complete
Test another
Feel like another test just to see another recipe run? This time try the java cookbook.
root@01-30o89l8:~# sudo cs --install java chef-solo-wrapper 0.0.16 [Sun Jun 3 18:19:49 UTC 2012] Log level: INFO [Sun Jun 3 18:19:49 UTC 2012] INFO: [wget] Fetching java cookbook from GitHub... [Sun, 03 Jun 2012 18:19:52 +0000] INFO: *** Chef 0.10.10 *** [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Setting the run_list to ["recipe[java::default]"] from JSON [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Run List is [recipe[java::default]] [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Run List expands to [java::default] [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Starting Chef Run for 01-30o89l8.localdomain [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Running start handlers [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Start handlers complete. [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Processing ruby_block[set-env-java-home] action create (java::openjdk line 36) [Sun, 03 Jun 2012 18:19:53 +0000] INFO: ruby_block[set-env-java-home] called [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Processing ruby_block[update-java-alternatives] action nothing (java::openjdk line 43) [Sun, 03 Jun 2012 18:19:53 +0000] INFO: Processing package[openjdk-6-jdk] action install (java::openjdk line 80) [...]
Setup for RightScale OSS
Next, lets use a handy command line option available with chef-solo-wrapper that will setup Chef Solo and download the same cookbooks used by the RightScale Linux Server RL 5.7 ServerTemplate:
sudo cs --setup config-oss
Your should see 2 repositories check-out from GitHub:
chef-solo-wrapper 0.0.15 [Sun Jun 3 17:32:25 UTC 2012] Log level: INFO [Sun Jun 3 17:32:25 UTC 2012] INFO: Inspecting repos, 'cookbooks_public'... Cloning into 'cookbooks_public'... remote: Counting objects: 1563, done. remote: Compressing objects: 100% (942/942), done. remote: Total 1563 (delta 763), reused 1027 (delta 454) Receiving objects: 100% (1563/1563), 104.06 MiB | 877 KiB/s, done. Resolving deltas: 100% (763/763), done. [Sun Jun 3 17:35:39 UTC 2012] INFO: Inspecting repos, 'cookbooks'... Cloning into 'cookbooks'... remote: Counting objects: 338, done. remote: Compressing objects: 100% (222/222), done. remote: Total 338 (delta 113), reused 233 (delta 69) Receiving objects: 100% (338/338), 119.10 KiB | 167 KiB/s, done. Resolving deltas: 100% (113/113), done. [Sun Jun 3 17:35:42 UTC 2012] INFO: Setup complete.
Checkout (more) Chef Cookbooks
If you plan on using more cookbooks, you can download them easily with chef-solo-wrapper. This examples re-fetches the same two repositories:
sudo cs --fetch git://github.com/flaccid/cookbooks_public.git sudo cs --fetch git://github.com/flaccid/cookbooks.git
Example output:
root@01-30o89l8:~# sudo cs --fetch git://github.com/flaccid/cookbooks_public.git chef-solo-wrapper 0.0.15 [Sun Jun 3 17:36:58 UTC 2012] Log level: INFO [Sun Jun 3 17:36:58 UTC 2012] INFO: Inspecting repos, 'cookbooks_public'... [Sun Jun 3 17:36:59 UTC 2012] INFO: [git] Already up-to-date. root@01-30o89l8:~# sudo cs --fetch git://github.com/flaccid/cookbooks.git chef-solo-wrapper 0.0.15 [Sun Jun 3 17:36:59 UTC 2012] Log level: INFO [Sun Jun 3 17:36:59 UTC 2012] INFO: Inspecting repos, 'cookbooks'... [Sun Jun 3 17:36:59 UTC 2012] INFO: [git] Already up-to-date.
As you can see, this is a basic git clone of flaccid's cookbooks and RightScale cookbooks_public (fork). A git pull will be issued if you have already checked out the repository to the same location.
Alternatively, this can be carried out in the shell manually (also supports pull if the repos already exists):
src_dest="/usr/src/chef-cookbooks" && mkdir -p "$src_dest" && cd "$src_dest"
[ -e "$src_dest/cookbooks_public/.git" ] && ( cd "$src_dest/cookbooks_public" && git pull ) || git clone git://github.com/flaccid/cookbooks_public.git
[ -e "$src_dest/cookbooks/.git" ] && ( cd "$src_dest/cookbooks" && git pull ) || git clone git://github.com/flaccid/cookbooks.git(Re)-Configure solo.rb for Chef Solo
You can edit Chef Solo's configuration files at any time, adding in more paramaters at anytime.
The chef-solo-wrapper enables you to easily edit or overwrite Chef's configurations.
Using this command, you can either paste in your own solo.rb or choose 'e' to edit with a CLI text editor; it will be saved for you!
root@01-30o89l8:~# sudo cs --setup config
chef-solo-wrapper 0.0.15
[Sun Jun 3 17:41:19 UTC 2012] Log level: INFO
[Sun Jun 3 17:41:19 UTC 2012] INFO: Setting up /etc/chef/solo.rb.
Use default solo.rb or edit existing [y/n/e] <enter> ?
e
DEBUG: Using /usr/bin/nano
[Sun Jun 3 17:41:26 UTC 2012] INFO: Setup complete.
Or, just re-instate the default (bare) config:
sudo cs --setup config --defaults
Its also easy to use a Bash heredoc to quickly set one up to taste:
cat <<EOF> /etc/chef/solo.rb
file_cache_path "/var/chef-solo"
cookbook_path [ "/usr/src/chef-cookbooks/cookbooks_public/cookbooks", "/usr/src/chef-cookbooks/cookbooks/cookbooks" ]
json_attribs "/etc/chef/node.json"
EOF(Re)-Configure node.json for Chef Solo
sudo cs --setup node
Chooe 'e' <enter> to edit, a CLI text editor will open in the same way as done with solo.rb, or paste in a full config.
Quick access to the node.json will allow you to change the node attributes as well as the boot recipes/run_list.
chef-solo-wrapper 0.0.15
[Sun Jun 3 09:52:07 PDT 2012] Log level: INFO
[Sun Jun 3 09:52:07 PDT 2012] INFO: Setting up /etc/chef/node.json.
Create empty node.json or edit existing [y/n/e] <enter> ?
e
DEBUG: Using /usr/bin/nano
[Sun Jun 3 09:52:18 PDT 2012] INFO: Setup complete.
Or, re-instate the default config:
sudo cs --setup node --defaults
For this exercise, lets setup /etc/chef/node.json on your test machines to use the same recipes and inputs as your test server.
Example
Use the example below and change any attributes as desired. Nameservers for resolvers only need to be specified if you want to change your existing/default nameservers in /etc/resolv.conf (for dynamic nodes, this is usually setup by DHCP).
Note: if you are performing this tutorial on your normal desktop or workstation, the Chef run will modify your /etc/hosts and /etc/resolv.conf accordingly! By placing in your current hostname, domain name and resolvers, you have effectively configured Chef to manage this node in the same way that RightScale Linux Server configures itself on boot!
cat <<EOF> /etc/chef/node.json
{
"rs_utils": {
"short_hostname":"starbug",
"domain_name":"reddwarf.local",
"timezone":"Australia/Sydney",
"private_ssh_key":"",
"process_list":"",
"process_match_list":"ssh/ssh* cron/cron*"
},
"resolver": {
"nameservers":"",
"search":""
},
"ruby": {
"install_source":"package"
},
"run_list": [ "recipe[rs_utils::default]", "recipe[resolver::default]", "recipe[python::package]", "recipe[ruby::default]" ]
}
EOFRunning Chef Solo
Lets print the configs out to make sure we set these up:
sudo cs --setup show
Example output:
chef-solo-wrapper 0.0.5
* Chef Solo Setup *
/etc/chef/solo.rb:
--
file_cache_path "/var/chef-solo"
cookbook_path [ "/usr/src/chef-cookbooks/cookbooks_public/cookbooks", "/usr/src/chef-cookbooks/cookbooks/cookbooks" ]
json_attribs "/etc/chef/node.json"
--
/etc/chef/node.json:
--
{
"rs_utils": {
"short_hostname":"starbug",
"domain_name":"reddwarf.local",
"timezone":"Australia/Sydney",
"private_ssh_key":"",
"process_list":"",
"process_match_list":"ssh/ssh* cron/cron*"
},
"resolver": {
"nameservers":"",
"search":""
},
"ruby": {
"install_source":"package"
},
"run_list": [ "recipe[rs_utils::default]", "recipe[resolver::default]", "recipe[python::package]", "recipe[ruby::default]" ]
}
--
If you have any issues/errors with the stack, these should be addressed before moving forward.
All good? Lets do a normal run:
sudo cs
You should see a fairly lengthy output due to the number of recipes used:
chef-solo-wrapper 0.0.16 [Sun Jun 3 18:29:23 UTC 2012] Log level: INFO [Sun Jun 3 18:29:24 UTC 2012] INFO: Starting Chef Solo. [Sun, 03 Jun 2012 18:29:24 +0000] INFO: *** Chef 0.10.10 *** [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Setting the run_list to ["recipe[rs_utils::default]", "recipe[resolver::default]", "recipe[python::package]", "recipe[ruby::default]"] from JSON [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Run List is [recipe[rs_utils::default], recipe[resolver::default], recipe[python::package], recipe[ruby::default]] [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Run List expands to [rs_utils::default, resolver::default, python::package, ruby::default] [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Starting Chef Run for 01-30o89l8.localdomain [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Running start handlers [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Start handlers complete. [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Auto-tuning PostgreSQL parameters. Total memory: 590M [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Auto-tuning MySQL parameters. Total memory: 590M [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Setting query_cache_size to: 5M [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Setting query_cache_size to: 472M [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Processing rs_utils_hostname[set_system_hostname] action set (rs_utils::setup_hostname line 27) [Sun, 03 Jun 2012 18:29:25 +0000] INFO: Processing gem_package[dnsruby] action install (resolver::default line 24) [Sun, 03 Jun 2012 18:29:31 +0000] INFO: gem_package[dnsruby] installed version 1.53 [...]
Running with the RightLink Sandbox
Next, lets try running the chef-solo provided in the RightLink sandbox on the server. This can be done by adding an option to the command:
sudo cs --sandbox
We can also use the locally cached cookbooks that RightLink downloaded from Repose by re-configuring the node.json first:
#!/bin/bash -e
unset cookbook_path
rs_cookbook_cache_path=/var/cache/rightscale/cookbooks/default # RL 5.8
#rs_cookbook_cache_path=/var/cache/rightscale/cookbooks # RL =< 5.7
for file in "$rs_cookbook_cache_path"/*
do
cookbook_path="$cookbook_path${cookbook_path:+, }\"$file/cookbooks\"" # assumes cookbooks/ for each
done
cat <<EOF > /etc/chef/solo.rb
file_cache_path "/var/chef-solo"
cookbook_path [ $cookbook_path ]
json_attribs "/etc/chef/node.json"
EOF
# run chef-solo-wrapper again
sudo cs --sandbox
Running with debug and verbose
Verbose
sudo cs -v
Debug
sudo cs --debug
Chef Log Level debug
sudo cs --loglevel debug
This option is extremely useful for debugging errors with chef runs.
The most output available w/ rs sandbox, overriding a RightScale credential as well as overriding the run_list:
SYSLOG_SERVER=syslog.foo.bar sudo cs -v --debug --loglevel debug --sandbox --run "recipe[boto::default]"
External Links
- © Copyright 2015 RightScale Technical Support