An Hour with Chef

Today I sat down and decided I would block out an hour to try out Chef, an ultrahip, super buzzed "systems integration framework." I really don't have an immediate need for Chef (well at least not that I know of yet!) but I love the idea of managing servers "by writing code, not by running commands" and it's all open source, Ruby goodness under the sheets, so I thought I'd give it a whirl.

I install the gem and head over to the "Configuring Chef" page. Right away I'm informed that working with Mac OS X probably is NOT going to go well out of the box. I do a few google searches and see there are a couple of Mac OS X cookbooks out there, but for this exercise, I decide to stay as close to official Opscode supported code as I can, so I dive in.

I get my ~/solo.rb and ~/chef.json setup per the instructions and now it's time to run chef-solo.

mark-mcspaddens-computer:~ mark$ sudo chef-solo -c ~/solo.rb -j ~/chef.json -r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz
Password:
[Sun, 16 Aug 2009 16:05:47 -0500] INFO: Starting Chef Solo Run
/tmp/chef-solo/cookbooks/ruby/recipes/default.rb:45:in `from_file': undefined method `each' for nil:NilClass (NoMethodError)
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/cookbook.rb:139:in `load_recipe'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/recipe.rb:79:in `include_recipe'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/recipe.rb:64:in `each'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/recipe.rb:64:in `include_recipe'
from /tmp/chef-solo/cookbooks/passenger_apache2/recipes/default.rb:26:in `from_file'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/cookbook.rb:139:in `load_recipe'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/recipe.rb:79:in `include_recipe'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/recipe.rb:64:in `each'
... 16 levels...
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/lib/chef/application.rb:57:in `run'
from /usr/local/lib/ruby/gems/1.8/gems/chef-0.7.8/bin/chef-solo:26
from /usr/local/bin/chef-solo:19:in `load'
from /usr/local/bin/chef-solo:19

Awesome. Well, I was warned. Since this is just an educational exercise for me, and decide to dig in and see what I can find. So I pull up /tmp/chef-solo in TextMate, and look at /tmp/chef-solo/cookbooks/ruby/recipes/default.rb. Ok. I see what's going on here. We're trying to install some ruby packages and my OS isn't listed the case statement. Well I'm a Ruby developer and know I already have all the Ruby libs I'll need, so let's just add an else case with an empty array.

# cookbooks/ruby/recipes/default.rb

# Line 24: Start of case
# Line 45: Add else case
extra_packages = case node[:platform]
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
when "centos","redhat","fedora"
%w{
ruby-libs
ruby-devel
ruby-docs
ruby-ri
ruby-irb
ruby-rdoc
ruby-mode
}
else []
end

I'm pretty confident that will fix our error and not leave us without any libs we'll need, so let's try again this again, only this time we won't tell chef-solo to use the remote cookbook and by default it will use our local one instead. (Which is a big bonus...if I jack things up too bad, we just go back to the remote version.)

mark-mcspaddens-computer:~ mark$ sudo chef-solo -c ~/solo.rb -j ~/chef.json
Password:
[Sun, 16 Aug 2009 16:32:48 -0500] INFO: Starting Chef Solo Run
[Sun, 16 Aug 2009 16:32:54 -0500] INFO: Installing gem_package[stompserver] version 0.9.9
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating directory[/stompserver] at /stompserver
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Setting mode to 755 for directory[/stompserver]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating directory[/stompserver/log] at /stompserver/log
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Setting mode to 755 for directory[/stompserver/log]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating directory[/stompserver/log/main] at /stompserver/log/main
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Setting mode to 755 for directory[/stompserver/log/main]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating template[/stompserver/run] at /stompserver/run
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Setting mode to 755 for template[/stompserver/run]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating template[/stompserver/log/run] at /stompserver/log/run
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Setting mode to 755 for template[/stompserver/log/run]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating a symbolic link from -> /etc/init.d/stompserver for link[/etc/init.d/stompserver]
[Sun, 16 Aug 2009 16:33:03 -0500] INFO: Creating a symbolic link from /stompserver -> /stompserver for link[/stompserver]
[Sun, 16 Aug 2009 16:33:03 -0500] ERROR: service[stompserver] (/tmp/chef-solo/cookbooks/runit/definitions/runit_service.rb line 65) had an error:
No such file or directory - /etc/init.d/stompserver status

Ok...something's going down with stompserver. Quick 'gem list' shows I have the gem. Let's see if the stompserver command works.

mark-mcspaddens-computer:~ mark$ stompserver -h
Usage: stompserver [options]
-C, --config=CONFIGFILE Configuration File (default: stompserver.conf)
-p, --port=PORT Change the port (default: 61613)
-b, --host=ADDR Change the host (default: localhost)
-q, --queuetype=QUEUETYPE Queue type (memory|dbm|activerecord|file) (default: memory)
-w, --working_dir=DIR Change the working directory (default: current directory)
-s, --storage=DIR Change the storage directory (default: .stompserver, relative to working_dir)
-d, --debug Turn on debug messages
-a, --auth Require client authorization
-c, --checkpoint=SECONDS Time between checkpointing the queues in seconds (default: 0)
-h, --help Show this message
mark-mcspaddens-computer:~ mark$ which stompserver
/usr/local/bin/stompserver

Ok I have stompserver, but I'm not getting a link into /etc/init.d/stompserver. So I hack around with trying to pass this directory into the runit_service command in the stompserver default recipe. After a few unsuccessful tries, resort to my old caveman ways and run a command:

sudo ln -s /usr/local/bin/stompserver /etc/init.d/stompserver

Time to run chef-solo again...it spins for a while...and spins for a long while...so I decide to investigate. I know the next command we're trying to hit is "/etc/init.d/stompserver status" So I run that in another Terminal and get:

mark-mcspaddens-computer:~ mark$ /etc/init.d/stompserver status
status
MemoryQueue initialized
TopicManager initialized
QueueManager initialized
Stomp protocol handler starting on 127.0.0.1 port 61613
/usr/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:500:in `start_tcp_server': no acceptor (RuntimeError)
from /usr/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:500:in `start_server'
from /usr/local/lib/ruby/gems/1.8/gems/stompserver-0.9.9/bin/stompserver:34
from /usr/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `call'
from /usr/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `run_machine'
from /usr/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.0/lib/eventmachine.rb:224:in `run'
from /usr/local/lib/ruby/gems/1.8/gems/stompserver-0.9.9/bin/stompserver:17
from /etc/init.d/stompserver:19:in `load'
from /etc/init.d/stompserver:19

[NOTE: After the steps that follow, I realize I probably got this error because chef-solo was still running while I tried it. But it's still always fun to see things step by step, so I'll include my next steps even through they're probably not necessary.]

Super. I'm waaaayyyy past the "not running commands" so I do "sudo gem install eventmachine" and it upgrades me from 0.12.0 to 0.12.8. I kill the original chef-solo command (which is still just hanging) and next, I see if these acts have appeased stompserver.

mark-mcspaddens-computer:~ mark$ /etc/init.d/stompserver status
status
MemoryQueue initialized
TopicManager initialized
QueueManager initialized
Stomp protocol handler starting on 127.0.0.1 port 61613

This sits there for a while, but no errors, and since we're trying to run some kind of server here, I take no news as good news. I kill off the stompserver and decide it's time to try chef-solo again. And it hangs. But I've got an idea.

In one terminal I do "sudo /etc/init.d/stompserver status" and after that is rolling I open a different Terminal and run chef-solo. Looks like we're getting somewhere.

Sun, 16 Aug 2009 17:13:33 -0500] INFO: Starting Chef Solo Run
[Sun, 16 Aug 2009 17:13:34 -0500] INFO: Creating a symbolic link from -> /etc/init.d/stompserver for link[/etc/init.d/stompserver]
[Sun, 16 Aug 2009 17:13:34 -0500] INFO: Creating a symbolic link from /stompserver -> /stompserver for link[/stompserver]
[Sun, 16 Aug 2009 17:13:39 -0500] INFO: Installing package[apache2] version 2.2.10

So I've got stompserver behind me, but this Apache install is taking a bit of time...and then I get.

[Sun, 16 Aug 2009 17:18:08 -0500] ERROR: service[apache2] (/tmp/chef-solo/cookbooks/apache2/recipes/default.rb line 30) had an error:
wrong number of arguments (0 for 1)

Ok. Let's open up the apache2 cookbook and see what's up. I open up /tmp/chef-solo/cookbooks/apache2/recipes/default.rb and start trying a few ways on including "mac_os_x" and my specific system setup in into a few of the case statements, but after about 5 minutes I've still got the same error.

----

And my hour is up*. Kind of a bummer...but not really. To me, this is really kind of cool. Sure I would have loved to get my chef server configured, but getting to crack open a few recipes and even the chef source itself is a really cool exercise.

Next time I go Chefing, I'll probably start with one of the Mac OS X cookbooks out there or fire up my Ubuntu laptop that normally sits idle in the office. Honestly, I would love to see an Opscode endorsed/supported/linked_to Mac OS X cookbook, it would have given me the confidence to start with one of those today. (But then again I may have missed out on some good code spelunking.)

Overall I enjoyed my hour with Chef and look forward to trying this all again at a future date. I'm very surprised at how straight forward the framework seems to be and how easy it was to jump in and get dirty with it.

All this talk about Chef, and cookbooks, and recipes has me hungry. Time for a snack! :)


* A younger less disciplined me would have kept hacking until I got it working, but that really wasn't the point of the exercise. Also, the timestamps will show I went a little longer than an hour. My brother called and we chatted for a bit. Family > Code...most of the time.