Saturday, July 15, 2006

Beware the intersection of packaging systems

I used to use Dave Thomas's rublog. I edited the blogs in vi, dropped the files onto my home directory and it just worked. Then my hosting provider destroyed my site, and I switched to using typo. Which worked fine for a few months and as long as I didn't use it. Let me explain:

I was using FreeBSD and it's ports system to keep the system up to date:

bash-2.05b$ pkg_info|grep -i ruby
ruby-1.8.4_8,1 An object-oriented interpreted scripting language
[snip]
rubygem-rails-1.1.2 MVC web application framework
rubygem-rake-0.7.1 Ruby Make
rubygem-redcloth-3.0.3_1 A module for using Textile in Ruby


So that seemed ok. Even though Ruby has a packaging system of it's own, I assumed that the ports system would be good enough. I'd rather have one packaging system to keep my system up to date, so I never ran a rubygem update. All the gems that I needed were in the ports system so I just used portmanager to stay up to date. It didn't help:

Rendering admin/pages/list within layouts/administration
Rendering admin/pages/list


NoMethodError (undefined method `controller_name' for nil:NilClass):
/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/caching.rb:541:in `callback'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/caching.rb:534:in `after'

So when I started setting up my reloaded blog, typo worked. I guess I upgraded some of the ruby components as well:

-bash-2.05b$ pkg_info | grep -i ruby
ruby-1.8.4_8,1 An object-oriented interpreted scripting language
ruby18-fcgi-0.8.6 FastCGI library for Ruby
ruby18-gems-0.8.11 Package management framework for the Ruby language
ruby18-mysql-2.7.1 Ruby module for accessing MySQL databases with a C API like
rubygem-actionmailer-1.2.1 Easy email delivery and testing for Ruby
rubygem-actionpack-1.12.1 Action Controller and Action View of Rails MVC Framework

Here's what I think the issue is: Ruby components are distributed using Gems. The FreeBSD ports system has to encapsulate the gems to get the dependencies:

-bash-2.05b$ find /usr/ports -type d | grep rubygem
/usr/ports/audio/rubygem-mp3info
/usr/ports/databases/rubygem-activerecord
/usr/ports/databases/rubygem-memcache-client
/usr/ports/devel/rubygem-needle
/usr/ports/devel/rubygem-rscm

It's probably time to give up on using Ruby from the ports system. I'm also considering giving up using FreeBSD (because of the Java support), and my hosting provider isn't looking so good either.

This problem won't go away in a hurry. Try installing Eclipse on a Debian system and then adding a plugin as a non-priveledged user. It'll barf because your user process will try and install plugins under /usr. That could be the reason that the Debian eclipse package rotted for so long - why bother fixing the package when you have to install the plugins by hand?

Monday, July 10, 2006

More on why we made Buildix

Chris already blogged about Buildix. I thought I'd share some more of the background, and my take on why it is such a good thing.

Shorter Iteration Zero: When you start an Agile project you need to have some tools in place: SCM, CI etc. While you may not even have code if you're on a greenfield project, you still need to have some basic build system in place to compile, test and possibly deploy your code. This all takes time to do, and it generally involves some trial and error to get all the tools plumbed in together. So: Buildix helps you deliver code faster. You could be able to check in code in minutes. We demonstrated this at the TW London away day by interactively breaking and fixing the demo build during the presentation.

Easier upgrades: One pattern that comes up time and time again is checking all of the tools used into the source control system used on the project. So if you need to set up a new CI Server, you check out the tools from SCM. Fine. But upgrading them always seemed clumsy; especially when it's easy to fork a tool and check it back into SCM. So having a Debian based Linux distribution as the basis for Buildix was key - we can distribute tools as a Debian package, and leverage the APT system in Debian to take care of the mechanics of distributing upgrades and calculating dependencies.

Doing the right thing: We wanted some way of promoting good build and deploy practise in TW without trying to enforce anything or standardise on tools. Buildix is the container for those ideas, although we have a long way to go before we have a good sample of the lessons learned and techniques from all the projects worked on. I hope that we'll address some of this in the next Buildix release. I hope we can also improve on things like deployment to enterprise application servers. Stay tuned.

Working for ThoughtWorks: the good

Simon Stewart wrote about some good and bad things about working for TW. Including travel, which is both good and bad.

You'll be exceptionally lucky or well-placed to avoid travel for work if you work for a consultancy firm. My projects have taken me to the home counties, West London, Poole, and now ... San Francisco. The great thing is, I got to take my family with me - and we're staying 2 blocks from Japan Town .

This could work out nicely.

My hosting provider destroyed my server

It turns out that they didn't verify their backups, as the tape drive wasn't physically connected to the server. While I was waiting to have them make me a new BSD virtual machine, they changed the terms of service, conveniently removing the part about the backup service that they offer.

Trying to complain didn't get a great response:

Well, this system did have a backup drive, that appeard to have not faired so well, and went shamefully unnoticed, this was recent in the last 2 months or so, hence the 2 months credit. Also our TOS always said "Customers should also keep their own backups in case ours arenot available for any reason. We make no guarantees about being able to restore any data at any time. ?" The reason why it was modified is so during this time we can implement a more robust backup system, that has more capabilities and redundancy. If there is anything else please don't hesitate to email me back.

Thanks Jay. Nice one.

It has made me think about the way I have been running my site: not just the choice of hosting company, but also the operating system and blog software. FreeBSD 4.11 has been pretty robust, but the FreeBSD jail system doesn't offer as complete a separation of jail nodes as you can get in User Mode Linux. So I'm moving to blogger while I make up my mind.

Thanks to Ade (and Aggrevator) I have my old blog entries. I'll have a look at them and see if they are worth reposting.