Wednesday, December 27, 2006

Goodbye, rubbish hosting provider

I just removed the DNS entry pointing back to my old hosting provider, who destroyed my site so thoroughly. Sometimes you get exactly what you pay for. Note to ISPs: If you're providing a backup service, make sure that the tape drive is connected to the server.

I'm now hosted on a Virtual Dedicated Server with Black Cat Networks: run by Debian developers who really know what they are doing. They also support IPv6.

And no. They don't provide backup services. I'm taking responsibility for that.

Wednesday, October 18, 2006

VOIP over WiFi: very nice

Lesley and I have been lent some Nokia E60 mobile phones to try out. We're making and receiving VOIP calls (via Truphone )over our home wireless router. For me the benefit isn't just the free or cheap calls to landline phones, and free calls to other VOIP phones (once I get this working in the office I'll save a fortune calling home); its the fact that I can actually get coverage in my own house. I get marginal coverage in most of the house, and taking phone calls in the garden is starting to be less appealing now with autumn kicking in.

The service is still in beta; we had some trouble getting the service integrated with the phone - it's an application that installs on the phone with SIP and WiFi configuration, and it was this last part that caused the issue. Our router was also blocking registration to the VOIP service as well. I had to read some FAQ's and re-install a few times.

Your mobile number also gets registered with Truphone so they can forward calls to your GSM number if you don't have WiFi coverage. I think it's well worth looking at if you have a WiFi compatible mobile phone.

Thursday, October 12, 2006

CITCON was awesome

I've been back from San Francisco for a month now. At the weekend I went to CITCON , the Continuous Integration and Testing Conference. And I had a great time.

It started on the Friday night, for registration and the chance to submit session ideas in an open spaces format. Then back in the morning for the conference itself.

I arrived at about 9:30 thanks to weekend engineering work with the train lines, so had a coffee and went to the second slot, and went to a session that was originally called "is ant scalable". The consensus was that Ant was misused as a deployment tool and was just fine for building single projects with no dependencies. Things get more complex when you try to manage dependencies using Ant.

Next I sat in and mainly observed the session on Continuous Code Policing which was interesting: barely anybody had run a build that would fail a given build because of code quality issues - it was seen as something to review and then take up with developers later if they kept reducing the amount of code coverage.

I also caught How to prevent the build from taking over your life, which was interesting. I think we all agreed that CI was still in it's infancy and that tools would improve over time.

We also got to see a preview of Bamboo, Atlassian's new CI server product. Mike Cannon-Brooks wanted feedback on the metrics that the final product would ship with, and there was a very frank discussion about what you could glean from CI metrics anyhow. Jeffrey Frederick (one of the CruiseControl committers) also demonstrated metrics from his product, Agitar Management Dashboard.

Tom and I did a Buildix demo and got some very good feedback on improvements that we could make.

My overall impression was very positive. It was great to meet some of the people from the CruiseControl mailing list, and very nice to find other people who had the same role and concerns. The conference will be held next in Sydney - and it's free!

Tuesday, October 03, 2006

CruiseControl builds my web site

I'm back in London. San Francisco was fantastic: I got to see the Airplane House.

Since I got back last month I've spent some time migrating from my silly hosting provider to a shiny new one. This time I decided that I would play with some new toys , so I built it with:
  • BuildMaster (because life is too short to format html, or do trivial build automation)
  • Rake (because I wanted to explore alternatives to Ant)
  • CruiseControl (I had made an instance anyway)
It's been fun. Cruise calls rake via the new-ish exec builder, and rake does all the work of calling buildmaster to update the checkout, generate the content, install the new site etc. I still need a way to be pass the label from cruise to rake. I also need to write some actual content for the site.

Initial thoughts on Rake: it is refreshing to use; the syntax is easy to migrate to if you've used ruby and/or Make. Rake also has a feature that I miss from Make: it will generate files only if their dependant source files have changed. Nice.

Tuesday, August 15, 2006

Test Driven Systems Administration

I wrote this in 2005. It was lost with my server. Thanks to Ade for the cached copy!

I have been thinking about how to apply Agile to the field of Systems Administration for a while now. I haven't been thinking too hard, mind you; but I have been thinking.

I googled, and found this and this, but nothing that really provided any great insight.

So I decided that a good starting place would be testing. Not monitoring, but testing. I had intended to play with Ruby or Python and extend their test frameworks so I could make assertions about the operating system, e.g. asserting that a process was bound to a socket, or that a server responded correctly. This week, I was asked to work on a project doing some migrations of servers, so it was a nice opportunity to test the theory out.

All the servers had Perl installed, so I chose Test::More as a test framework - again it was installed everywhere, and I didn't need much. So then it was simply:

ok(connect_host($ip,$port)==0,""can_connect_to_foo01"");

and it was good. connect_host is a subroutine to connect to a given host and TCP port using IO::Socket. So now I had tests to make sure that the services were available on the correct IP address. Will definitely try and expand on this.

Saturday, July 15, 2006

Beware the intersection of packaging systems

I used to use Dave Thomas's rublog. I edited the blogs in vi, dropped the files onto my home directory and it just worked. Then my hosting provider destroyed my site, and I switched to using typo. Which worked fine for a few months and as long as I didn't use it. Let me explain:

I was using FreeBSD and it's ports system to keep the system up to date:

bash-2.05b$ pkg_info|grep -i ruby
ruby-1.8.4_8,1 An object-oriented interpreted scripting language
[snip]
rubygem-rails-1.1.2 MVC web application framework
rubygem-rake-0.7.1 Ruby Make
rubygem-redcloth-3.0.3_1 A module for using Textile in Ruby


So that seemed ok. Even though Ruby has a packaging system of it's own, I assumed that the ports system would be good enough. I'd rather have one packaging system to keep my system up to date, so I never ran a rubygem update. All the gems that I needed were in the ports system so I just used portmanager to stay up to date. It didn't help:

Rendering admin/pages/list within layouts/administration
Rendering admin/pages/list


NoMethodError (undefined method `controller_name' for nil:NilClass):
/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/caching.rb:541:in `callback'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/caching.rb:534:in `after'

So when I started setting up my reloaded blog, typo worked. I guess I upgraded some of the ruby components as well:

-bash-2.05b$ pkg_info | grep -i ruby
ruby-1.8.4_8,1 An object-oriented interpreted scripting language
ruby18-fcgi-0.8.6 FastCGI library for Ruby
ruby18-gems-0.8.11 Package management framework for the Ruby language
ruby18-mysql-2.7.1 Ruby module for accessing MySQL databases with a C API like
rubygem-actionmailer-1.2.1 Easy email delivery and testing for Ruby
rubygem-actionpack-1.12.1 Action Controller and Action View of Rails MVC Framework

Here's what I think the issue is: Ruby components are distributed using Gems. The FreeBSD ports system has to encapsulate the gems to get the dependencies:

-bash-2.05b$ find /usr/ports -type d | grep rubygem
/usr/ports/audio/rubygem-mp3info
/usr/ports/databases/rubygem-activerecord
/usr/ports/databases/rubygem-memcache-client
/usr/ports/devel/rubygem-needle
/usr/ports/devel/rubygem-rscm

It's probably time to give up on using Ruby from the ports system. I'm also considering giving up using FreeBSD (because of the Java support), and my hosting provider isn't looking so good either.

This problem won't go away in a hurry. Try installing Eclipse on a Debian system and then adding a plugin as a non-priveledged user. It'll barf because your user process will try and install plugins under /usr. That could be the reason that the Debian eclipse package rotted for so long - why bother fixing the package when you have to install the plugins by hand?

Monday, July 10, 2006

More on why we made Buildix

Chris already blogged about Buildix. I thought I'd share some more of the background, and my take on why it is such a good thing.

Shorter Iteration Zero: When you start an Agile project you need to have some tools in place: SCM, CI etc. While you may not even have code if you're on a greenfield project, you still need to have some basic build system in place to compile, test and possibly deploy your code. This all takes time to do, and it generally involves some trial and error to get all the tools plumbed in together. So: Buildix helps you deliver code faster. You could be able to check in code in minutes. We demonstrated this at the TW London away day by interactively breaking and fixing the demo build during the presentation.

Easier upgrades: One pattern that comes up time and time again is checking all of the tools used into the source control system used on the project. So if you need to set up a new CI Server, you check out the tools from SCM. Fine. But upgrading them always seemed clumsy; especially when it's easy to fork a tool and check it back into SCM. So having a Debian based Linux distribution as the basis for Buildix was key - we can distribute tools as a Debian package, and leverage the APT system in Debian to take care of the mechanics of distributing upgrades and calculating dependencies.

Doing the right thing: We wanted some way of promoting good build and deploy practise in TW without trying to enforce anything or standardise on tools. Buildix is the container for those ideas, although we have a long way to go before we have a good sample of the lessons learned and techniques from all the projects worked on. I hope that we'll address some of this in the next Buildix release. I hope we can also improve on things like deployment to enterprise application servers. Stay tuned.

Working for ThoughtWorks: the good

Simon Stewart wrote about some good and bad things about working for TW. Including travel, which is both good and bad.

You'll be exceptionally lucky or well-placed to avoid travel for work if you work for a consultancy firm. My projects have taken me to the home counties, West London, Poole, and now ... San Francisco. The great thing is, I got to take my family with me - and we're staying 2 blocks from Japan Town .

This could work out nicely.

My hosting provider destroyed my server

It turns out that they didn't verify their backups, as the tape drive wasn't physically connected to the server. While I was waiting to have them make me a new BSD virtual machine, they changed the terms of service, conveniently removing the part about the backup service that they offer.

Trying to complain didn't get a great response:

Well, this system did have a backup drive, that appeard to have not faired so well, and went shamefully unnoticed, this was recent in the last 2 months or so, hence the 2 months credit. Also our TOS always said "Customers should also keep their own backups in case ours arenot available for any reason. We make no guarantees about being able to restore any data at any time. ?" The reason why it was modified is so during this time we can implement a more robust backup system, that has more capabilities and redundancy. If there is anything else please don't hesitate to email me back.

Thanks Jay. Nice one.

It has made me think about the way I have been running my site: not just the choice of hosting company, but also the operating system and blog software. FreeBSD 4.11 has been pretty robust, but the FreeBSD jail system doesn't offer as complete a separation of jail nodes as you can get in User Mode Linux. So I'm moving to blogger while I make up my mind.

Thanks to Ade (and Aggrevator) I have my old blog entries. I'll have a look at them and see if they are worth reposting.