Monday, July 30, 2007

The pipeline of doom

Simon wrote recently about the Build Pipeline. His enthusiasm for the technique is infectious, but I have to add a note of caution. The big idea is to reduce the feedback cycle for developers ("the unit and db tests passed on CruiseControl, our checkin looks okay") while still allowing slower automated acceptance or regression tests to run afterward. Of course, the entire team needs to treat the second build with as much reverence as the first.

I haven't always found that to be the case. So I wouldn't recommend this technique unless the team already has enough discipline to keep a Continuous Integration build green, for starters.

The other aspect of the pipeline that I find troubling is this: by not forcing the developers to sit through the functional tests before they check in their code, you prevent their fantastic brains from reflecting on improving them. If people are feeling the pain of running them, they have a good motivation to fix them. Those tests ought to provide the biggest bang for your buck: unless you're careful you could be running poor tests dozens of times a day.

Don't simply push those tests down the pipeline, but fix them where you can, and keep the feedback loop as fast as you can for the entire team. After all, there's no silver bullet.


Miles said...

I have noticed the same problem on teams that I have worked with.

What if the most recent functional tests were automatically noted and executed in the short build. Those could be done in the short build, and the full set of functional, regression, etc, could be done later. Is there already a tool for that?

Julian said...

Hi Miles. Good to hear from you. Did you see this post from Josh Graham? When I get back from vacation I'm going to try it out at my current project ...