tinderbox

If you get more than 10 developers together without tools, there is going to be an explosion. Tinderbox keeps this potentially explosive situation under control.

Essentially, Tinderbox is a detective tool. It allows you to see what is happening in the source tree. It shows you who checked in what (by asking Bonsai); what platforms have built successfully; what platforms are broken and exactly how they are broken (the build logs); and the state of the files that made up the build (cvsblame) so you can figure out who broke the build, so you can do the most important thing, hold them accountable for their actions.

These are the Mozilla projects using Tinderbox (there are many more inside Netscape.) They are:

SeaMonkey

This is our main tinderbox page. SeaMonkey is the project name for the latest incarnation of the mozilla browser. It uses NGLayout (raptor) and the new front end code (xpfe). These tinderbox machines live both in and outside of Netscape.

Mozilla Classic

This is what's left of the old version of mozilla before we made the change to nglayout and xpfe. This branch is mostly dead.

The rules of development for Mozilla are as follows:

• If there are any flaming builds, you can't check in code except to fix the build.

• If you check in code, you must watch the builds until they are green on all platforms.

• You can't check in and hope it works or plan to have someone else fix it. You should be very surprised if a build breaks.

• Get help with platforms you don't have access to before checking in changes. Don't checkin if you aren't sure it will build on all platforms.

• If you break these rules too often you won't be allowed to check in anymore. We're serious.

• If you break the tree, and someone has to come after you to fix it, you owe that someone a bribe of their choice. The traditional bribe to the release team has been tequila (but absolutely no Jose Cuervo.) Cookies and donuts make adequate substitutions.

How does it work?

There is a flock of dedicated build machines that do nothing but continually check out the source tree and build it, over and over again. When a build finishes, each machine sends a message (via email) to the Tinderbox server. Tinderbox saves copies of the build logs and some interesting build state (the project, whether each build succeeded, etc.) Tinderbox then runs a Bonsai query to figure out whose changes went into that build.

The Tinderbox web pages organize all this information for you in one place.

How do I use it?

When you check in code, your name will appear in the guilty column. When there are successful (green) builds in all columns in a row higher than your name, you know you are ok: you did not break the build.

The tree is broken, how do I find out what's wrong?

Clicking `L' in the first red box above a green box (that is, the first build to break) will show you a build log for the broken build. You can also click `C' in this box to see exactly what code was checked in since the previous build.

The tree is on fire, and it's not my fault, and I would like to get some work done. What do I do?

One thing you can do is check your CVS tree by date: look backward in the Tinderbox log until you see green. Note the time at which the tree built correctly. Update your tree as it was at that date (with the ``cvs update -D date'' option.) That should get you a build that works. (However, you won't be able to check in until the tree is again green.)

What do ``Clbr'' and ``Dep'' mean?

``Clobber'' builds are checked out and compiled from scratch. ``Depend'' builds are updated and rebuilt in the same directory, and incrementally rebuilt. (So ``depend'' builds complete faster, but we do both to make sure our dependencies are right, and out of sheer paranoia.)

You build on platforms Foo, Bar, and Baz, but my favorite platform is Quux, will you build on that too?

Sure. All you have to do is donate us a machine, and we'll add it to the list! Contact Daniel Nunes for information on setting up your own external tinderbox.

Tinderbox and Bonsai are great, can I have the source so that I can run them on my own projects?

The source code is available on our CVS server. You can check it out by doing

    cvs checkout mozilla/webtools/tinderbox
    cvs checkout mozilla/webtools/bonsai

Please see our CVS page for information on using our CVS server. (The Tinderbox and Bonsai sources are currently only available via CVS.)

Note: the source code to Tinderbox is not pretty. It won't be easy to get it working at your site. See the README file within the Tinderbox sources for more extensive disclaimers and apologies.