FreeBSD Developers' Handbook
Prev | Chapter 6 Regression and Performance Testing | Next

# 6.2. The FreeBSD Source Tinderbox

The source Tinderbox consists of:

- A build script, `tinderbox`, that automates checking out a specific version of the FreeBSD source tree and building it.

- A supervisor script, `tbmaster`, that monitors individual Tinderbox instances, logs their output, and emails failure notices.

- A CGI script named `index.cgi` that reads a set of tbmaster logs and presents an easy-to-read HTML summary of them.

- A set of build servers that continually test the tip of the most important FreeBSD code branches.

- A webserver that keeps a complete set of Tinderbox logs and displays an up-to-date summary.

The scripts are maintained and were developed by Dag-Erling C. Smørgrav `<des@FreeBSD.org>`, and are now written in Perl, a move on from their original incarnation as shell scripts. All scripts and configuration files are kept in /projects/tinderbox/.

For more information about the tinderbox and tbmaster scripts at this stage, see their respective man pages: tinderbox(1) and tbmaster(1).

## 6.2.1. The `index.cgi` Script

The `index.cgi` script generates the HTML summary of tinderbox and tbmaster logs. Although originally intended to be used as a CGI script, as indicated by its name, this script can also be run from the command line or from a cron(8) job, in which case it will look for logs in the directory where the script is located. It will automatically detect context, generating HTTP headers when it is run as a CGI script. It conforms to XHTML standards and is styled using CSS.

The script starts in the `main()` block by attempting to verify that it is running on the official Tinderbox website. If it is not, a page indicating it is not an official website is produced, and a URL to the official site is provided.

Next, it scans the log directory to get an inventory of configurations, branches and architectures for which log files exist, to avoid hard-coding a list into the script and potentially ending up with blank rows or columns. This information is derived from the names of the log files matching the following pattern:

    tinderbox-$config-$branch-$arch-$machine.{brief,full}

The configurations used on the official Tinderbox build servers are named for the branches they build. For example, the `releng_8` configuration is used to build `RELENG_8` as well as all still-supported release branches.

Once all of this startup procedure has been successfully completed, `do_config()` is called for each configuration.

The `do_config()` function generates HTML for a single Tinderbox configuration.

It works by first generating a header row, then iterating over each branch build with the specified configuration, producing a single row of results for each in the following manner:

- For each item:

  - For each machine within that architecture:

    - If a brief log file exists, then:

      - Call `success()` to detemine the outcome of the build.

      - Output the modification size.

      - Output the size of the brief log file with a link to the log file itself.

      - If a full log file also exists, then:

        - Output the size of the full log file with a link to the log file itself.

    - Otherwise:

      - No output.

The `success()` function mentioned above scans a brief log file for the string tinderbox run completed in order to determine whether the build was successful.

Configurations and branches are sorted according to their branch rank. This is computed as follows:

- `HEAD` and `CURRENT` have rank 9999.

- `RELENG_`x`` has rank `xx`99.

- `RELENG_`x`_`y`` has rank `xxyy`.

Tthis means that `HEAD` always ranks highest, and `RELENG` branches are ranked in numerical order, with each `STABLE` branch ranking higher than the release branches forked off of it. For instance, for FreeBSD 8, the order from highest to lowest would be:

- `RELENG_8` (branch rank 899).

- `RELENG_8_3` (branch rank 803).

- `RELENG_8_2` (branch rank 802).

- `RELENG_8_1` (branch rank 801).

- `RELENG_8_0` (branch rank 800).

The colors that Tinderbox uses for each cell in the table are defined by CSS. Successful builds are displayed with green text; unsuccessful builds are displayed with red text. The color fades as time passes since the corresponding build, with every half an hour bringing the color closer to grey.

## 6.2.2. Official Build Servers

The official Tinderbox build servers are hosted by Sentex Data Communications, who also host the FreeBSD Netperf Cluster.

Three build servers currently exist:

freebsd-current.sentex.ca builds:

- `HEAD` for amd64, arm, i386, i386/pc98, ia64, mips, powerpc, powerpc64, and sparc64, which takes approximately four hours.

- `RELENG_9` and supported 9.`X` branches for amd64, arm, i386, i386/pc98, ia64, mips, powerpc, powerpc64, and sparc64, which takes approximately three and a half hours.

freebsd-stable.sentex.ca builds:

- `RELENG_8` and supported 8.`X` branches for amd64, i386, i386/pc98, ia64, mips, powerpc and sparc64, and each branch takes approximately six hours.

freebsd-legacy.sentex.ca builds:

- `RELENG_7` and supported 7.`X` branches for amd64, i386, i386/pc98, ia64, powerpc, and sparc64, and each branch takes approximately three hours.

## 6.2.3. Official Summary Site

Summaries and logs from the official build servers are available online at http://tinderbox.FreeBSD.org, hosted by Dag-Erling C. Smørgrav `<des@FreeBSD.org>` and set up as follows:

- A cron(8) job checks the build servers at regular intervals and downloads any new log files using rsync(1).

- Apache is set up to use `index.cgi` as `DirectoryIndex`.

- A Varnish instance in front of Apache ensures that `index.cgi` does not need to run more than once every two minutes.

Prev | Home | Next
Regression and Performance Testing | Up | Interprocess Communication