SpecialistOff.NET / Вопросы / Статьи / Фрагменты кода / Резюме / Метки / Помощь / Файлы

Buildmaster Architecture

Метки: buildbot

The buildmaster consists of several pieces:

Change Sources

Which create a Change object each time something is modified in the VC repository. Most ChangeSources listen for messages from a hook script of some sort. Some sources actively poll the repository on a regular basis. All Changes are fed to the schedulers.

Schedulers

Which decide when builds should be performed. They collect Changes into BuildRequests, which are then queued for delivery to Builders until a worker is available.

Builders

Which control exactly how each build is performed (with a series of BuildSteps, configured in a BuildFactory). Each Build is run on a single worker.

Status plugins

Which deliver information about the build results through protocols like HTTP, mail, and IRC.

Each Builder is configured with a list of Workers that it will use for its builds. These workers are expected to behave identically: the only reason to use multiple Workers for a single Builder is to provide a measure of load-balancing.

Within a single Worker, each Builder creates its own WorkerForBuilder instance. These WorkerForBuilders operate independently from each other. Each gets its own base directory to work in. It is quite common to have many Builders sharing the same worker. For example, there might be two workers: one for i386, and a second for PowerPC. There may then be a pair of Builders that do a full compile/test run, one for each architecture, and a lone Builder that creates snapshot source tarballs if the full builders complete successfully. The full builders would each run on a single worker, whereas the tarball creation step might run on either worker (since the platform doesn’t matter when creating source tarballs). In this case, the mapping would look like:

Builder(full-i386)  ->  Workers(worker-i386)
Builder(full-ppc)   ->  Workers(worker-ppc)
Builder(source-tarball) -> Workers(worker-i386, worker-ppc)

and each Worker would have two WorkerForBuilders inside it, one for a full builder, and a second for the source-tarball builder.

Once a WorkerForBuilder is available, the Builder pulls one or more BuildRequests off its incoming queue. (It may pull more than one if it determines that it can merge the requests together; for example, there may be multiple requests to build the current HEAD revision). These requests are merged into a single Build instance, which includes the SourceStamp that describes what exact version of the source code should be used for the build. The Build is then randomly assigned to a free WorkerForBuilder and the build begins.

The behaviour when BuildRequests are merged can be customized, Collapsing Build Requests.