Build Management

Description

Software systems consist of sources (things directly manipulated by humans), and derivates (things that are derived automatically from the sources). Derivates are constructed from the sources by applying tools. Examples are the construction of executables from C source files using a C compile, or the generation of C source files from Yacc parser specifications using Yacc.

Complex systems can consist of tens of thousands of source files, hundreds of tools, and enormous amounts of derived information. It is infeasible to create the derivates by manually running the tools on the sources; this should be done automatically using a build manager. Also, it is impractical to rebuild all derivates every time a programmer makes a (small) change to the sources. The build manager should ensure that only those derivates that are actually influenced by the change are rebuilt.

In other words, the job of a build manager is to ensure that the software system is consistent; i.e., that all derivates are derived from current sources, or are equal to what they would be if they had been derived from current sources. In practice, this turns out to be a surprisingly hard problem.

Tools

• The "industry standard" build tool is Feldman's make program, which was introduced in the Seventh Edition of the Unix operating system. There are countless clones (not all source compatible), the most commonly used being GNU Make which is installed on virtually every GNU/Linux system. Make's model is very simple: systems are described as a set (a makefile) of rules that specify a command with which a list of derivates (targets) can be created from a list of sources (dependencies). Make rebuilds derivates if any of the sources has a newer file system timestamp. Unfortunately, make often causes inconsistent builds, since (a) makefiles tend to specify incomplete dependency information, and (b) the up-to-date detection is unreliable; e.g., changes to compiler flags will not trigger recompilation. Make's input language is also quite simplistic, making it hard to specify variants.

• There are some Make clones that attempt to fix Make's fundamental defects. For example, Makepp tracks derivates, including the commands used to build them, and can generate dependency lists automatically.

• Odin is quite a bit more advanced than Make. Odin's input language is essentially functional: derivates are things that can be derived by applying tools - functions - to sources or other derivates. For example, the expression hello.c denotes a source, while hello.c :exe denotes the executable obtained by compiling hello.c. Variant builds can be specified easily by extending an expression with parameters: hello.c +debug :exe yields an executable with debug information turned on.

• Amake was the build tool for the Amoeba distributed operating system. An important aspect is the separation of build tools and system specifications, leading to higher abstraction. Amake can be found in the Amoeba 5.3 distribution.

• Other build tools: GNU cons, cook, Makepp

Related tools

• Since makefiles have a tendency to become big and hard to maintain, there are tools that generate them from a higher-level description of the system. GNU Automake is a well-known example. It often suffices to specify the names of the sources and derivates; Automake automatically generates rules based on its knowledge of certain programming languages.

• Systems often must be build on a wide variety of platforms. GNU Autoconf generates configuration scripts - to be run on the target platform - that automatically detect properties of the platform. Such properties typically include presence of certain header files, libraries or packages, endianness and word sizes, etc. The scripts are typically used to generate makefiles or C header files to adapt the system to the platform.

Papers

• Feldman, Make - A Program for Maintaining Computer Programs.
• Miller, Recursive make considered harmful.
• Baalbergen, Verstoep, Tanenbaum, On the design of the Amoeba Configuration Manager.
• Amake User Reference Manual, section 6.3 of the Amoeba Reference Manual.
• Clemm, The Odin System - Reference Manual.

-- EelcoDolstra - 21 Oct 2002