EBuild Design: Modules and Products

A lot has been made of dependency management, the dependency graph and its role in software builds — and with good justification. However fitting builds into this system has not been without its difficulties and compromises. One major problem is that current build systems, such as Maven, fit all dependencies into one uniform system. It is the contention of this article — and by extension, the design of EBuild — that there are two fundamental types of dependencies in builds, which are sufficiently different that they warrant different treatment.

In EBuild terminology the two types of dependencies are Modules and Products. The generic term for them is Elements. Current build systems such as Maven<a href="#footnotes" />1</a> are optimised to support Modules, but in fact Modules are just a special case. A really important special case, but a special case nevertheless.

Modules

Modules are just shareable libraries of code. One key property that Modules have is that they have transitive dependencies. What this means is that usage of a Module implies usage of all its dependencies, and all their dependencies and ... so on<a href="#footnotes" />2</a>. This is known as the transitive closure of the dependency graph.

Before a compilation step in a build, the transitive closure of dependencies must be fetched (and built if any of them are source dependencies). From this, what is logically created is a superimposition of their contents. In Java this is called a classpath, and as every Java developer knows a dozen jars in a classpath are equivalent to one large jar made by combining them all together<a href="#footnotes" />4</a>.

Transitive dependencies in Modules are a consequence of them being shared. They can be shared at build time, but also at runtime. In both instances it implies the need for dependency management (well, to be done properly). The dependency management at runtime is done by what is often called a package manager. Here we are more concerned with the build time situation, but the same principles apply.

Taking Java as an example. We could be creating a standalone desktop application. In which case we could take all the compilation output from our Modules — some jars (excluding the J2SE runtime jars, which we require the user to have installed) — and combine them together as an executable jar. The result of this process is a Product.

Products

Products are arbitrary collections of files. No generic assumptions can be made about their use (so superimposition as if they were Modules is a nonsense) and once assembled they do not have dependencies. A Product could be an executable, but equally it could be documentation, or an archive containing the file structure of an application ... etc.

Products have to be recognized in the dependency graph so that they can be re-used sanely within a build. For example we may wish to include documentation in a programs download Product, but also in the programs website Product.

Building and Assembling

Building Products is also a different process to building Modules. The EBuild terminology reflects this difference: Products are assembled by Assemblers, and Modules are built by Builders. Again, due to the nature of Modules, underlying assumptions can be made. Building Modules is usually a generic process of translating source code into compiled code, however in more advanced cases there may be several stages e.g. preprocessing, generating code using a compiler generator or web services stub generator ... etc.

Assembling Products is a more specific process, e.g. selecting artifacts from Modules to combine into an application, generating documentation, laying out files, signing files, archiving files ... etc.

<a name="footnotes" />

  1. EBuild Modules are in essence the same as the Maven Modules.
  2. In some situations it is desirable to not use some dependencies at build time for API reasons, an important feature, but an unnecessary digression here.
  3. At this point users of dynamic languages may think it no longer applies to them. Well it does. Generating autodocs from code is isomorphic with the process of compilation.
  4. Ignoring signing, sealing and other features. Once Jars have gained such specific properties they cease to be Modules, and become Products.

ADVERTISEMENTS