OpenSUSE Build System

For the last couple of years the folks at SUSE have been building a new build system that could help us make packages for multiple distributions from the same source code.

Yesterday, Andreas Jaeger announced the open sourcing of the Open SUSE Build Service (the post is archived here).

The service has been in beta testing for a few months.

The Problem

One of the problems that the Linux community faces is that a binary package built in one distribution will not necessarily work on a different distribution. Distributions have adopted different file system layout configurations (from the trivial to the more complex), different versions of system libraries and configuration files are placed in different locations. In addition different revisions of the operating system will differ from release to release, so the same software will not always build cleanly.

Although there are efforts to unify and address this problem, these efforts have failed because they require all major distributions to agree on file system layout, in the adoption of a core. Without the buy-in of everyone, it is not possible to solve this problem, which has made the life of the Linux Standard Base folks incredibly frustrating.

If you are ahead of the pack, the mentality has been that you will prevail over other distributions and become the only Linux distribution so there is no need to spend extra engineering work in trying to unify distributions.

At Ximian, when we were a a small independent software vendor trying to distribute our software for Linux users (Evolution, and as a side effect the "Ximian Desktop") we had to set up internal build farms to target multiple operating systems. We created a build system (build-buddy) with "jails" which were a poor man's operating system virtualization system to build packages for all of the Linux distributions.

To this date shipping software for multiple Linux distributions remains a challenge. For instance, look at the downloads page for Mono. We are targeting various distribution of Linux there, and we know we are missing many more, but setting up multiple Linux distributions, building on every one of them, running the tests and maintaining the systems takes a lot of time.

And this problem is faced by every ISV. Folks at Mozilla and VMware have horror stories to tell about how they have to ship their products to ensure that they run on a large number of distributions.

The OpenSUSE Build System

Originally, the OpenSUSE Build System was a hosted service running at SUSE. The goal of the build system was to build packages for various distributions from the same build specifications.

Currently it can build packages for the Novell, Red Hat, Mandriva, Debian and Ubuntu (the versions supported currently are: SUSE Linux 9.3, 10.0 and 10.1, OpenSUSE 10.2, SUSE Linux Enterprise 10, Debian Etch, Fedora Core 5, 6, Mandriva 2006 and Ubuntu 6.06).

For example, Aaron is packaging Banshee for a number of platforms, you can browse the packages here.

This hosted service means that developers did not have to setup their own build farms, they could just use the build farms provided by SUSE.

With the open sourcing of the build system, third party ISVs and IT developers no longer have to use the hosted services at SUSE, they are now able to setup their own build farms inside their companies and produce packages for all of the above distributions. If you need to keep your code in-house, you can use the same functionality and setup your own build farms.

Currently those creating packages can use either a web interface or a command line tool to get packages built by the system (a nice API is available to write other front-ends; GUI front-ends are currently under development).

Testimonials

From the Scribus blog:

While we recently released 1.3.3.7 into the wild, one of the tools we used for the first time is the new OpenSuse Build Server. So why the rant ?

In a word it is terrific. :) It still is in beta phase and there are features to be added. However, it is a major helper for time stretched developers to package their application in a sane way and then automagically push them to rotating mirrors worldwide without a lot of fuss.

It is all fine and well if you write the best code in the world if no one can use it. And that means packaging. For Linux, that means creating a .deb or creating rpm packages which many Linux distributions use.

The magic is one can upload the source and a spec file which is a recipe to build rpms. Select which platforms to build, push a button and voila. In an hour or two you have all your packages built in a carefully controlled manner. This allowed us to supply 1.3.3.7 for all Suse releases 9.3+ including the Enterprise Desktop 10 for the first time. Even better for users, they can use what ever tool be it Yast, Smart or Yum to add these repositories to get the latest greatest Scribus with all dependencies automatically satisfied. Push button packaging for users too :)

Installation

The resulting packages that are submitted today to the build service can be installed with a number of popular software management tools including Smart, apt, yum and yast.

We hope that this cross-distribution build system will remove the burden of picking a subset of Linux distributions to support and will make more packages available for more distributions.

A tutorial on getting started is here.

Congratulations to the OpenSUSE team for this release!