The Mozilla build system and Tamarin
As the owner of the Mozilla build system, I get to hear endless moaning about how byzantine the build system is and how it would be nice to use something more modern. I sympathize with the problem: our build system is difficult to set up, has a very steep learning curve, and uses lots of unusual languages. On the other hand, our build system works amazingly well: we have grown a set of rules that build an amazing variety of file types, and works on a wide variety of platforms. Out build system is complex for the very good reason that our needs are complex.
A few of the more aware complainers have suggested alternatives such as SCons. I have avoided making huge changes to our build system because it is overwhelming to contemplate migrating our existing build logic to a new build system. Any new build system would have to compare favorably to our existing system, which has years of bugfixes and features to its credit.
I was excited when I was asked to help implement a cross-platform build system for the Tamarin project. Tamarin was released with platform-specific Visual Studio and XCode project files. I thought that perhaps this was a perfect opportunity to try one of the newer build systems, without paying the cost of porting our entire existing build system over.
I created a requirements list. Unfortunately, I don’t think that any of the existing tools are going to meet our needs without extensive customization. I’m going to continue to examine SCons and WAF more over the next few days, but it looks like the path of easiest success is still going to be an old-school autoconf + make build system. Hopefully at least I’ll be able to fix some issues with recursive make while implementing the new system.
November 8th, 2006 at 11:46 pm
What about Cmake by kitware – http://www.cmake.org. Its used by KDE as their build system
November 9th, 2006 at 12:11 am
“byzantine”… Yeah, the mozilla build system is very complex but that’s not the problem IMHO. The problem is only the learning curve ; when one wants to add/hack/chatnge something, it’s a game of “let’s try that to see if it does something” and “I have not a single clue about the first step needed to do that thing”. Given the complexity of the system, I even wonder if doc is the solution.
November 9th, 2006 at 1:17 am
I agree; I’m not so sure the problem is with the system as with the paucity of docs and the difficulty of finding those that do exist (especially if you’re not aware of d.m.o/REQUIRES and the like, which are new enough that not even the people who should know about them actually do). I would prefer not to have to relearn a language (Makefile-speak + some sort of shell amalgam) to do stuff with the build system, but unless something shows up which has a syntax which has less of a learning curve (SCons and Python appeal to me for this reason, but I’m sure you’ve done literally infinitely more research into it than I have, so it’s sad that’s not a viable option yet), has better basic documentation, and perhaps can be implemented piecemeal so as not to make the transition process too monolithic, I say stick with what we have.
November 9th, 2006 at 3:56 am
I’m not an expert on build systems, but from what I’ve read cmake seems to fulfill all your requirements as well. Any particular reason it’s not on the list of possible solutions?
November 9th, 2006 at 4:33 am
i’m sure the current build system works fine, as there are binaries available on the ftp, but for me, building a win32 binary was a pain and i’m not yet satisfied with the result – i got a working 1.8.0.4 xulrunner, but i don’t really know how/why it built at all.
while on linux the build pretty much worked out of the box, i have yet to find a win32 tutorial/doc that doesn’t leave you with make exit codes, linker errors or even suddenly missing autoconf requirements. things like distclean can corrupt the whole sandbox and the only way to continue is to checkout again.
my only successful win32 build with a working binary at the end of the story was a static build after some header and source hacking. trial and error. but trial and error takes a LONG time with the moztree (compiling for 1-2 hours per take).
November 9th, 2006 at 12:31 pm
Benjamin,
Great news to hear that you’ve got a guinea pig to investigate something better than autoconf/make for the Moz build system. I’ve been doing the ActivePython and Komodo build systems (including the Mozilla builds that Komodo uses) for a number of years using mostly Python and Cons (the now unmaintained Perl-based precursor to SCons). Let me know if you have questions or if I can help in a minor way.
Regarding your concerns with SCons:
– “a poor C/C++ dependency tool”: Could the one from WAF (if it has a better one, haven’t looked) be slapped into SCons?
– “poor support for configuration”: What we do for the Komodo build is have our own custom configure code (written in Python) that generates a config.py (*). We then explicitly import that in the Python build scripts to have configuration variables available. For the Mozilla example you could have the SConscript files explicitly import that config.py.
I’m not a aware of a nice convenient Python-based configuration system (basically an autoconf replacement). Most projects focus on replacing just “make”. I could show you what we have for that if that would be useful to you.
(*) Komodo’s configure script actually generates bkconfig.pm (Perl), bkconfig.py (Python) and bkconfig.{sh|bat} (environment changes) used to setup the build environment appropriately and for importing by the various Perl and Python builds scripts in the system.
February 23rd, 2007 at 6:35 pm
FYI, “The Road to KDE 4: CMake, a New Build System for KDE”
http://dot.kde.org/1172083974/