Faster Linux builds
After two failed attempts last year, and a few glitches yesterday, we finally managed to get our Linux (and, obviously, Linux64) builds to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled. This means we are finally using a more modern toolchain, opening opportunities for things such as static analysis. This also means we are now producing a faster Firefox, now much closer to the Windows builds on the same hardware on various performance tests.
A nice side effect of some of the work I have done to make the switch possible is that these builds will also work on older Linux platforms such as RedHat/CentOS 5, or possibly older (as long as they come with libstdc++ from GCC 4.1).
The first Firefox release to benefit these new settings should be Firefox 6.
A few branches other than mozilla-central have also been switched, most notably Try, for which there is a known issue if you push something too old. Please make sure to read the corresponding information on wiki.m.o for a workaround. A Mercurial hook is going to be put in place to issue a warning if there are chances your build will fail (it will, however, not prevent the push).
Thanks to Chris Atlee, Rail Aliiev, Taras Glek, Justin Lebar and all those I forgot or am not aware of for their assistance and/or past involvement in the previous attempts.
2011-04-29 11:31:18+0200
You can leave a response, or trackback from your own site.
2011-04-29 12:52:18+0200
Why GCC 4.5 and not 4.6?
http://gcc.gnu.org/gcc-4.6/
2011-04-29 13:42:56+0200
Awesome! Do you have any idea if these optimizations will be easy to implement in packages provided by various distributions?
2011-04-29 16:57:49+0200
@CCG: I guess: Too many regressions in 4.6. When it can build firefox with LTO and PGO with -g (for example: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48207) then they’ll upgrade. Have you tried rebuilding your entire distro with gcc 4.6?
2011-04-29 18:42:34+0200
> with aggressive optimization (-O3)
This is a bad, bad idea. -O3 should only be applied to code that will benefit from it the most (see GCC documentation) – otherwise it just induces bloat that makes the program SLOWER.
Big apps like Firefox benefit a LOT more from -Os, since they can more easily fit into the processor’s cache then.
2011-04-29 20:02:56+0200
Firefox 4.0.1 just came out today, why must we wait until Firefox 6.x?? Lets make 4.5.2 the next release, the 0.0.2 being the minor upgrade, and the 0.5.0 being this major compile improvement.
2011-04-29 20:04:08+0200
This is true without profile-guided optimization, but one of the things PGO does is back down from -O3 to -Os for cold code. We wouldn’t be making this change without performance numbers to back it up; we see across-the-board improvements of at least 5%, 20% on some benchmarks (I don’t have links, but maybe Mike does).
2011-04-30 06:14:34+0200
> Firefox 4.0.1 just came out today, why must we wait until Firefox 6.x??
This can’t be shipped as part of a stable release without a lot of testing. There are lots of potential issues that it can cause, that aren’t immediately apparent.
But, you can try Firefox 6 right now! If you run Nightly, then that is the code that will eventually ship as Firefox 6. Or, wait a few weeks and it will be available as Aurora, which is a little more stable. (And 6 weeks later, as Beta.)
Nightly and Aurora builds are *exactly* meant for this situation – when you want to benefit from a cool new feature like GCC 4.5/-O3/PGO – without waiting for all the testing to be done, and are ok with something not as stable as a fully tested release.
2011-04-30 18:02:21+0200
[...] Source Free subscription: Subscribe RSS feed or get daily tips in your email Jump to Comments ↓ * Click confirmation link sent in email * Don't see the email? check spam folder [...]
2011-04-30 19:35:26+0200
[...] and today a group of Mozilla developers has repaid their devotion with some good news. Mozilla’s Mike Hommey reported this morning that his team of coders finally managed to get both 32 and 64-bit Firefox builds for Linux to [...]
2011-05-01 02:15:44+0200
I’m guessing this is x86/amd64 only?
2011-05-01 02:49:24+0200
> I’m guessing this is x86/amd64 only?
Android is still building with -Os, I think primarily to keep the binary size down, but perhaps also because we use an older compiler there. I’m not sure about Maemo or other ARM platforms, but it’s all in configure.in if you can find it. :)
http://hg.mozilla.org/mozilla-central/file/068d876996c6/configure.in#l2693
2011-05-01 05:36:20+0200
Wow that’s lovely. The GCC 4.6 release notes mention building firefox with LTO, but I guess that still needs polishing?
2011-05-01 05:52:58+0200
[...] – Mike Hommey, [...]
2011-05-01 05:53:48+0200
Well I looked at perf-o-matic, and the improvements are enormous. For example a year’s worth of speed up in page load, Wow just wow.
for 32 bit
Ts -8.4%
Tp4 -11.6% This one is so good it needs a link:
http://graphs.allizom.org/graph.html#tests=38,1,14&sel=none&displayrange=7&datatype=running
Txul -10.0%
DHTML -1.0%
for 64 bit
Ts -5.0%
Tp4 -6%
Txul -5.5%
DHTML -0.9%
2011-05-01 06:26:22+0200
[...] - Mike Hommey, Digitizor Categories: Uncategorized LikeBe the first to like this post. [...]
2011-05-01 08:00:06+0200
[...] Hommey在博客上宣布，他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-01 09:07:06+0200
[...] проекта Mozilla и мэйнтейнер пакета Iceweasel в Debian GNU/Linux, объявил о готовности к переходу при формировании 32- и [...]
2011-05-01 12:01:26+0200
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48207 is only about compiling with link time optimization. I am building mozilla with GCC 4.6 and current GCC mainline quite regularly and it seems to just work. Did not do any serious testing.
Also -O2 and -O3 is pretty much same with PDO. Inlining and other stuff is implied by -fprofile-use.
It could be nice to analyze where GCC still lacks compared to Windows build se we can do something about it in 4.7 ;) (and of course watch Mozilla performance with mainline GCC so we don’t have any stupid regressions)
2011-05-01 12:04:27+0200
Also note that the link time optimization issues are tracked here http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45375.
To make story short, GCC 4.6. builds working mozilla with LTO and binary is a lot smaller. There are however problems with debug info and fact that it needs 8GB of memory.
Mainline GCC is getting better on memory usage and hopefully will hammer out the debug info issues. Still however it won’t build with -g and LTO, yet.
2011-05-01 12:12:24+0200
Here is a Dromaeo ff4->ff6 before-after: http://dromaeo.com/?id=139110,139113
One test became slower (bitwise and), and a few show big improvements. Though I don’t know which tests call into C code, are intepreted javascript, or are jit-ed javascript.
2011-05-01 12:16:25+0200
(ff6 is the latest firefox 6a1 nightly, and the results are very noisy, though Dromaeo uses statistics to compensate. Also this was not a clean profile.)
2011-05-01 14:13:38+0200
[...] Hommey, membro di Mozilla, ha annunciato tramite il suo blog che dopo ben due tentativi falliti, si è riuscito ad ottenere un incremento delle prestazioni [...]
2011-05-01 18:06:15+0200
[...] Mike Hommey laat op zijn weblog weten dat zij bij de ontwikkeling van de laatste Linux-testbuilds er in zijn geslaagd om de GCC [...]
2011-05-01 18:11:15+0200
[...] проекта Mozilla и мэйнтейнер пакета Iceweasel в Debian GNU/Linux, объявил о готовности к переходу при формировании 32- и [...]
2011-05-01 20:27:02+0200
[...] Hommey在博客上宣布，他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-01 20:52:25+0200
[...] Mike Hommey laat op zijn weblog weten dat zij bij de ontwikkeling van de laatste Linux-testbuilds er in zijn geslaagd om de GCC [...]
2011-05-01 21:42:41+0200
Are the 6.0a1 Nightlies being built now with the new toolchain? I got the impression that was the case.
2011-05-01 21:58:05+0200
Do you know if the various distro will automatically pick up those changes as well when FF6 comes out (Not trying to troll or anything, I’m genuinely curious, I don’t know how much their build configuration differs from mozilla’s) ? Are there any plans in place to help them transition their FF packages to this new toolchain ?
2011-05-01 23:57:33+0200
[...] parole stesse di Hommey: “Dopo due tentativi falliti dello scorso anno e dopo qualche rognetta di ieri, [...]
2011-05-02 05:54:27+0200
I’ve noticed that the JPEG decoding code, e.g. jidctint.c
has SSE2 optimisations for Windows and MacOS. It seems to me there is a performance problem when scrolling pages of JPEGs.
Is there any chance of getting SSE2 optimisations for Linux?
2011-05-02 10:24:06+0200
[...] due tentativi falliti (lo scorso anno), gli sviluppatori sono riusciti a compilare una versione Linux di Firefox con GCC 4.5, attivando il livello di ottimizzazione più aggressivo (-O3) e la profile guided optimization [...]
2011-05-02 11:19:23+0200
[...] 6将和Windows版一样快。 Mozilla的Mike Hommey在博客上宣布，他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-02 14:06:41+0200
[...] έκδοση του προγράμματος στα Windows. Σύμφωνα με δημοσίευμα που προέρχεται από προγραμματιστή της Mozilla, η ομάδα [...]
2011-05-02 14:07:24+0200
[...] puntualizza lo stesso sviluppatore, la prima release a beneficiare del cambiamento (si parla di incrementi prestazionali tra il 5 e il [...]
2011-05-03 00:21:27+0200
[...] to use GCC 4.5, with assertive optimization (-O3) and form guided optimization enabled,” wrote developer Mike Hommey on [...]
2011-05-03 01:33:40+0200
[...] Hommey在博客上宣布，他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-03 01:45:32+0200
[...] optimization (-O3) and profile guided optimization enabled,” Mozilla developer Mike Hommey blogged. “This means we are finally using a more modern toolchain, opening opportunities for things [...]
2011-05-03 02:54:33+0200
[...] Hommey在博客上宣布， 他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-03 03:38:19+0200
[...] Hommey在博客上宣布， 他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-03 05:06:11+0200
[...] to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled,” wrote developer Mike Hommey on [...]
2011-05-03 07:55:06+0200
[...] palabra de Mike Hommey, desarrollador de Mozilla: Por fin nos las hemos arreglado para conseguir que nuestras versiones [...]
2011-05-03 15:38:59+0200
[...] builds to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled," wrote developer Mike Hommey on [...]
2011-05-03 15:38:59+0200
[...] builds to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled," wrote developer Mike Hommey on [...]
2011-05-03 15:40:28+0200
[...] see here [...]
2011-05-03 16:13:10+0200
[...] to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled,” wrote developer Mike Hommey on [...]
2011-05-03 18:40:40+0200
> I’m guessing this is x86/amd64 only?
Well, in that we don’t have desktop Firefox builds for ARM, yes. We could theoretically build our Android/Maemo builds with PGO, but it’s not as simple as the desktop case, since a) the profiling run has to be on a device, which makes it more complicated, and b) the toolchains are more of a pain to deal with, since they’re usually customized.
2011-05-04 00:12:22+0200
[...] to use GCC 4.5, with aggressive optimization (-O3) and profile guided optimization enabled,” wrote developer Mike Hommey on [...]
2011-05-04 03:13:37+0200
[...] The team was successful in compiling the web browser with GCC after two failed attempts last year, Mozilla developer Mike Hommey informed via blog post. [...]
2011-05-04 12:26:32+0200
[...] Hommey在博客上宣布，他的团队成功利用整合配置文件导引优化（Profile-Guided [...]
2011-05-04 22:10:20+0200
[...] אגרסיבית (-O3) וכן אופטימיזציה מונחית פרופיל פעילות", כך כתב המפתח מייק הומי ביום שישי האחרון, "זאת אומרת שאנחנו [...]
2011-05-05 01:22:46+0200
[...] Mike Hommey, member au développement du navigateur Firefox au sein de l’équipe de Mozilla, annonce en effet avoir finalement réussi à compiler une build de l’application avec le compilateur [...]
2011-05-05 11:43:27+0200
[...] de referencia: Glandium. – Publicidad [...]
2011-05-05 16:51:18+0200
[...] Mozilla Firefox bientôt plus rapide sous linux ? Ce serait pas un mal… Mais le plus intéressant, c’est le moteur Javascript et dont la nouvelle génération s’appellera IonMonkey. Qui ne sera pas disponible avant Mozilla Firefox 6, à mon très humble avis d’utilisateur des logiciels de la Fondation Mozilla depuis… 11 ans environ [...]
2011-05-10 05:02:25+0200
[...] palabra de Mike Hommey, desarrollador de Mozilla: Por fin nos las hemos arreglado para conseguir que nuestras versiones [...]
2011-05-13 23:05:03+0200
[...] Hommey, participant au développement du navigateur Firefox au sein de l’équipe de Mozilla, annonce en effet avoir finalement réussi à compiler une build de l’application avec le compilateur [...]
2011-05-14 07:49:41+0200
[...] distro also features FireFox 4.01, which is claimed to run much faster on Linux than it did previously. The Mozilla team announced in late April it had finally managed to [...]
2011-08-14 14:00:33+0200
[...] lässt sich auf das Wechseln des Compilers auf GCC 4.5 zurückführen. Der Firefox-Entwickler Mike Hommey schreibt dazu: After two failed attempts last year, and a few glitches yesterday, we finally managed to get [...]
2011-08-17 11:49:26+0200
[...] und mit aktivierter “profile guided optimization”, wie die Entwickler bereits im April in einem Blog-Eintrag verrieten. Nebeneffekt: Firefox 6 sollte nun auch auf älteren Linux-Plattformen, wie [...]
2011-09-03 16:25:09+0200
[...] Compiler umzusteigen, doch erst mit Firefox 6 ist dies nun gelungen. Mozilla-Entwickler Mike Hommey schreibt in seinem Blog: We finally managed to get our Linux (and, obviously, Linux64) builds to use GCC 4.5, with [...]