Archive for the 'xulrunner' Category

svk vs. git for xulrunner maintenance

I've been using svk for xulrunner maintenance since the first test packages. And I've been thinking about switching to git for a while now. I did some testing in march, which led to my post about diffing. I went a bit further today by actually trying to convert my svk repository. And I got both interesting and surprising results.

The repository is (only) about 250 revisions in svk, and includes 15 upstream releases (or CVS pulls). I had to reorganize the svk repository so that git-svnimport could do its work properly.

[ Note: these tests have been done on a 5 years old laptop ]

The first surprise is the time it took for the conversion : almost half an hour. Dumping (svnadmin dump) and reloading (svnadmin load) the svk repository takes a bit more than 3 minutes, in comparison... I guess git-svnimport is very ineffective.

The second surprise is that the svk repository is actually almost twice as small as the git equivalent: 116MB for svk vs. 224MB for git.

On the other hand, git is amazingly fast. One of the most common things I do on this repository is a diff between the upstream branch and the trunk. I create the .diff.gz file I give pbuilder this way. It's much quicker than doing a real dpkg-source, especially considering pbuilder does one itself.

After dropping caches (echo 3 > /proc/vm/sys/drop_caches), such a diff takes about 50 seconds with svk and... a little more 2 seconds with git. Without dropping caches, it gets down to 38 seconds with svk and... 0.05s with git. That is pretty damn fast !

Update: after a git repack -d, the git repository is down to 61MB.

2007-07-02 22:12:18+0900

xulrunner | 7 Comments »

Browser detection

Erich, the Gecko version number is pretty meaningless.

With mozilla releases, the date reflects when the build was done. Which means if you build Firefox 1.5 today, it will get a Gecko/20070410 string.

With Debian releases (except icedove), the date reflects the date for the client.mk in the source tarball, which is one of the last file upstream touches before a release. This helps having the same date for all 11 architectures (even after a binNMU) and is somehow more significant, but still not that much.

There are, at the moment, 3 main branches for Gecko-based code: MOZILLA_1_8_0_BRANCH, MOZILLA_1_8_BRANCH and HEAD. The latter currently contains Gecko 1.9 alpha, Firefox 3.0 alpha, etc., and will eventually be branched to a MOZILLA_1_9_BRANCH or something similar when going in beta. The other branches are respectively for Gecko 1.8.0.x (Firefox 1.5.0.x, Thunderbird 1.5.0.x, Seamonkey 1.0.y, Xulrunner 1.8.0.x) and Gecko 1.8.1.x (Firefox 2.0.0.x, Thunderbird 2.0.0.x (not yet released), Xulrunner 1.8.1.x).

[ In Debian Etch, we have Iceweasel 2.0.0.3 (Gecko 1.8.1.3), Iceape 1.0.8 (originally Gecko 1.8.0.10 but patched at version 1.8.0.11), Icedove 1.5.0.10 (Gecko 1.8.0.10 ; changes from version 1.8.0.11 didn't make it, but they don't affect the mailer code), and Xulrunner 1.8.0.11 (Gecko 1.8.0.11). The latter is used by kazehakase, galeon and epiphany. ]

Whenever a new security release for Firefox 1.5.0.x and other products from the Gecko 1.8.0.x branch are done, the Gecko date obviously changes, and doesn't reflect the fact that it's an older Gecko than that of Firefox 2.0.0.x...

Now if you take a closer look to the user agent string, you'll see something that is actually more significant than the Gecko date, i.e. the Gecko revision, such as "rv:1.8.0.11" in "Mozilla/5.0 (X11; U; Linux i686; ja_JP; rv:1.8.0.11) Gecko/20070324"

2007-04-10 08:12:46+0900

firefox, iceape, xulrunner | 4 Comments »

Facts about Debian and Mozilla® Firefox®

There have been quite some comments on the Iceweasel case all over the planets, and I saw several assertions, especially from the Mozilla® camp, that I, as the Firefox® co-maintainer, the xulrunner maintainer, and (soon) Seamonkey™iceape co-maintainer, have to rectify.

They broke the --enable-official-branding flag
Half-true. We just replaced Bon Echo/Deer Park with Firefox® at the appropriate places in the build tree so that we could have Firefox® with the "unbranded" logo instead of the official logo, as Gervase Markham gave us authorization for. You're still free to enable the official branding, except that since the logos and stuff are non-free, we removed the other-licenses/branding/ directory from the original tarball, thus yes, the flag is half broken.

Firefox® logos being subject to trademarks, Debian thinks they are not free.
Trademark and copyright are different things. Mozilla® has unnecessarily given a non-free license to "clarify" the trademark situation, but that is not required. To make it clear: Debian thinks the logos are not free because they are not free. Period.

Debian isn’t properly collaborating with Mozilla®, sending unusable 100000-lines patches for validation just before releases (as seen on Lucas Nussbaum's blog)
Let me take the firefox_2.0~rc1+dfsg-1.diff.gz file, strip the debian directory from it (it only contains maintainer scripts, our set of icons and some debian specific searchplugins), and strip the configure diff that is generated by autoconf due to some changes in configure.in... that's exactly 2654 lines of diff. Very far from the 100000-lines patches they are claiming.

The Mozilla people talked about Debian-specific changes that changed frozen APIs, breaking extensions (from Lucas's blog again).
So, let's dig into our firefox_2.0~rc1+dfsg-1.diff.gz:

  • Changes to disable application upgrade (we want that to happen through apt-get) and change some other default preferences,
  • Changes to fix "make distclean" so that it really cleans the build directory,
  • Change not to build the "mangle" utility,
  • Change not to call netstat to generate entropy, which is useless on linux,
  • Changes to make Firefox® build and work on architectures such as hppa, mips, mips64, m68k, ia64, sparc64, alpha, and arm, which the Mozilla® guys don't seem to care much for,
  • Change to add a preference directory so that users can put their set of customized preferences in /etc/firefox/pref,
  • Change to allow to build flat chrome without the zip utility,
  • Change to allow to use system library for myspell, instead of statically linking the bundled one,
  • Changes to allow to build s390 binaries on s390x host with s390 toolchain (same applies with x86 binaries on amd64 host with x86 toolchain),
  • Changes to work around bugs with the hidden visibility pragma on gcc,
  • Changes to make the pango backend actually build correctly,
  • Changes to avoid some error messages while trying to create Makefiles from inexistant Makefile.in's,
  • Change to install in /usr/lib/firefox instead of /usr/lib/firefox-x.y,
  • Change not to build useless chromelist.txt files,
  • Changes to make helper applications with parameters work,
  • Changes to allow builds against GTK 2.8,
  • Changes to work around an Xrender bug,
  • Changes to make the Gecko/yymmdddd string taken from preferences instead of being half-hard-coded (you could change it with preferences, but it would still be set to the hard-coded value at start time ; and you could change it again with preferences...),
  • Change to allow mice extra buttons to act as something else than a left button,
  • Change to allow to build with -Wl,--as-needed to avoid linking against a whole lot of useless libraries, without losing the link on libxpcom.so which is required by some extensions' components,
  • Changes not to shlibsign the NSS modules at build time, since we're stripping the binaries afterwards, thus breaking the signature. We do build the signatures later, within the maintainer scripts.

That's not that many changes, and most of them were taken from either some Mozilla® CVS trunk or the Mozilla® Bugzilla™. And most of those that were not taken from there have been sent, except those that really don't make much sense outside Debian. So, where are these frozen API changes ?

And we're not properly collaborating, huh ?

The Mozilla® project started by coming to a (admittedly uneasy) agreement with Debian for use of the name, but the Debian version diverged even further from the official version, so the permission was revoked. (from comments on Matthew Garrett's blog)

That one is really interesting, because between the time we got this understanding with Gervase and now, we are actually less diverging from the official version than by then. The main difference by that time was the extensions manager, which, in Debian, needed a lot of changes to actually act as it should, especially with globally installed extensions. I'm not saying the Debian one was perfect, it also had its own problems, but that was a whole lot less than the blatant crap that was the official one, obviously written for Windows without any thoughts for unix, and especially linux distributions.

The only main difference now, between the official Firefox® and ours, is that our build has the pango backend enabled, which we chose over the Xft backend for several reasons I won't explain here. The others differences are that we use system libraries where possible, instead of the bundled libpng, libjpeg, libtiff, libmyspell and libcairo. We also build a flat chrome, instead of having everything in .jar files.

Now, a little bit about differences the Mozilla® guys don't seem to care about while they really should: distributions build the Mozilla® products with gcc 4.x, while the official binaries are built with gcc 3.4, as well as the extensions distributed on addons.mozilla.org. Fortunately, not a lot of extensions make use of binary components, and not a lot are linked against the standard C++ library, but when that happens (like with colorzilla), you get a component linked against libstdc++5 to load on a Firefox® that is linked against libstdc++6. You are lucky if that setup doesn't crash.

Little extra from comments in Lucas's blog:
Is it possible for the Debian Firefox maintainers to create an installer package for contrib which will install the vanilla FireFox from Mozilla’s site.
How great would it be to have a package for one architecture instead of 12, and with a dependency on libstdc++5, that almost no other package uses any more.

Update:
Debian is going to replace Firefox® with a GNU fork called Iceweasel
Half-true. For the etch release, Iceweasel will only be Firefox® with a different branding. We are taking the Iceweasel name because it was already know as a possible alternative name for Firefox® when the trademark concerns have been raised more than 2 years ago (thanks Nathanael Nerode for this nice name, by the way). It appears that the GNU guys decided to start a fork with this name... that's quite unfortunate, actually. Anyways, the plan is to get in touch with them to see what we can do together, but with the etch release approaching, we can't and won't do more than a rename for the moment.

Update 2:
We (Mozilla®) presently have working relationships with most of the major Linux distributions, including Red Hat, Novell, and Ubuntu (As seen in several posts from people of the Mozilla® Corporation or Foundation)
Very interesting. Ubuntu uses the same set of patches as Debian, with some more of their own, and even releases beta software in their official releases. But when it's Ubuntu, it's fine. Sorry, I forgot Debian is lame, and DDs are frustrated fanatic integrists, on top of being bloody fanatic assholes.

Update 3: Added some precisions about other differences with the official binaries, and a small patch I somehow forgot.

2006-10-15 22:49:34+0900

firefox, xulrunner | 67 Comments »

More work on xulrunner

Xulrunner finally reached unstable on sunday, and it already needed some adjustments. I forgot a conflict, misnamed the libsmjs-dev package, the hppa assembler code got somehow duplicated, so xulrunner failed to build on hppa, and some preference files needed to be moved around the packages.

It also fails to build on alpha, but that's not my fault.

I also changed the way we identify a Debian built xulrunner in the user agent. I replaced the "Gecko/yyyymmdd" string with "Gecko/Debian/x.y.z.t-r". That has several advantages over the previous "Gecko/yyyymmdd Debian/x.y.z.t-r" string:

  • It shortens the already long user agent string so that additions such as product name are not painful,
  • removes pointless information (the date in the original string indicates the date of the build, not that of the API),
  • keeps the "Gecko" string (which some site might want, seing how Apple and Konqueror did put a "like Gecko" string),
  • and finally avoid confusion with other Debian release informations that may be present in the product specific part (I think Galeon puts one, for instance).

That needed 2 updates, because I realized in between that while you can set the general.useragent.product and general.usergagent.productSub variables, they are not actually taken into account until you change them a first time... The HTTP protocol initialization in xulrunner forces the respective values Gecko and yyyymmdd... I just completely removed that part of the code in xulrunner.

I took advantage of this needed fix to work even more on the package, enabling the typeaheadfind module, needed by galeon and epiphany, removing some old perl code that does nothing except producing useless errors during the build, and, last but not the least, enabling the "flat style" chrome, meaning that instead of being stuck in .jar files, everything is just here in a standard tree.

It appears upstream provides a way to build such trees (option --enable-chrome-format=flat to configure), but it's useless because it doesn't install the trees when running make install. In addition to that, the configure script still requires zip, even if it doesn't use it, except for some obscure .jar files in libnss, that are not installed either. So after disabling the build of these useless files and removing the zip strict requirement if we build flat chrome, and fixing the script responsible for installation of the chrome files, everything got much better.

These changes have been applied to firefox as well, so stay tuned, it's for next upload.

Having highjacked spidermonkey, I did some bug triage there as well. It appeared they could all be closed by the fact libmozjs0d is there. I still need to make some triage in my patches to xulrunner and send upstream those that I still have not sent, if they are of any interest there.

Apart from this xulrunner stuff, I also did some bug clean-up on libxml2 and libxslt, which I've been somehow neglecting for some time.

2006-02-22 21:06:21+0900

p.d.o, xulrunner | 6 Comments »

xulrunner 1.8.0.1-2

You know, nothing can be perfect from the first time, so I had to fix a few issues in a new upload. Well, for you, it makes no difference, since 1.8.0.1-1 never reached the archive, but these changes were needed so that epiphany and friends could properly build against xulrunner.

I also started filing wishlist bugs with patches to build some packages against xulrunner as soon as it gets into the archive. Galeon is undergoing. Next one might be kazehakase. Stay tuned.

Update: Patches for galeon, kazehakase, and devhelp sent.

2006-02-08 23:49:38+0900

xulrunner | 4 Comments »

xulrunner 1.8.0.1-1

Checking Signature on .changes
(...)
Good signature on /home/mh/pbuilder/sid/result/xulrunner_1.8.0.1-1_i386.changes.
Checking Signature on .dsc
(...)
Good signature on /home/mh/pbuilder/sid/result/xulrunner_1.8.0.1-1.dsc.
Uploading via ftp xulrunner_1.8.0.1-1.dsc: done.
Uploading via ftp xulrunner_1.8.0.1.orig.tar.gz: done.
Uploading via ftp xulrunner_1.8.0.1-1.diff.gz: done.
Uploading via ftp libnspr4-dev_1.8.0.1-1_all.deb: done.
Uploading via ftp libmozjs-dev_1.8.0.1-1_all.deb: done.
Uploading via ftp libsmjs1_1.8.0.1-1_all.deb: done.
Uploading via ftp libsmjs1-dev_1.8.0.1-1_all.deb: done.
Uploading via ftp libxul-dev_1.8.0.1-1_all.deb: done.
Uploading via ftp libnss3-dev_1.8.0.1-1_all.deb: done.
Uploading via ftp xulrunner_1.8.0.1-1_i386.deb: done.
Uploading via ftp xulrunner-gnome-support_1.8.0.1-1_i386.deb: done.
Uploading via ftp libnspr4-0d_1.8.0.1-1_i386.deb: done.
Uploading via ftp libmozjs0d_1.8.0.1-1_i386.deb: done.
Uploading via ftp spidermonkey-bin_1.8.0.1-1_i386.deb: done.
Uploading via ftp libxul0d_1.8.0.1-1_i386.deb: done.
Uploading via ftp libnss3-0d_1.8.0.1-1_i386.deb: done.
Uploading via ftp xulrunner_1.8.0.1-1_i386.changes: done.
Successfully uploaded packages.

I wonder how long it will take to the ftp-masters to accept it. It seems the NEW queue is not as fast as it used to be. It's been a week I've been waiting for libxml2 and libxslt to get through, and they still haven't made it. If enough people is interested in the package before it comes out of NEW, I'll probably upload it in my repository.

I have a patch for epiphany-browser that I'll send soon to the BTS. I'll come up with a patch for yelp as well. Then, bye bye mozilla-browser.

I'm also going to build an unofficial firefox on top of xulrunner, and maybe upload it to experimental, so that we can have some feedback about that. I'll see with Eric about that.

2006-02-07 21:21:03+0900

xulrunner | 2 Comments »

Dealing with mozilla.org

I've been dealing with some open source projects in the past, submitting patches and stuff, but dealing with mozilla.org is really a hassle. You never really know what to do at what moment. They even admit that it's not easy... And I'm not even talking about how bugzilla is a pain in the ass.

But this is the first time I've been asked to actually find a submitter myself.

I already sent patches that actually did land in the CVS without me requesting anything. So what's the difference ? That it's a feature request instead of a bug ?

What about my other patches ?

2005-11-05 09:31:20+0900

firefox, xulrunner | 1 Comment »

Debian Xulrunner, take 1

I finally got seriously into the xulrunner packaging today (see ITP) and managed to get something that is able to run the MyBrowser sample XUL application (provided that you change the MaxGeckoVersion in the application.ini file in the mybrowser directory) and to gracefully let mozilla-browser run, while we override quite a few of its libraries.

As the long term plan for xulrunner is to be the XUL runtime for some (all?) mozilla.org applications, I moved all its libs to /usr/lib, overriding mozilla-browser's libnss3 and libnspr4. Upstream has this special feature that it doesn't support libraries versioning, so that it is usually a mess, but they are supposedly trying to freeze the API, which will hopefully fix a lot of the current issues with mozilla releases.

Anyways, xulrunner provides its own libnspr4 and libnss3 packages (actually, libnspr4.6 and libnss3.10), while diverting the original mozilla-browser files into /usr/lib/mozilla, and provides a libmozjs package providing spidermonkey, and a libsmjs1 compatibility package. The standalone spidermonkey package will have to be removed, it's outdated and redundant.

While libnspr4 and libnss3 from mozilla-browser could be removed and mozilla-browser could use xulrunner's ones, the contrary is not possible for libnspr4 because of some missing symbols. libnss3 is fine, though, maybe I'll allow xulrunner to use it. On the other hand, xulrunner's libmozjs makes mozilla segfaults...

When everything will be ready, I'll make an upload to experimental and will ask galeon, epiphany and other maintainers to check out if they can build their browser with the libgtkembedmoz provided by xulrunner, which is IMHO, the thing we'll have to do some day.

2005-08-16 19:33:29+0900

xulrunner | 3 Comments »