Archive for October, 2006

When dnsbl is all but helpful

I was preparing security updates for all the Mozilla® products in stable and needed some feedback from their security team...

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  security@mozilla.org
    SMTP error from remote mail server after RCPT TO:<security @mozilla.org>:
    host dm-mail02.mozilla.org [63.245.208.176]: 554 5.7.1 Service unavailable; Client host [xx.yy.zz.tt] blocked using dnsbl.sorbs.net=127.0.0.10;
+Dynamic IP Addresses See:
    http://www.sorbs.net/lookup.shtml?xx.yy.zz.tt

Damn. And my ISP relay decided to reject my mail. Do we have a relay for DDs in need somewhere ?

Update: It seems retrying was enough, there might be a problem with one of my ISP relay servers...

2006-10-28 12:36:36+0900

firefox | 2 Comments »

Facts about Debian, Mozilla® Firefox® and Ubuntu

or How Mozilla® Corporation is FUDing on Debian. Again.

This time, Christopher Beard, Mr Mozilla® Marketing is talking about Firefox® in Ubuntu, that it's great, Ubuntu is a good boy, that it cooperates with Mozilla®, and that by cooperating, it can use the great Firefox® name and logo.

To be honest, I don't care that Ubuntu calls Firefox® Firefox® instead of Iceweasel or whatever. I don't care that Ubuntu doesn't care about freedom as much as Debian does. I could not care less.

But, indeed, I do care about (yet other) false claims about Firefox® in Debian.

Let's see what Christopher has to say about it:

I understand that Ubuntu is based upon Debian. Is that the same or different than the IceWeasel browser that Debian is shipping with their latest release?
It's different. The patches that Debian applied to the Mozilla source code (which then resulted in their IceWeasel product) are more significant in scope than those in what Ubuntu is shipping (and branding as an official Mozilla Firefox release). Firefox in Ubuntu represents a somewhat more modest set of divergences from original Mozilla source code.

Let's check out what it is about. And as I don't want people to doubt my words, I'll give you, readers, the ability to check for yourself what it is all about.

First, please download the latest currently available patch for Firefox® 2.0 beta 2 in Debian. It has been in experimental for some weeks now. It is a bit different from the patch I described earlier, so I'll also tell you what has been changed since this 2.0b2 patch:

  • security/coreconf/rules.mk is not patched for the perl dependencies thing any more because the patch (taken from bugzilla #325148) has been applied in Firefox® 2.0rc1
  • The patch from bugzilla #336056 has been applied because the directory in which we would build the 2.0rc1 version contained a ~ character, which would make the build fail because of this bug.
  • The patch from bugzilla #294879 has been applied for obvious reasons. It has been applied later in the Firefox® 2.0 release process.

(Also note that another change has been done since then, which is actually reverting a previous patch on gfx/src/gtk/mozilla-decoder.cpp, that was adding code that became dead code when we changed the way to fix the bug.)

Next, please download the official and supported Firefox® diff from Ubuntu.

Uncompress both these diff files, and create two directories (let's call them ubuntudiff and debdiff) which will contain the individual patches for each file. To fill these directories, please use the following commands:

$ filterdiff --strip=1 firefox_2.0+0dfsg-0ubuntu3.diff > ubuntu.diff
$ filterdiff --strip=1 firefox_1.99+2.0b2+dfsg-1.diff > deb.diff
$ cd ubuntudiff
$ splitdiff -d -a ../ubuntu.diff
$ cd ../debdiff
$ splitdiff -d -a ../deb.diff

If you don't have the splitdiff and filterdiff utilities, you can get them from the patchutils tools (you will also need another of these tools later).

Now, let's have fun with these split patches...

Files that are only patched in Debian: (excluding any file that would be in the debian subdirectory, that contains maintainer scripts for build and installation)

$ LANG=C diff -ru debdiff/ ubuntudiff/ | grep "^Only in deb" | grep -v debian | wc -l
6

So, that's 6 files that Debian patches that Ubuntu doesn't. Let's check what they are:

$ LANG=C diff -ru debdiff/ ubuntudiff/ | grep "^Only in deb" | grep -v debian

(reordering result for convenience)

Only in debdiff/: browser_base_content_aboutDialog.xul

Adds rows=5 to the user agent textbox so as to display the user agent string uncut by default

Only in debdiff/: content_html_content_src_nsGenericHTMLElement.cpp
Only in debdiff/: content_html_content_src_nsHTMLInputElement.cpp
Only in debdiff/: dom_src_base_nsGlobalWindow.cpp

It's a patch I submitted in bugzilla #343953 and that got applied in Firefox® 2.0. No surprise the patch is not applied on Ubuntu.

Only in debdiff/: extensions_reporter_Makefile.in

Patch I submitted in bugzilla #354413 and that got applied in Firefox® 2.0.

Only in debdiff/: security_coreconf_rules.mk

Patch from bugzilla #325148 that got applied in Firefox® 2.0 (see above), and a small patch to build the NSS library with debugging symbols (put -g in CFLAGS).

That leaves only 2 small patches : CFLAGS = -g added to security/coreconf/rules.mk and rows=5 to browser/base/content/aboutDialog.xul

Files that are only patched in Ubuntu: (same exceptions as for Debian)

$ LANG=C diff -ru debdiff/ ubuntudiff/ | grep "^Only in ubuntu" | grep -v debian | wc -l
45

Yes, that's right, that's 45 files that Ubuntu patches that Debian doesn't. Most are windows sizes and similar things that upstream can't get right because they are values adapted to Windows, but that also includes some changes to the code and some other things.

Let's now check differences in files that are patched in both. It's a bit of shell black magic, but you can check by hand that it does nothing wrong :

$ LANG=C diff -ru debdiff/ ubuntudiff/ | filterdiff -p1 -x configure -x debian"*" | lsdiff --strip=1 | while read f; do diff -u debdiff/$f ubuntudiff/ | awk "! /^---|^\+\+\+|^ |^[-+]*\@\@/ { print \"$f\"; exit}"; done

That gives a list of patch files that have more differences than line numbers changes, and that do not apply to the debian directory or the configure script (which is generated from configure.in), which is what we really want to compare here. You can take this list and run interdiff between these files from the ubuntudiff and debdiff directories. I'll explain for you what these differences are:

  • Makefile.in: Ubuntu adds a line in order to install the NSS include files,
  • browser_app_Makefile.in: Patch from bugzilla #314927 applied by Debian
  • browser_app_profile_firefox.js: (interdiff fails because the changes apply to different versions of the file) Ubuntu changes the homepage, and sets profile.allow_automigration to false,
  • config_autoconf.mk.in: Ubuntu changes includedir and idldir,
  • configure.in: Ubuntu changes MOZ_APP_DISPLAYNAME, and adds some echos,
  • gfx_src_gtk_nsFontMetricsXft.cpp: Ubuntu sets FC_ANY_METRICS to some patterns,
  • intl_lwbrk_src_nsJISx4501LineBreaker.cpp: Patch from bugzilla #161826 applied by Debian for sparc64, which is not an architecture supported by Ubuntu,
  • modules_libpref_src_init_all.js: Ubuntu changes some default fonts from serif to sans-serif, while Debian changes fonts for serif, sans-serif and monospace (setting to generic names instead of Times, Helvetica and Courier) ; I think both should be applied in Debian, actually. Ubuntu also sets dom.event.contextmenu.enabled to false,
  • security_nss_lib_freebl_unix_rand.c: (interdiff fails because the changes apply to different versions of the file): both patches have the exact same effect, but are different due to changes to original file between 2.0rc1 and 2.0 final.

Overall, Ubuntu applies the same set of patches as Debian, plus some more. A somewhat more modest set of divergences, huh ?!?

For what it's worth, Ubuntu, like Debian, builds its Firefox® with flat chrome and pango enabled.

What's different in the shipping Ubuntu version of Firefox than the proposed Debian version of Firefox (that didn't ultimately ship)?
Technically, changes include fixes to the User Agent string and the feed preview, a well as addressing issues of coherent branding. More significant than any specific difference in code, however, is Ubuntu's commitment to work together with Mozilla and our community on releases going forward to insure product quality and integrity.
Why are you working with Ubuntu when you wouldn't work with Debian?
We did try to work with Debian and would prefer a situation in which we work together. Ultimately, Debian took a position that was consistent with their own policies, and not compatible with some of the exceptions to Mozilla trademark policies that we offered. While we understand and respect their decision not to work with us under our branding guidelines, Mozilla believes that brands like Firefox are important for consumer protection. In any event, Ubuntu developers are working closely with Mozilla developers to insure product quality and features that are what users expect when they use Mozilla Firefox, which means that they'll ship (and will continue to ship) a fully branded version.

Reading between the lines, that means Debian is not working with Mozilla. It's not like we're submitting patches. No. Never. Ever. I'm also glad to hear from Asa Dotzler, in comments to Mr Beard's article, that this great collaboration with Ubuntu will lead to patches applied to Firefox® (I guess the paid Canonical employees having more time to deal with Mozilla® than the volunteer Debian maintainers may have helped, especially considering they didn't have to do all what we already prepared). Anyways, it's not like some of the patches we sent got applied.

So, while I'm at it, here is an exhaustive list of the bugs where we took or sent the patches that are applied to Iceweasel: #51429, #161826, #252033, #258429, #273524, #287150, #289394, #294879, #307168, #307418, #314927, #319012, #322806, #323114, #325148, #326245, #330628, #331781, #331785, #331818, #333289, #333308, #343953, #345077, #345079, #345080, #345413.

These don't cover the following patches (see the rationale for these in my previous article):

  • security/manager/Makefile.in, security/nss/cmd/shlibsign/Makefile: Don't build the shlib signatures, see the rationale in my previous article,
  • security/nss/cmd/shlibsign/manifest.mn: Don't build the mangle utility,
  • browser/app/Makefile.in: Force linking against libxpcom.so despite -Wl,--as-needed,
  • browser/app/profile/firefox.js, gfx/src/gtk/fontEncoding.properties, modules/libpref/src/init/all.js: Preferences changes,
  • browser/base/content/aboutDialog.xul: That's one of the patches that Debian applies that Ubuntu doesn't. See above,
  • config/autoconf.mk.in: Remainings of an old patch for pangocairo, which is now useless, and setting of mozappdir,
  • config/rules.mk, configure.in: Patch from Thiemo Seufer to increase stability and performance on mips, useless upstream until bugzilla #258429 is fixed, but on my to_send list,
  • extensions/inspector/Makefile.in: Something that should be added to #331785, but well, it's WONTFIX, anyway,
  • layout/build/Makefile.in, layout/build/nsLayoutModule.cpp: Change to the Gecko string,
  • modules/libpref/src/nsPrefService.cpp: Add a preferences directory for /etc/firefox/pref,
  • widget/src/gtk2/nsWindow.cpp: Extended mouse buttons support, on my to_send list,
  • configure.in, xpcom/reflect/xptcall/src/md/unix/Makefile.in: Basic patch for bugzilla #343975,
  • configure.in: Check for pangoxft.

It's so great to spend a great amount of time on a package, send patches, try to understand how things work to get patches applied, and yet, see such denial and false claims about our work. So please, Christopher, Asa, and the others, just stop talking about Debian, it will be better for everyone.

PS for Rob in comments out there: No, ColorZilla won't work on Ubuntu, because of the ABI incompatibility I explained in my previous entry, that Mozilla® doesn't seem to care much about.

2006-10-26 22:48:01+0900

firefox | 26 Comments »

Iceweasel hype ?

Interestingly, I see in my apache logs quite some hits from Iceweasel, even though we haven't uploaded an Iceweasel package yet. A few are coming from the gnuzilla Iceweasels, but some others obviously come from people changing their User-Agent string on purpose:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061003 IceWeasel/2.0
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060830 Iceweasel/1.5.0.7 (Debian-1.5.dfsg+1.5.0.7-2)

(...)

2006-10-17 19:42:51+0900

firefox | 3 Comments »

Facts about Debian and Mozilla® Firefox®

There have been quite some comments on the Iceweasel case all over the planets, and I saw several assertions, especially from the Mozilla® camp, that I, as the Firefox® co-maintainer, the xulrunner maintainer, and (soon) Seamonkey™iceape co-maintainer, have to rectify.

They broke the --enable-official-branding flag
Half-true. We just replaced Bon Echo/Deer Park with Firefox® at the appropriate places in the build tree so that we could have Firefox® with the "unbranded" logo instead of the official logo, as Gervase Markham gave us authorization for. You're still free to enable the official branding, except that since the logos and stuff are non-free, we removed the other-licenses/branding/ directory from the original tarball, thus yes, the flag is half broken.

Firefox® logos being subject to trademarks, Debian thinks they are not free.
Trademark and copyright are different things. Mozilla® has unnecessarily given a non-free license to "clarify" the trademark situation, but that is not required. To make it clear: Debian thinks the logos are not free because they are not free. Period.

Debian isn’t properly collaborating with Mozilla®, sending unusable 100000-lines patches for validation just before releases (as seen on Lucas Nussbaum's blog)
Let me take the firefox_2.0~rc1+dfsg-1.diff.gz file, strip the debian directory from it (it only contains maintainer scripts, our set of icons and some debian specific searchplugins), and strip the configure diff that is generated by autoconf due to some changes in configure.in... that's exactly 2654 lines of diff. Very far from the 100000-lines patches they are claiming.

The Mozilla people talked about Debian-specific changes that changed frozen APIs, breaking extensions (from Lucas's blog again).
So, let's dig into our firefox_2.0~rc1+dfsg-1.diff.gz:

  • Changes to disable application upgrade (we want that to happen through apt-get) and change some other default preferences,
  • Changes to fix "make distclean" so that it really cleans the build directory,
  • Change not to build the "mangle" utility,
  • Change not to call netstat to generate entropy, which is useless on linux,
  • Changes to make Firefox® build and work on architectures such as hppa, mips, mips64, m68k, ia64, sparc64, alpha, and arm, which the Mozilla® guys don't seem to care much for,
  • Change to add a preference directory so that users can put their set of customized preferences in /etc/firefox/pref,
  • Change to allow to build flat chrome without the zip utility,
  • Change to allow to use system library for myspell, instead of statically linking the bundled one,
  • Changes to allow to build s390 binaries on s390x host with s390 toolchain (same applies with x86 binaries on amd64 host with x86 toolchain),
  • Changes to work around bugs with the hidden visibility pragma on gcc,
  • Changes to make the pango backend actually build correctly,
  • Changes to avoid some error messages while trying to create Makefiles from inexistant Makefile.in's,
  • Change to install in /usr/lib/firefox instead of /usr/lib/firefox-x.y,
  • Change not to build useless chromelist.txt files,
  • Changes to make helper applications with parameters work,
  • Changes to allow builds against GTK 2.8,
  • Changes to work around an Xrender bug,
  • Changes to make the Gecko/yymmdddd string taken from preferences instead of being half-hard-coded (you could change it with preferences, but it would still be set to the hard-coded value at start time ; and you could change it again with preferences...),
  • Change to allow mice extra buttons to act as something else than a left button,
  • Change to allow to build with -Wl,--as-needed to avoid linking against a whole lot of useless libraries, without losing the link on libxpcom.so which is required by some extensions' components,
  • Changes not to shlibsign the NSS modules at build time, since we're stripping the binaries afterwards, thus breaking the signature. We do build the signatures later, within the maintainer scripts.

That's not that many changes, and most of them were taken from either some Mozilla® CVS trunk or the Mozilla® Bugzilla™. And most of those that were not taken from there have been sent, except those that really don't make much sense outside Debian. So, where are these frozen API changes ?

And we're not properly collaborating, huh ?

The Mozilla® project started by coming to a (admittedly uneasy) agreement with Debian for use of the name, but the Debian version diverged even further from the official version, so the permission was revoked. (from comments on Matthew Garrett's blog)

That one is really interesting, because between the time we got this understanding with Gervase and now, we are actually less diverging from the official version than by then. The main difference by that time was the extensions manager, which, in Debian, needed a lot of changes to actually act as it should, especially with globally installed extensions. I'm not saying the Debian one was perfect, it also had its own problems, but that was a whole lot less than the blatant crap that was the official one, obviously written for Windows without any thoughts for unix, and especially linux distributions.

The only main difference now, between the official Firefox® and ours, is that our build has the pango backend enabled, which we chose over the Xft backend for several reasons I won't explain here. The others differences are that we use system libraries where possible, instead of the bundled libpng, libjpeg, libtiff, libmyspell and libcairo. We also build a flat chrome, instead of having everything in .jar files.

Now, a little bit about differences the Mozilla® guys don't seem to care about while they really should: distributions build the Mozilla® products with gcc 4.x, while the official binaries are built with gcc 3.4, as well as the extensions distributed on addons.mozilla.org. Fortunately, not a lot of extensions make use of binary components, and not a lot are linked against the standard C++ library, but when that happens (like with colorzilla), you get a component linked against libstdc++5 to load on a Firefox® that is linked against libstdc++6. You are lucky if that setup doesn't crash.

Little extra from comments in Lucas's blog:
Is it possible for the Debian Firefox maintainers to create an installer package for contrib which will install the vanilla FireFox from Mozilla’s site.
How great would it be to have a package for one architecture instead of 12, and with a dependency on libstdc++5, that almost no other package uses any more.

Update:
Debian is going to replace Firefox® with a GNU fork called Iceweasel
Half-true. For the etch release, Iceweasel will only be Firefox® with a different branding. We are taking the Iceweasel name because it was already know as a possible alternative name for Firefox® when the trademark concerns have been raised more than 2 years ago (thanks Nathanael Nerode for this nice name, by the way). It appears that the GNU guys decided to start a fork with this name... that's quite unfortunate, actually. Anyways, the plan is to get in touch with them to see what we can do together, but with the etch release approaching, we can't and won't do more than a rename for the moment.

Update 2:
We (Mozilla®) presently have working relationships with most of the major Linux distributions, including Red Hat, Novell, and Ubuntu (As seen in several posts from people of the Mozilla® Corporation or Foundation)
Very interesting. Ubuntu uses the same set of patches as Debian, with some more of their own, and even releases beta software in their official releases. But when it's Ubuntu, it's fine. Sorry, I forgot Debian is lame, and DDs are frustrated fanatic integrists, on top of being bloody fanatic assholes.

Update 3: Added some precisions about other differences with the official binaries, and a small patch I somehow forgot.

2006-10-15 22:49:34+0900

firefox, xulrunner | 67 Comments »

Plugin API for browsers

Reading about the future of the (now) Adobe Flash Player plugin for Linux on the Penguin.SWF blog, I came to the conclusion that the plugin API for browsers (NP Runtime) just sucks.
Why on earth do a plugin need to depend on openssl, while the browser already deals with SSL and associated certificates. That also means plugins will deal very badly with client-side certificates.
I guess plugins also have to implement HTTP requests themselves...

2006-10-01 15:48:02+0900

firefox | 3 Comments »