Archive for August, 2011

I have a pathological use of browser tabs: I use a lot of them. A lot is probably an understatement. We could say I use them as bookmarks of things I need to track. A couple weeks ago, I was saying I had around two hundred tabs opened. I now actually have much more.

It affected startup until I discovered that setting the browser.sessionstore.max_concurrent_tabs pref to 0 was making things much better by only loading tabs when they are selected. This preference has/will become browser.sessionstore.restore_on_demand. However, since I only start my main browser once a day, while other applications start and while I begin to read email, I hadn't noticed that this was still heavily affecting startup time: about:startup tells me reaching the sessionRestored state takes seven seconds, even on a warm startup.

It also affects memory usage, because even when tabs are only loaded on demand, there is a quite big overhead for each tab.

And more importantly, it gets worse with time. And I think the user interface is actively making it worse.

So, to get an idea how bad things were in my session, I wrote a little restartless extension. After installing it, you can go to the about:tabs url to see the damage on your session. Please note that the number of groups is currently wrong until you open the tab grouping interface.

This is what the extension has to say about my session 2 days ago, right after a startup:

• 556 tabs across 4 groups in 1 window
• 1 tab has been loaded
• 444 unique addresses
• 105 unique hosts
• 9 empty tabs
• 210 http:
• 319 https:
• 14 ftp:
• 2 about:
• 2 file:
• 55 addresses in more than 1 tab
• 39 hosts in more than 1 tab

The first thing to note is that when I filed the memory bug 4 days earlier, I had a bit more than 470 tabs in that session. You can see 4 days later, I now have 555 tabs (if excluding the about:tabs tab).

The second thing to note is something I suspected because it's so easy to get there: a lot of the tabs are opened on the same address. Since Firefox 4.0, if I'm not mistaken, there is a great feature in the awesomebar, that allows to jump to an existing tab matching what you type in the urlbar. That is very useful, and I use it a lot. However, there are a lot of cases where it's not as useful as it could be.

One of the addresses I visit a lot is http://buildd.debian.org/iceweasel. It gives me the build status of the latest iceweasel package I uploaded to Debian unstable. That url is particularly known in my browsing history, and is the first hit when I type "buildd" in the urlbar (actually, even typing "b" brings it first). Unfortunately, that url redirects to https://buildd.debian.org/status/package.php?p=iceweasel through an HTTP redirection. I say unfortunately because when I type "buildd" in the urlbar, I get 6 suggestions for urls in the form http://buildd.debian.org/package (I also watch other packages build status), and the suggestion to switch to the existing tab for what the first hit would get me to is 7th. Guess what? The suggestion list only shows 6 items ; you have to scroll to see the 7th.

The result is that I effectively have fifteen tabs open on that url.

I also keep a lot of bugzilla.mozilla.org bugs open in different tabs. The extension tells me there are 255 of them... for 166 unique bugs. Largely, the duplicate bug tabs are due to having these bugs open in some tab, but accessing the same bugs from somewhere else, usually a dependent bug or TBPL. I also have 5 tabs opened on my request queue. I usually get there by going to the bugzilla home page and clicking on the "My Requests" link. And I have several tabs opened on the same bug lists. For the same reason.

When I started using tab groups, I splitted in very distinct groups. Basically, one for Mozilla, one for Debian, one for stuff I want to follow (usually blog posts I want to follow comments from), and one for the rest. While I was keeping up with grouping at the beginning, I don't anymore, and the result is that each group is now a real mess.

Firefox has hundreds of millions users. It's impossible to create a user experience that works for everyone. One thing is sure, it doesn't work for me. My usage is probably very wrong at different levels, but I don't feel my browser is encouraging me to use it better, except by making my number of opened tabs explode to an unmanageable level (I already have 30 tabs more than when I started writing this post 2 days ago).

There are a few other things I would like to know about my usage that my extension hasn't told me yet, either because it doesn't tell, or because I haven't looked:

• How many tabs end up loaded at the end of a typical day?
• How many tabs do I close?
• How many duplicate tabs do I open and close?
• How long has it been since I looked at a given tab?
• How do the number of tabs and duplicates evolve with time?

Reflecting on my usage patterns, I think a few improvements, either in the stock browser, or through extensions, could make my browsing easier:

• Auto-grouping tabs: When I click on a link to an url under mozilla.org, I most likely want it in the Mozilla group. An url under debian.org would most likely go in the Debian group.
• Switch to an existing tab when following a link to an already opened url: That might not be very useful as a general rule, but at least for some domains, it would seem useful for me that the browser switches to an existing tab not only through the urlbar, but also when following links in a page. If I'm reading a bug, click on a bug it depends on, and that bug is already opened in another tab, get me there. There would be a history problem to solve, though. (e.g. where do back and forward bring?)

Maybe these exist as extensions, I don't know. It's hard to find very specific things like that through an add-on search (though I haven't searched very hard). [Looks like there is an experiment for the auto tab grouping part]

I think it would also be interesting to have something like Test Pilot, but for users that want to know the answer to "How do I use my browser?". As I understand it, Test Pilot can show individual user data, but it only can do so if there is such data, and you can't get data for past studies you didn't take.

In my case, I'm not entirely sure that, apart from the pinned tabs, I use the tab bar a lot. And even for pinned tabs, most of the time I use keyboard shortcuts. I'm not using the menu button that much either. I already removed the url and search bar (most of the time) with LessChrome HD. Maybe I could go further and use the full window for web browsing.

I got Profile Guided Optimization (a.k.a. Feedback Directed Optimization) to work for Android builds, using GCC 4.6.1 and Gold 2.21.53.

Getting such a build is not difficult, just a bit time consuming.

• Apply the patches from bug 632954
• Get an instrumented build with the following command:
  

  $ make -f client.mk MOZ_PROFILE_GENERATE=1 MOZ_PROFILE_BASE=/sdcard/mozilla-pgo

• Create a Fennec Android package:
  

  $ make -C $objdir package

  If you get an elfhack error during this phase, make sure to update your tree, the corresponding elfhack bug has been fixed.

• Install the package on your device:
  

  $ adb install -r $objdir/dist/fennec-8.0a1.en-US.android-arm.apk

• Open Fennec on your device, and do some things in your browser, so that execution data is collected. For my last build, I installed the Zippity Test Harness add-on, and ran V8, Sunspider and PageLoad tests
• Collect the execution data:
  

  $ adb pull /sdcard/mozilla-pgo /

• Clean-up the build tree:
  

  $ make -f client.mk clean

• Build using the execution data:
  

  $ make -f client.mk MOZ_PROFILE_USE=1

• Create the final Fennec Android package, install and profit:
  

  $ make -C $objdir package
$ adb install -r $objdir/dist/fennec-8.0a1.en-US.android-arm.apk

As the title indicates, though, this actually leads to some disappointment. On my Nexus S, the resulting build is actually slightly slower on Sunspider than the corresponding nightly. It is however much faster on V8 (down to around 1200 from around 1800), but... is just as fast as a non PGO/FDO build with GCC 4.6. Even sadder, the non PGO/FDO build with GCC 4.6 is faster on Sunspider than the PGO/FDO build, and on-par with the GCC 4.4-built nightly.

So, my experiments suggest that switching to GCC 4.6 would give us some nice speed-ups, but enabling PGO/FDO wouldn't add to that.

If you want to test and compare my builds on different devices, please go ahead, with the following builds:

• Yesterday's nightly build, built with GCC 4.4 (13,522,584 bytes)
• Build of the same commit, with GCC 4.6 (12,984,560 bytes)
• Build of the same commit, with GCC 4.6 and PGO/FDO enabled (13,992,139 bytes)

The former will install as "Nightly", while the two others will install as "Fennec".

The sizes are also interesting: while the PGO build is bigger than the Nightly build, the plain GCC 4.6 build is smaller.

As of writing, the latest Native-code Development Kit for Android (r6) comes with gcc 4.4.3 and binutils 2.19 for ARM. The combination is a quite old toolchain, that lacks various novelties, such as properly working Profile Directed Optimization (a.k.a. Profile Guided Optimization), or Identical Code Folding.

The first thing that is needed to rebuild a custom NDK, is the NDK itself.

$ wget http://dl.google.com/android/ndk/android-ndk-r6-linux-x86.tar.bz2
$ tar -xjf android-ndk-r6-linux-x86.tar.bz2
$ cd android-ndk-r6

Next, you need to get the NDK source (this can take a little while and requires git, but see further below if you want to skip this part):

$ ./build/tools/download-toolchain-sources.sh src

Rebuilding the NDK toolchain binaries is quite simple:

$ ./build/tools/build-gcc.sh $(pwd)/src $(pwd) arm-linux-androideabi-4.4.3

But this doesn't get you anything modern. It only rebuilds what you already have.

The GCC 4.4.3 that comes with the NDK is actually quite heavily patched. Fortunately, only a few patches are required for gcc 4.6.1 to work with the NDK (corresponding upstream bug).

In order to build a newer GCC and binutils, you first need to download the source for GCC (I took 4.6.1) and binutils (I took the 2.21.53 snapshot, see further below), as well as GMP, MPFR and MPC. The latter was not a requirement to build GCC 4.4. GMP and MPFR are with the NDK toolchain sources, but the versions available there are too old for GCC 4.6.

All the sources must be placed under src/name, where name is gcc, binutils, mpc, mpfr, or gmp. The sources for MPC, MPFR and GMP need to remain as tarballs, but the sources for GCC and binutils need to be extracted (don't forget to apply the patch linked above to GCC). In the end you should have the following files/directories:

• src/gcc/gcc-4.6.1/
• src/binutils/binutils-2.21.53/
• src/gmp/gmp-5.0.2.tar.bz2
• src/mpc/mpc-0.9.tar.gz
• src/mpfr/mpfr-3.0.1.tar.bz2

If you skipped the NDK toolchain source download above, you will also need the gdb sources. NDK comes with gdb 6.6, so you should probably stay with that one. The source needs to be extracted like GCC and binutils, so you'll have a src/gdb/gdb-6.6/ directory. Another part you will need is the NDK build scripts, available on git://android.git.kernel.org/toolchain/build.git. They should be put in a src/build/ directory. For convenience, you may directly download a tarball.

You then need to edit the build/tools/build-gcc.sh script to add support for MPC:

Add the following lines somewhere around similar lines in the script:

MPC_VERSION=0.8.1
register_var_option "--mpc-version=<version>" MPC_VERSION "Specify mpc version"

And add the following to the configure command in the script:

--with-mpc-version=$MPC_VERSION

If you want to use gold by default instead of GNU ld, you can also add, at the same place:

--enable-gold=default

If you want a GNU libstdc++ compiled as Position Independent Code (note that by default, the NDK won't use GNU libstdc++, but its own), you can add, at the same place:

--with-pic

Once all this preparation is done, you can build your new NDK toolchain with the following command:

$ ./build/tools/build-gcc.sh --gmp-version=5.0.2 --mpfr-version=3.0.1 --mpc-version=0.9 --binutils-version=2.21.53 $(pwd)/src $(pwd) arm-linux-androideabi-4.6.1

If you're running a 64-bits system on x86-64, you can also add the --try-64 option to the above command, which will give you a 64-bits toolchain to cross-build ARM binaries, instead of the 32-bits toolchain you get by default.

When building Firefox with this new toolchain, you need to use the following in your .mozconfig:

ac_add_options --with-android-toolchain=/path/to/android-ndk-r6/toolchains/arm-linux-androideabi-4.6.1/prebuilt/linux-x86

Or the following for the 64-bits toolchain:

ac_add_options --with-android-toolchain=/path/to/android-ndk-r6/toolchains/arm-linux-androideabi-4.6.1/prebuilt/linux-x86_64

Note that currently, elfhack doesn't support the resulting binaries very well, so you will need to also add the following to your .mozconfig:

ac_add_options --disable-elf-hack

Or, if you don't want to build it yourself, you can get the corresponding pre-built NDK (32-bits) (thanks to Brad Lassey for the temporary hosting). Please note it requires libstdc++ from gcc 4.5 or higher.

Here is a list of things you may need to know if you want to try various combinations of versions, and that I had to learn the hard way:

• GCC 4.6.1 doesn't build with binutils 2.19 (GNU assembler lacks support for a few opcodes it uses)
• GNU ld >= 2.21.1 has a crazy bug that leads to a crash of Firefox during startup. There is also a workaround.
• Gold fails to build with gcc 4.1.1 (I was trying to build in the environment we use on the buildbots) because of warnings (it uses -Werror) in some versions, and because of an Internal Compiler Error with other versions.
• When building with a toolchain that is not in the standard directory and that is newer than the system toolchain (like, in my case, using gcc 4.5 in /tools/gcc-4.5 instead of the system gcc 4.1.1), gold may end up with a libstdc++ dependency that is not satisfied with the system libstdc++. In that case, the NDK toolchain build will fail with the error message "Link tests are not allowed after GCC_NO_EXECUTABLES.", which isn't exactly helpful to understand what is wrong.
• At some point, I was getting the same error as above when the build was occurring in parallel, and adding -j1 to the build-gcc.sh command line solved it. It hasn't happened to me in my recent attempts, though.
• Gold 2.21.1 crashes when using Identical Code Folding. This is fixed on current binutils HEAD (which is why I took 2.21.53).