June 8th, 2010

Playing with icegrind

In the past few days, I've been playing with icegrind, a little valgrind plugin that Taras Glek wrote to investigate how Firefox startup spends its time in libxul.so reading/initialization, mostly. I invite you to check out his various posts on the subject, it's an interesting read.

After some fiddling with the original icegrind patch in the bug linked above, I got some better understanding on the binary initialization phase (more on that in another post), as we could call it, and hit some problems due to problems in icegrind and elflog. The latest version of the icegrind patch as of writing solved my main problem half way (though it needs further modifications to work properly, see below).

What icegrind actually does is to track memory accesses in mmap()ed memory. When the program being profiled mmap()s a file, icegrind looks for a corresponding "sections" file next to it (e.g. "libxul.so.sections" for "libxul.so"). The "sections" file contains offsets, sizes, and sections names, that will be used by icegrind to report the accesses. Whenever the profiled program makes memory accesses (whether it is for instructions or data) within the mmap()ed range, icegrind will add the corresponding section name in a "log" file (e.g. "libxul.so.log" for "libxul.so"). Please note it will only do so for the first access in the given section.

To generate section information for ELF files, Taras also hacked an elflog program that scans an ELF binary (program or library) and will output sections for symbols present in its symtab. This means the binary needs not to be stripped, though the full fledged debugging symbols are not needed. It outputs the sections names in a form that would be reusable in a linker script after a build with the "-ffunction-sections" and "-fdata-sections" compiler options, but that's mostly irrelevant when what you are after is only to see what is going on, not to reorder the sections at link time. There are glitches in the way elflog works that makes the section names for .bss symbols wrong (they will start with .symtab, .shstrtab, .comment, etc. instead of .bss). Also, technically, the .bss section is not mapped from the file.

Note icegrind has rough edges, is still experimental, and isn't very user friendly, as in unix permissions, because its input and output files need to be next to the file being mmap()ed, and when that is a library in /usr/lib, well, you need write access there ; or you need to copy the library in a different directory and adjust LD_LIBRARY_PATH. Anyways, it's already useful as it currently is.

If you want to play with it yourself, here are steps that worked well for me:

  • svn co svn://svn.valgrind.org/valgrind/trunk valgrind -r 11100
    When I first tried icegrind, valgrind trunk was broken ; revision 11100 is known to work properly, at least for icegrind.
  • cd valgrind
  • wget -O - -q https://bug549749.bugzilla.mozilla.org/attachment.cgi?id=449748 | patch -p0
    You need to open icegrind/ice_main.c, go on the last line of the track_mmap function and replace mmaps with dest_mmaps.
  • mkdir icegrind/tests ; touch icegrind/tests/Makefile.am
    The valgrind build system is a bit lunatic.
  • libtoolize ; aclocal ; automake --add-missing ; autoreconf
  • ./configure; make
  • wget http://hg.mozilla.org/users/tglek_mozilla.com/startup/raw-file/6453ad2a7906/elflog.cpp
  • g++ -o elflog elflog.cpp -lelf
    You will need the libelf-dev package.
  • make install ; install -m 755 elflog /usr/local/bin
    as root, to install in /usr/local.

Once you're done with this setup, you are ready to start playing:

  • elflog --contents libxul.so > libxul.so.sections
    Be aware that if libxul.so is a symbolic link pointing to a file in another directory, as it happens in the dist/bin directory in mozilla builds, the "sections" file needs to be in that other directory.
  • LD_LIBRARY_PATH=. valgrind --tool=icegrind ./firefox-bin -P my-profile -no-remote about:blank

Obviously, you don't have to play around with Firefox. You can try with other libraries, or even programs. You'll see in a subsequent post, however, that to get more interesting information, the elflog output is unfortunately not enough.

2010-06-08 18:46:42+0900

mozilla, p.m.o | 1 Comment »

May 22nd, 2010

Recommended for You

There is a part on the Youtube home page which recommends some videos based on what you already watched.

I'm still trying to understand what kind of link there can be.

2010-05-22 11:33:24+0900

miscellaneous, p.m.o | No Comments »

May 11th, 2010

Google日本語入力がオープンソースになりました

前からGoogle日本語入力ソフトが気になっていましたが、残念ながらLinux版がなくて試せませんでした。今日はChromium OSのためにオープンソースになったそうです。それでLinux版も出てきました。
さっきインストールして、今使ってみています。Primeみたいに予測入力システムになっています。元のGoogle日本語入力の辞書はインターネット上から生成されたそうですがオープンソース版は別の辞書を使用しているそうです。もっと入力しないとPrimeとAnthyとどう違うか、どれが一番使い安いか分かりませんな〜。
とにかくその入力ソフトがibus上で働くので、ibusも必要です。

2010-05-11 10:33:36+0900

p.d.o | No Comments »

May 3rd, 2010

Important milestone for Iceweasel in Debian

I just uploaded a pre-release of Iceweasel 3.6.4 to experimental. This is an important milestone for several reasons:

  • Last time I was able to push a pre-release was almost 2 years ago. This means the keeping up with upstream is mostly done. This also means I should be ready to create a 3.7alpha5 package when upstream releases it.
  • It includes a new feature: Out of Process Plugins. This means plugins instances are sandboxed and if they crash, they won't take the browser down with them (finally). In this release, only the Adobe flash plugin is sandboxed, but it should be enough for the vast majority of plugin induced crashes.
  • Upstream is particularly interested in feedback from Debian (experimental) users for this pre-release, as they usually don't get much feedback from their own linux pre-release builds except from the test suite.

2010-05-03 14:49:12+0900

firefox, xulrunner | 10 Comments »

April 26th, 2010

日本語入力をibusで

最近まで、scimを使っていましたが、一週間前ibusを使ってみました。Input Method自体がまだanthyですけれどナントナクibusの方が使い易いというか楽というか。

とにかく、Debianでインストールするのは簡単です。最初はrootで:

$ apt-get install ibus-anthy ibus-gtk ibus-qt4 (Gnome/Gtk+のソフトだけを使ってる場合はibus-gtkだけでいいですし、KDE/Qtの場合はもちろんibus-qt4だけでもいいです)

次にユーザで:

$ im-switch -s ibus

2010-04-26 09:11:26+0900

miscellaneous, p.d.o | No Comments »

April 19th, 2010

A new world of possibilities is opening up

Account created for myaddress<at>glandium.org with hg_mozilla and mg_mozsrc bits enabled.

Yes, this means I now have commit access on hg.mozilla.org. Thanks to those who vouched for me, namely Justin Wood, Christian Biesinger and Benjamin Smedberg.

And thanks to the Mozilla governance for having changed the commit access requirements just in time for me to take advantage of the new ones: under the old rules, it was required that the vouching superreviewer had never reviewed a patch from the person applying for commit access. Of the 31 superreviewers, 14 (almost half of them) already had reviewed one or more patches from me (which is not really unexpected, considering the number of patches I sent in the past).

One could wonder why I never applied for access earlier, though.

2010-04-19 23:21:57+0900

firefox, xulrunner | 5 Comments »

April 15th, 2010

Re: git reflex

Joey Hess writes about putting snapshot.debian.org into git, which is something I already mentioned on this blog. It turns out I used the snapshot.debian.org data a few months ago to try the idea again, and while I haven't done the math yet with the data I got in the end, the data during the experiment tells me we shouldn't expect much more than a 50% space reduction.

There are two main reasons for that: a few number of packages actually take a whole lot of the archive space, and a whole lot of packages are simply data that don't benefit from being put in a git repository.

2010-04-15 08:13:13+0900

debian | 3 Comments »

April 3rd, 2010

Iceweasel 3.6.3を公開

2010-04-03 01:10:09+0900

firefox, xulrunner | No Comments »

Announcing Iceweasel 3.6.3

2010-04-03 01:09:43+0900

firefox, xulrunner | 8 Comments »

April 2nd, 2010

Iceweasel 3.6のプレビューパッケージのセキュリティアップデート第2号

experimentalへxulrunnerの1.9.2.2バージョンをアップロードしましたが、ftp-masterが壊れていて、そして、NEWでftpmastersの賛成が必要なので,すぐにはaptでダウンロード出来ません。ですから、下記のパッケージを使ってみて下さい。

後数時間でiceweaselも出来ると思います。

2010-04-02 16:33:17+0900

firefox, xulrunner | No Comments »