Archive for July, 2007

Nicer, compared to previous version.

Following the current trend on planet... or not.

Ganneff, I could not agree more with you, especially since it matches what I first replied to aj's proposal. That went unnoticed, though.

So, after having dumped Mozilla Suite, the Mozilla Foundation is now pushing Thunderbird out to pursue its goal of an "open web" (understand: MoFo only cares about Firefox).

Why is it called the Mozilla Foundation, again ?

... is that more than one are likely to have the same crazy ideas.

When I started to play with xulrunner, which according to my oldest post in the xulrunner category would be near 2 years ago, I had this crazy idea (and here, when I say crazy, I almost think dumb) that it would be neat to have a window manager based on libxul, being able to display both "native" windows and XUL or HTML windows.

Indeed, it would be neat, as in web-2.0-hype or i-can-use-gmail-directly. But that would be so much impossible to secure, and introduce so many different new ways to compromise users...

Guess what... It now exists.

Update: Waw, the GUADEC keynote slides are really full of crap. My favorite ones are about "The Fox" : Good engineering practices and Small, extensible core.

Guess what happens when someone complains to ffmpeg developers that their software management and API suck. Well, he gets answered repeatedly to include a CVS snapshot of ffmpeg and link statically against it, "like everyone does". Splendid.

... is that more than one are likely to have the same crazy ideas.

A while ago, when I was fooling around with VMware ESX and wrote diskimgfs (which I hope to be able to release some day), I found another implementation of the same root idea (without partition level access, though) in dm-userspace.

A few months ago, Sam posted his code for a SSH/HTTPS port sharing program, idea of which had been on my mind since a few weeks earlier.

And now that I'm playing around with git, I was thinking about having a FUSE filesystem to access a git repository. Guess what, it already exists.

And it actually happens quite often. This is also one of the reasons there are so many implementations of the same things (another being the NIH syndrome).

I got the idea reading this question on the webkit-dev list, and from my recollection of the git documentation. Well, for the webkit case, the basic script I put up would be of no help because the tarballs don't contain all the files (plus, they use subversion, not git, so it would also require a long importing process). Anyways...

Considering how SHA-1 hashes of objects are created with git, it is actually pretty easy to generate a SHA-1 hash from a random (non-git) tree, and then, the corresponding commit. First, you have git-hash-object that helps you creating a hash for a particular object (though it's also trivial to do with sha1sum). For regular files, git-hash-object -t blob $filename is enough. For symbolic links, you have to read the link destination, and give it without a trailing character (be it the NULL character or a carriage return) to git-hash-object -t blob --stdin. For directories, you have to generate a tree "structure" by yourself and pass it to git-hash-object -t tree --stdin. I haven't bothered looking at other file types.

The tree structure can be guessed by either looking at mktree.c or at the output of git-cat-file tree $sha1 where $sha1 is the SHA-1 hash for a tree object. It contains 3 informations for each node in the tree : the file mode, with the same format as what stat() returns, except for some reason, permissions are 000 for directories and symbolic links ; the file name ; and the SHA-1 hash. These informations are written with the following format : file mode in octal ascii and no padding zero ("%o") followed by a space character, then the filename followed by a NULL character, and the binary form of the SHA-1 hash.

Nodes are sorted in a not-so-quite lexical order (take a look at base_name_compare in read-cache.c) and are not separated by any special character: the mode of a file just follows the SHA-1 hash of its predecessor.

With all this new knowledge, you should be able to write some code that would return the SHA-1 from an arbitrary directory. Okay, since you must be at least as lazy as I am, you can take the script I wrote.

Now, let's take a look at a real life case : what commit is the latest nightly snapshot for the linux kernel from ? First, download the latest snapshot patch and its baseline, and extract the whole. Then, run my git-hash-tree.pl script with the directory containing the extracted kernel as an argument. It will return, after a while, the SHA-1 hash for the whole tree. During this long process, you also have plenty of time to git clone linus's tree.

Once you're all done, you can search for the commit corresponding to the tree hash (let's call it $hash) with the following command :

git-rev-list --all | while read h; do git-cat-file commit $h | grep -q "^tree $hash" && echo $h && break; done


If you just followed these steps, you should just have spent a great moment having no result at all. There are actually 2 things that prevent this method to properly work with the linux kernel nightlies :

• The snapshot patches contain a change to the top Makefile that doesn't exist in the repository. You need to remove the -gitn from the EXTRAVERSION variable in the Makefile.
• git diff only includes diff headers for removal of empty files, so if you apply the snapshot patch with the patch utility (and you can't apply it with git-apply since you don't have a .git directory), empty files that were marked as deleted will still be on your tree. It happens with the current snapshot patch (2.6.22-rc7-git6): it doesn't remove include/asm-blackfin/macros.h.

Note this is a naive method, because I haven't dedicated much time going through git documentation and code to find better ways, if there are any. Also note it's pretty much worthless to do this with the kernel nightly snapshots, since a file containing the SHA-1 hash of the corresponding commit can be found alongside the patch.

I guess a similar method could be used with mercurial, though I could not find a documentation detailing what are the hashes calculated from (I've not searched a lot, I must say, but for git, it was just before my eyes).

After Camino and Seamonkey, Epiphany is joining the let's have a dumb UA club.

Instead of actually fixing the problem at its root by forcing stupid people not to try to match Firefox, by totally removing Firefox in Firefox's UA, which would definitely have had a good result, they go the opposite direction.

If I see that crap coming in Debian in either epiphany, galeon or kazehakase, or any other browser based on libxul, I promise I'll hack the thing that puts the user-agent together so that it suppresses all occurrences of Firefox from it.

If only people at mozilla were as thoughtful as Robert Kaiser...

I've been using svk for xulrunner maintenance since the first test packages. And I've been thinking about switching to git for a while now. I did some testing in march, which led to my post about diffing. I went a bit further today by actually trying to convert my svk repository. And I got both interesting and surprising results.

The repository is (only) about 250 revisions in svk, and includes 15 upstream releases (or CVS pulls). I had to reorganize the svk repository so that git-svnimport could do its work properly.

[ Note: these tests have been done on a 5 years old laptop ]

The first surprise is the time it took for the conversion : almost half an hour. Dumping (svnadmin dump) and reloading (svnadmin load) the svk repository takes a bit more than 3 minutes, in comparison... I guess git-svnimport is very ineffective.

The second surprise is that the svk repository is actually almost twice as small as the git equivalent: 116MB for svk vs. 224MB for git.

On the other hand, git is amazingly fast. One of the most common things I do on this repository is a diff between the upstream branch and the trunk. I create the .diff.gz file I give pbuilder this way. It's much quicker than doing a real dpkg-source, especially considering pbuilder does one itself.

After dropping caches (echo 3 > /proc/vm/sys/drop_caches), such a diff takes about 50 seconds with svk and... a little more 2 seconds with git. Without dropping caches, it gets down to 38 seconds with svk and... 0.05s with git. That is pretty damn fast !

Update: after a git repack -d, the git repository is down to 61MB.

"12 years old harddisk" - I don't own any that old.
"dead browsers" - I see them.
"dm-crypt luks header size partition block device" - 1032 blocks for 128bit keys and 2056 for 256bit ones.
"ebtables vmware server" - I'd like it too.
"fact on what horse represents" - We definitely have a recurring theme.
"hobbit debian install" - I didn't know Debian was popular in the Shire.
"i agree with you blog" - I don't
"perverts" - Where ?
"what kind of files are needed to installed properly a psp7 plugin" - damn, how did you end up here ?
"what version of debian 4 do we use" - hum... 4 ?