[Update: git-remote-hg being a moving target, these instructions are now outdated. Please now refer to this new article on the git-cinnabar wiki that will be updated with fresh instructions.]

If you've been following this blog, you know I've been working on a new tool to allow to use git with mercurial repositories. See the previous blog posts for some detail if you don't know what I'm talking about.

Today, a new milestone has been reached. After performing tens of thousands of pushes^[1] and having had no server corruption as a result, I am now confident enough with the code to remove the limitation preventing to push to remote mercurial servers (by an interesting coincidence, it's exactly the hundredth commit). Those tens of thousands of pushes allowed to find and fix a few corner-cases, but they were only affecting the client side.

So here is my recommended setup for Gecko development with git:

• Install mercurial (only needed for its libraries). You probably already have it installed. Eventually, this dependency will go away, because the use of mercurial libraries is pretty limited.
• If you can, rebuild git with this patch applied: https://github.com/git/git/commit/61e704e38a4c3e181403a766c5cf28814e4102e4. It is not yet in a released version of git, but it will make small fetches and pushes much faster Update: The patch is in git version 2.2.2 and newer.
• Install this git-remote-hg. Just clone it somewhere, and put that directory in your PATH.
• Create a git repository for Mozilla code:
  

  $ git init gecko
  $ cd gecko

• Set fetch.prune for git-remote-hg to be happier:
  

  $ git config fetch.prune true

• Set push.default to "upstream", which I think allows a better workflow to use topic branches and push them more easily:
  

  $ git config push.default upstream

• Add remotes for the mercurial repositories you pull from:
  

  $ git remote add central hg::http://hg.mozilla.org/mozilla-central -t tip
  $ git remote add inbound hg::http://hg.mozilla.org/integration/mozilla-inbound -t tip
  $ git remote set-url --push inbound hg::ssh://hg.mozilla.org/integration/mozilla-inbound
  (...)
  

  -t tip is there to reduce the amount of churn from the old branches on these repositories. Please be very cautious if you use this on beta, release or esr repositories, because their tip can switch mercurial branches, and can be very confusing. I'm sure I'm not alone having pushed something on a release branch, when actually intending to push on the default branch, and that was with mercurial... Mercurial branches and their multiple heads are confusing, and git-remote-hg, while it supports them, probably makes the confusion worse at the moment. I'm still investigating how to make things better. For Mozilla integration branches, though, it works fine because their tip doesn't switch heads.

• Setup a remote for the try server:
  

  $ git remote add try hg::http://hg.mozilla.org/try
  $ git config remote.try.skipDefaultUpdate true
  $ git remote set-url --push try hg::ssh://hg.mozilla.org/try
  $ git config remote.try.push +HEAD:refs/heads/tip
  

• Update all the remotes:
  

  $ git remote update

  This essentially does git fetch on all remotes, except try.

With this setup, you can e.g. create new topic branches based on the remote branches:

$ git checkout -b bugxxxxxxx inbound/tip

When you're ready to test your code on try, create a commit with the try syntax, then just do:

$ git push try

This will push whatever your checked-out branch is, to the try server (thanks to the refspec in remote.try.push).

When you're ready to push to an integration branch, remove any try commit. Assuming you were using the topic branch from above, and did set push.default to "upstream", push with:

$ git checkout topic-branch
$ git pull --rebase
$ git push

But, this is only one possibility. You're essentially free to pick your own preferred workflow. Just keep in mind that we generally prefer linear history on integration branches, so prefer rebase to merge (and, git-remote-hg doesn't support pushing merges yet). I'd recommend setting the pull.ff configuration to "only", by the way.

Please note that rebasing something you pushed to e.g. try will leave dangling mercurial metadata in your git clone. git hgdebug fsck will tell you about them, but won't do anything about them, at least currently. Eventually, there will be a git-gc-like command. [Update: actually, rebasing won't leave dangling mercurial metadata because pushing creates head references in the metadata. There will be a command to clean that up, though.]

Please report any issue you encounter in the comments, or, if they are git-remote-hg related, on github.

1. In case you wonder what kind of heavy testing I did, this is roughly how it went:

• Add support to push a root changeset (one with no parent).
• Clone the mercurial repository and the mozilla-central repository with git-remote-hg.
• Since pushing merges is not supported yet, flatten the history with git filter-branch --parent-filter "awk '{print \$1,\$2}'" HEAD, effectively replacing merges with "simple" commits with the entire merged content as a single patch (by the way, I should have written a fast-import script instead, it took almost a day (24 hours) to apply it to the mozilla-central clone ; filter-branch is a not-so-smart shell script, it doesn't scale well).
• Reclone those filtered clones, such that no git-remote-hg metadata is left.
• Tag all the commits with numbered tags, starting from 0 for the root commit, with the following command:
  

  git rev-list --reverse HEAD | awk '{print "reset refs/tags/HEAD-" NR - 1; print "from", $1}' | git fast-import

• Create empty local mercurial repositories and push random tags with increasing number, with variations of the following command:
  

  python -c 'import random; print "\n".join(str(i) for i in sorted(random.sample(xrange(n), m)) + [n])' | while read i; do git push -f hg-remote HEAD-$i || break; done

  [ Note: push -f is only really necessary for the push of the root commit ]

• Repeat and rinse, with different values of n, m and hg-remote.
• Also arrange the above test to push to multiple mercurial repositories, such that pushes are performed with both commits that have already been pushed and commits that haven't. (i.e. push A to repo-X, push B to repo-Y (which doesn't have A), push C to repo-X (which doesn't have B), etc.)
• Check that all attempts create the same mercurial changesets.
• Check that recloning those mercurial repositories creates the same git commits.

2014-12-23 08:10:04+0900

p.m.o

You can leave a response, or trackback from your own site.