So everyone knows my job at Google is to tech-lead the team responsible for our Subversion servers, as part of our larger open-source project hosting service. Thus, having come to a reasonable temporary stopping point with my previous 20% project, I’ve turned to a new 20% project: making Subversion itself better.

Specifically, I want to right a wrong, undo something I’ve felt nasty about for years. Subversion’s HTTP protocol is very complicated and unintelligible to mere mortals. Honestly, if Greg Stein and I got hit by buses, nobody would really understand what’s going on inside mod_dav_svn. What’s the backstory here? Basically, we tried to make mod_dav_svn implement a reasonable subset of DeltaV, which was a mostly-failed spec written long ago to implement Clearcase^H^H^H^H version control over HTTP. Eight years later, this extra complexity hasn’t bought us any interoperability with other version control systems — just a big headache to maintain and a icky performance penalty. The Subversion client, in being a “good DeltaV citizen”, isn’t allowed to directly talk about URLs that represent revisions, transactions, historical objects, and so on. Instead, it has to play dumb and continually issue a series of requests to “discover” opaque URLs that represent these concepts. It’s sort of like the client playing a formal game of 20 Questions, when it already knows the answers.

So after some chats with Greg Stein and others, I’ve collected ideas on how to streamline our existing protocol into something much more simple, tight, and comprehensible. Way fewer requests too. You can read our evolving design document and send questions/feedback to dev@subversion.tigris.org. Subversion 1.6 is planned to be released at year’s end, so if we’re lucky we’ll see this new protocol in Subversion 1.7 next summer.

Speaking of Subversion 1.6, however: a smaller sort of glastnost is happening there as well. In this new spirit of HTTP openness, we’re officially ending our policy of “not telling people how to access older revisions” over HTTP. If you recall, the Subversion book has always said:

Q: Can I view older revisions?
A: Your web browser speaks ordinary HTTP only. That means it knows only how to GET public URLs, which represent the latest versions of files and directories. [...] To find an older version of a file, a client must follow a specific procedure to “discover” the proper URL; the procedure involves issuing a series of WebDAV PROPFIND requests and understanding DeltaV concepts. This is something your web browser simply can’t do.

I’m here to break the chains! Reveal the lies! RELEASE THE KRAKEN. In Subversion 1.6, we’ve gone and implemented an official public query syntax for accessing older (revision, path) coordinate pairs:

http://host/repos/path?r=REV
http://host/repos/path?p=PEGREV
http://host/repos/path?p=PEGREV&r=REV

This query syntax offers the same peg-revision concept that one sees in the Subversion commandline client. The first syntax means “start at PATH in the latest revision, then follow the object back in time to revision REV.” This works even if the object was renamed and exists at a different place in the older revision. The second syntax allows one to pinpoint an object with no history tracing: just jump to revision PEGREV, and find PATH. The third syntax is very much like running “svn subcommand -r REV path@PEGREV”: start at PEGREV, find PATH, then trace the object back into older revision REV.

In any case, this means source code browsers and other tools can stop using “secret” internal urls to access older objects.

[cue Doogie Howser theme music]

Dear diary,

Google is probably one of the only companies in the U.S. that gives an actual paid paternity leave (7 weeks!)… why, it’s almost as civilized as living in Europe. It not only gives us a chance to bond with babies, but try to reset our lives as we walk around in a sleepless stupor. Highlights of my downtime so far:

• When mommies and babies are sleeping, it’s still possible to practice the banjo with the magic of Mike’s Mute, the mostly highly rated of mutes on all of Banjo Hangout (where the cool kids are!). The photos don’t do it justice. It just clamps onto the bridge and makes your banjo as quiet as an unplugged electric guitar… yet it still sounds like a banjo!
• Did you know that the bottom of a newborn baby’s foot, being completely unused, looks exactly like the bottom of a hand? Delicate little lines everywhere — the same ones palm readers look for. It’s amazing to think that those features are hiding under the thick dead layer of smoothness on the bottom of adult feet. In that same vein, did you know that when a newborn straightens out his arm, all one can see is a single puffy tube of flesh? There is no visible elbow, no indication of how or where the arm bends… thus proving my scratchy elbows are a product of nurture, not nature.
• As the temperatures drop into the 40’s in Chicago, our pantry is overflowing from potatoes delivered by our local community supported farm. It’s time again to dig into my favorite recipe for potato leek cheese soup. Seriously: just boil some potatoes and leeks, add cream cheese, and you’re in heaven. Blending with a stick blender is strictly optional; I can’t get enough of the stuff. Go make this soup right now.

Back to the diapers.

One of the cool trends I’ve seen is the use of distributed version control systems as “super clients” against Subversion. You suck down the entire history of a Subversion repository into a local, private repository, do all of your commits locally, make branches, experiment all you want, then “push” back up to Subversion again. On the internet, nobody knows you’ve been using DVCS (or that you’re a dog.) What’s particularly cool about these bridging tools is that they allow users to try out DVCS before deciding to officially convert a whole project over. Or, if a project happens to be using Subversion but you still want most of the power of a DVCS for local work, it’s a perfect solution.

For all the blabbing I’ve done about distributed version control systems, I’m still a big fan of Mercurial. Of all the DVCSes, I think it’s the easiest to learn for svn users. It has a small, tight set of commands, and the community which runs the project is polite and sane.

In any case, there have been a collection of Mercurial-Subversion bridges available for the last couple of years, but they’ve all been deficient in various ways: either not capturing svn history entirely, or being unable to push back to svn correctly (or only very awkwardly). So I’ve pretty much stayed away. But today I want to plug a new bridge written by a friend of mine ( Augie Fackler) who finally did it Right: he wrote a bridge called hgsubversion which (1) uses the actual Subversion API to pull history down (which is faster, more accurate, and long-term sustainable), and (2) actually knows how to push changes back to Subversion correctly. I want the world to be aware of this tool, because I think it’s the first Mercurial-Subversion bridge which deserves to be promoted into the popular ranks with tools like git-svn.

The tool is still young and not generally installable by the public (i.e. you’re not going to find any magic .rpm, .dpkg, .zip or .dmg for it yet)… but here are my cliff notes if you want to start playing with it.

Requirements

• The latest (unreleased) Mercurial
• Local Subversion libraries, at least 1.5, with swig-python bindings built
• A Subversion server that is 1.4 or later

To get the latest Mercurial:

$ hg clone http://selenic.com/repo/hg hg-latest
$ cd hg-latest
$ make
$ sudo make install

To get the latest Subversion python bindings:

$ # if you don't have a binary package for svn-1.5-python-bindings already,
$ # this is a summary of subversion/bindings/swig/INSTALL instructions:
$ svn checkout http://svn.collab.net/repos/svn/tags/1.5.3 svn
$ cd svn
$ ./autogen.sh && ./configure
$ make
$ sudo make install
$ make swig-py  # make sure you have swig 1.3 installed already
$ make check-swig-py
$ sudo make install-swig-py

To get hgsubversion:

$ hg clone http://bitbucket.org/durin42/hgsubversion/ ~/hgsubversion
$ cat >> ~/.hgrc
[extensions]
rebase=
svn=/home/user/hgsubversion
^D

To make sure you’re ready to go, do a final sanity check:

$ python -c "import svn.core; print svn.core.SVN_VER_MINOR"
5
$ # if you get something less than 5, you may have conflicting
$ # versions installed, and may need to set PYTHONPATH

Now you can clone your favorite svn repository, and use it locally:

$ hg svnclone http://svn.example.com/repos hg-repos
converting r1 by joe
A trunk/
committed as 24dfb7b51d606a921333e2b8f19a9a6aa5661a69 on branch default
[...]
converting r100 by jane
M trunk/goo
M trunk/moo
committed as 54dfb7b51d6d6a931333e2b8f19a9a6005661a62 on branch default
$

The tool currently assumes a ’standard’ svn layout of /trunk, /branches, /tags, and then tries to pull them into sane mercurial equivalents. After you’ve made a bunch of local commits, you can push the changes back to subversion:

$ # First merge the latest public svn changes into your repository:
$ hg svn pull
converting r101 by pinky
A trunk/awesomefile
committed as e85afd44dc83d5df2599157096a95b0868de6955 on branch default
$ hg up -C
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ # We now have two hg HEADs, because the public svn changes are
$ # considered a different line of development from my own.
$ # For now, rebasing is preferred to merging:
$ hg up -C 9985f017b3ab
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg svn rebase
saving bundle to /home/user/project/.hg/strip-backup/c4b9dfce6b09-temp
adding branch
adding changesets
adding manifests
adding file changes
added 4 changesets with 4 changes to 4 files
rebase completed
$ # At the moment, each changeset is pushed separately;
$ # changeset flattening not yet implemented
$ hg svn push