The Advisory Boar

By Abhijit Menon-Sen <ams@toroid.org>
2010-05-11

Git: post-receive hook for XMPP notifications

I wrote long ago about the trouble I had with Net::XMPP while setting up a notification hook for Archiveopteryx, but I didn't think anyone would find the script itself particularly interesting. But people have asked me about it, so here it is.

I wrote a Perl script named post-receive-notify-xmpp.pl that sends a message via XMPP (Jabber) every time something is committed to a repository. It depends on the Net::Jabber module (and perhaps also Net::DNS, but see below). You can download and install it directly as a Git hook:

$ cd .git/hooks
$ wget -O post-receive \
    http://toroid.org/ams/post-receive-notify-xmpp.pl
$ chmod +x post-receive

(I tend to just call this from a separate .git/hooks/post-receive script, because I sometimes need to do other things in the hook, like updating mirrors.)

You have to tell the hook whom to send notifications to, and optionally which branches to pay attention to (failing which it pays attention only to master):

$ git config hooks.notify.branches 'master next foo'
$ git config hooks.notify.recipients 'a@example.org b@example.org'

Finally, you have to edit the script to set the correct jid and password to use when sending the notifications, and the name used to identify the repository to the recipients:

my $repo = "foo.git";

my $JID = 'foo@example.org';
my $password = "cd2e1f7d4a8500f394771b620a85aa5e";

If you want, you can also edit the "git log" command in the script to include whatever information you want in each change description (like --diffstat and so on).

That's all there is to it. Let me know if you have any problems getting it to work.

(If you want commit notifications by email instead, look at the post-receive-notify-email script in Git contrib/hooks.)

The old Net::XMPP bug

If the XMPP server you want to connect to in order to send notifications has the same hostname as the jid you want to use, you're fine. In other words, if you want to send a notification as foo@example.org and your XMPP server is reachable at example.org, you're safe from the bug.

If XMPP service for your domain is provided by a different server, i.e. you want to send the notification as foo@example.org, and example.org has SRV records that point you to xmpp.example.net, then you will need to install Net::DNS and patch Net::XMPP (which is a lower-level dependency of the Net::Jabber module) to cope.

See my earlier post for details about the problem and the patch.

Tags: git, aox • Link: etc/git-post-receive-notify-xmpp

2010-05-02

Managing release branches: git merge vs. p4 integrate

When the Archiveopteryx source code lived in Perforce, we would submit everything to the src/main branch, review and test, then use p4 integrate to merge selected changes into release branches like src/rel/2.0. The only changes we submitted directly to the latter branches were release-specific, like setting the version number in Jamsettings. We could safely re-run p4 integrate at any time, and it would show us only those changes that we had not already reviewed.

When we moved to git, we continued to work this way—development happened in master, and we would use git cherry-pick to integrate or backport selected changes into older release branches. New release branches were created by branching from the current master, and maintained the same way. We did this for almost two years and several releases, but it was not much fun.

There was no easy way to answer the question Which commits do I need to consider for inclusion? for any given release branch. In theory, git log --cherry-pick will tell you, but it doesn't work very well. We used to do monthly releases, but it was so painful to deal with the build-up of commits at release time that we were forced to backport changes in smaller batches throughout the month (but that was not, in itself, a bad thing).

Read more…

Tags: git, aox • Link: etc/git-merge-vs-p4-integrate

2010-04-17

#ifdef considered harmful

Speaking of portability, here's a link to Henry Spencer and Geoff Collyer's classic 1992 USENIX paper #ifdef Considered Harmful, or Portability Experience With C News.

We believe that a C programmer's impulse to use #ifdef in an attempt at portability is usually a mistake. Portability is generally the result of advance planning rather than trench warfare involving #ifdef.

It's been eighteen years since its publication, but not enough people have read that paper yet.

Tags: aox • Link: etc/spencer-ifdefs

2010-04-06

Mirroring a git repository

In a recent conversation on the Archiveopteryx mailing list, someone suggested that we move the code to Github because it feels very far away right now (hosted in a git repository on our own server). Some people agreed, and said that a project hosted on github (or SourceForge, or code.google.com) would get us more visibility, while others strongly preferred the status quo.

Although we decided against moving (and will instead focus on other ways to gain transparency and visibility as the project moves away from being company-driven), it was clear that having a mirror of our repository on Github (or elsewhere) couldn't hurt; and that's what this post is about.

The ten-second summary

The Archiveopteryx source code lives in a repository on git.aox.org, and the developers push commits to it. We set up github and gitorious as remote repositories on the server, and added a(nother) post-receive hook to push any new commits on to those two repositories. Voilà! Zero-effort mirrors of our code.

Gory details

On Github: I created an "aox" account, created an "aox" repository, and added arnt and myself (amenonsen, since both ams and abhijit were taken by people who aren't using their accounts) as collaborators.

On Gitorious: I created an account for myself (ams was free!), created a project named "aox", created a new "aox.git" repository, and added arnt and myself as collaborators on the project.

Then, in aox.git on git.aox.org, I did an initial push to both repository mirrors:

$ git remote add github git@github.com:aox/aox.git
$ git remote add gitorious git@gitorious.org:aox/aox.git
$ git push github
...
$ git push gitorious
...

To automatically push commits to both repositories in future, I created aox.git/hooks/post-receive with the following contents:

#!/bin/bash

nohup git push github &>/dev/null &
nohup git push gitorious &>/dev/null &

Update (2010-04-13): I put these two pushes into the background because I didn't want to wait for them to finish every time I pushed something to git.aox.org. But I was relying on ssh agent forwarding to authenticate with the remote servers, and that didn't work once my ssh client had disconnected. (I forgot to mention earlier that I have an entry in .ssh/config that sets ForwardAgent on for git.aox.org.)

So I switched back to blocking pushes, but that was so slow that Arnt and I decided to generate new keys to push to these repositories, and put them on git.aox.org rather than rely on agent forwarding. So the pushes run in the background again now, and it works fine.

Github and multiple accounts

The Github Terms Of Service say you shouldn't share accounts or create multiple accounts.

I found out about those restrictions not—I'm ashamed to admit—by reading the TOS but when I tried to add Arnt's SSH public key to the aox account so that he could push to it. It didn't work, because he'd added the same key to his own newly-created account. I was able to add my own key only because the amenonsen account I had created some time ago had an older ssh key on record.

I asked for advice, and was told that I could set up an aox account with no ssh keys, and just add anyone who was supposed to push to aox.git as a collaborator on the project. That sounded good, and seems to be common practice, but I didn't want to go against the spirit of the TOS. I found an old question on the Github support site where someone from Github says We try to discourage people from creating "project" accounts, but we won't stop you from making one.

That answer was dated December 2008, however, so I asked Scott Chacon of Github on IRC today if the practice was still discouraged. He said it was no problem, so I left our github mirror active.

Tags: aox • Link: etc/git-repository-mirrors

2010-04-02

Portability and optimism

We've always taken a very conservative approach to portability with Archiveopteryx. We run it ourselves under Linux and FreeBSD on x86 and x86_64 systems, and those are the only platforms we've ever claimed to support. Between the developers and our major users, we have access to enough systems of that description that we can be reasonably confident that the software will work without unpleasant surprises.

Beyond that, we've never been very ambitious. Once in a while, someone would ask “Does it work on NetBSD?” or Solaris or OS X. The answer was usually yes, perhaps after a few (relatively minor) tweaks. We would add #ifdefs and other portability patches to our code only after someone suffered from an actual problem, never before. If we couldn't (or didn't, regularly) test it, it wasn't supported.

This gloomy approach is in sharp contrast to the confidence encouraged by autoconf. Had we used it, we could have saved a few users the trouble of dealing with compile errors. We would have been testing our software only on the few systems we did anyway (which happen to be the commonest platforms in use), but autoconf would have helped us to feel more portable. I don't want to think about what that would have done to the number of unreproducable bugs we had to analyse; they were enough trouble as it was.

I was recently reminded of our curmudgeonly attitude in a conversation with Hassath, who did a project on FLOSS video editing software. One of her complaints was that it was hard to find out not just what the hardware and software dependencies of some program were, but also what configurations were known to work. The documentation usually implied that any Linux system would work fine, but in practice, people running a different distribution routinely encountered problems that the developers had never seen and had no idea how to address. This was the case even on her quite unremarkable Ubuntu and amd64 test system, and she found it extremely frustrating.

Try as I might, I can't see optimism as a good approach to portability.

Tags: aox • Link: etc/portability-and-optimism