Avoiding asynchronous callback hell in Archiveopteryx

I've read many mentions of “callback hell” recently, especially in discussions about Javascript programming. This is the problem of deeply nested code when trying to manage a sequence of asynchronous actions. The many suggested remedies range from splitting the code up into many small named functions to async to promises to futures (and probably other things besides; I haven't tried to keep up).

FutureJS, for example, is described as helping to tame your wild, async code.

I have no opinion about any of these solutions. I don't work with any complex Javascript codebases, and asynchronous actions in my Mojolicious applications have been easy to isolate so far. But I do have opinions about writing asynchronous code, and this post is about why I'm not used to treating it as though it needed “taming”.

Here's some asynchronous code picked more or less at random from Archiveopteryx: the handler for the IMAP CREATE command (edited to remove uninteresting code to generate better error messages).

void Create::execute()
{
    if ( state() != Executing )
        return;

    if ( !d->parent ) {
        d->parent = Mailbox::closestParent( d->name );
        if ( !d->parent ) {
            error( No, "…syntax error…");
            return;
        }

        requireRight( d->parent,
                      Permissions::CreateMailboxes );
    }

    if ( !permitted() )
        return;

    if ( !transaction() ) {
        d->m = Mailbox::obtain( d->name, true );
        setTransaction( new Transaction( this ) );
        if ( !d->m ) {
            error( No, "…invalid name…" );
            return;
        }
        else if ( d->m->create( transaction(),
                                imap()->user() ) == 0 )
        {
            error( No, "…already exists…" );
            return;
        }
        Mailbox::refreshMailboxes( transaction() );
        transaction()->commit();
    }

    if ( !transaction()->done() )
        return;

    if ( transaction()->failed() ) {
        error( No, "…database error…" );
        return;
    }

    finish();
}

This code is straight-line enough that I only had to break two long lines for it to fit in a 37em-wide web page, but its operation is entirely asynchronous.

The Create class inherits from the EventHandler class. Any object of this class can have its execute() method called at any time—the code must be able to figure out what remains to be done and continue its work. Asynchronous processes receive a copy of the caller's this pointer, perform some operations (e.g. database queries), and call execute() on the invoking object. A few carefully-written helper functions make the control flow obvious.

In the example above, the method is first called by the IMAP handler. It calls requireRight() on the parent mailbox, and expects to be called back when permitted() is true. (If the permissions are not sufficient, requireRight() issues a suitable error.)

Then the code sets up a transaction to create the requested mailbox and executes it via commit(), expects to be called back when it's done(), issues an error if the transaction failed(), or completes successfully. Once setTransaction() is called, transaction() returns a pointer to the current transaction, so that branch will never be executed on future invocations of execute().

This pattern of checking each pre-requisite one by one and updating state along the way scales cleanly to more complex commands, including those that issue database queries, wait for the results, and then issue more queries, all within a transaction.

Notice that this code doesn't look dramatically different from blocking code, but it's easy to read and understand its requirements and the sequence of operations. We depended on this property while reviewing code.

Arnt and I were using this technique extensively in 2003. I'm not making any claims about its wider applicability, but it certainly prevented Archiveopteryx from ever descending into callback hell.