Inspector34: record/replay HTTP requests

By Abhijit Menon-Sen <>

Inspector34 is a transparent web proxy that records requests and responses for later playback and comparison. It is meant to help with regression testing.

It consists of three programs: i34-record is the proxy server, which keeps a journal of HTTP requests and responses; and i34-replay recreates the original requests from this journal. i34-diff compares responses from the original and replayed sessions.

Inspector34 is written in Perl, and is distributed under a BSD-style open source license. Feedback is welcome.

Usage

Here is a brief example of how one might use Inspector34 (complete documentation is included with the program).

Suppose your web server is listening at www.example.org:80. You would start i34-record (probably, but not necessarily, on the same machine) like this:

$ i34-record from www.example.org:8080 to www.example.org:80 in orig

Then you point a browser at http://www.example.org:8080 and interact with the site, while i34-record records a session transcript under the directory orig. When you're done, you restart i34-record like this:

$ i34-record from www.example.org:8080 to www.example.org:80 in new

Then run "i34-replay orig" to recreate the HTTP requests recorded in orig, and send them through the proxy again. You will end up with a second transcript under the new directory. To compare the two, you would run:

$ i34-diff orig new

That's all there is to it.

Download

Download inspector34.tar.gz (from github.com/amenonsen/loathsxome)

Inspector34 depends on Rocco Caputo's excellent POE framework, as well as the HTTP::Parser and POE::Filter::HTTP::Parser modules. These are all available from the CPAN.

At the time of writing, HTTP::Parser 0.04 needs a trivial patch in order to work correctly. I have sent the patch to the author, and I hope an updated 0.05 release will be available soon. In the meantime, the diff is included in the distribution above, along with instructions on how to apply it.

This code is provided here with no warranty, and may be freely used, modified, or redistributed provided that derivative works are clearly identified as being different from the original, and copyright notices in the source are preserved. (The package includes a complete license statement.)

History

In October 2007, my friend Josh Purinton asked me if I wanted to write a prototype of a transparent proxy that could record and replay requests. I said yes; and we agreed that the program would be released under an open-source license. Josh suggested the name inspector34, after a perfectionist underwear quality inspector from an episode of "The Adventures of Pete & Pete".

I used Philippe Bruhat's very convenient HTTP::Proxy module to write the first version, which was ready by the end of January 2008. It worked well enough as a proof of concept, but it took me a while to figure out how to cooperate with HTTP::Proxy's somewhat eccentric callback behaviour.

Even after jumping through some hoops, the program was troubled by bugs in and around HTTP::Proxy that showed up in corner cases; and the fork-per-request model made it unsuitable for its intended use in production.

So I welcomed an opportunity in May 2008 to rewrite it using POE, this time adding a comprehensive test suite, and making it robust enough for real use. Due to inexcusable delays on my part, the final version was ready only in mid-March 2009. Nevertheless, it is a great improvement over the prototype.

Acknowledgements

The development of Inspector34 was sponsored by The Daily Source, a news aggregator—like Google News, but "with human editors and higher ideals".

Josh Purinton deserves thanks not only for coming up with the idea, but also for suggesting a much cooler name for the program than I would ever have thought of myself.

Inspector34 is dedicated to my memory of Bertie, who didn't quite live long enough to see it working (and wouldn't have known what to make of it if he had).