The Advisory Boar
My friend Nicolai Langfeldt (author of the
DNS HOWTO) works at a
large ISP, where he recently needed to ping somewhere between 10,000 and
100,000 hosts every ten seconds and feed the up/down data to Nagios. He
had started hacking on fping to do this, but I thought it could be done
better by a smaller program written specifically for this purpose.
I wrote most of
on a flight home. It takes a list of IP addresses on the command line,
writes ICMP echo requests for each address to a raw socket every ten
seconds, and creates another process to print the response time (or
"unreachable") for each response. It does nothing else, and does not
need to allocate any memory after startup.
Nicolai's preliminary tests show that it can send out the 10k pings in
under 100ms, which far exceeds the performance I was hoping for (though
his networking colleagues may yet frown upon sending that many pings so
fast). With that kind of performance, monitoring 100k hosts isn't out of
the question, even if the sender has to be slowed down a bit.
The code is on github in case anyone else finds it useful. Feedback is
While looking for something that could be adapted for this use, I found
the source code of the original ping.c
on the ISOC web site. I decided against modifying it because of several
misleading comments, type abuse (e.g. allocating a struct sockaddr and
casting its address to struct sockaddr_in *), and other K&R/BSD
oddities. But it's good code overall, and it was instructive to look
announced snippets on rant.gulbrandsen last month, I thought I'd
write a quick introduction to it here as well.
Snippets is a Jabber bot that periodically asks its subscribers what
they're doing, and provides access to the answers through a simple web
interface. It is meant primarily to help teams of developers to
communicate with each other.
Snippets is a Perl program that depends on a few CPAN modules, and needs
access to a Postgres database. You can get the source from the
github page, and
installation instructions are given in the
Feedback and feature requests are welcome. Please send them to
Hassath wanted to download a couple of gigabytes worth of mail from her
GMail account for offline storage, but was frustrated by the IMAP server
closing the connection during large transfers, which made Thunderbird go
into a sulk, and forced her to restart the download by hand every time.
Using POP was not an option, since it had no way to preserve the labels
she had assigned to the messages (which show up as mailboxes via IMAP).
I tried some other programs, but they didn't work the way I wanted. Some
of them provided little or no progress information. Some would fetch all
of the messages into memory before writing anything to disk. Some would
fetch messages one by one, with separate command-response cycles. Most
importantly, none of them coped well with the frequent disconnections.
Having exhausted my patience, and being in need of a weekend hack, I set
out to write a program to download the mail. I tried the IMAP modules on
CPAN, but found them lacking in one way or another. IMAP::Client parses
IMAP responses in a fundamentally broken way (scanning literal data for
things which look like OK responses). Net::IMAP::Client doesn't provide
debugging information or allow MSN fetches. Net::IMAP doesn't know about
IMAP+SSL. Most importantly, none of the modules would allow me to stream
messages to disk sensibly. So I wrote one from scratch.
I started using Blosxom a few weeks
ago. I liked the basic idea (that each post is one file) very much, but
I found the code hard to read and poorly documented. Plugins varied
widely in quality, and I had to jump through hoops to implement some of
the things I wanted. By the time I got everything working the way I
liked it, I ended up with a complete rewrite.
The code is well-documented, supports tagging and pagination, generates
an RSS feed, works nicely with Git, and lets me publish URLs like
(Disadvantages: it isn't quite plugin-compatible with Blosxom, and it
doesn't support static generation.)
I'm sure I could have used a more modern, featureful program to achieve
the same effect, but I'm glad I didn't have to. I was happier to spend
a couple of hours writing something worse than a few days trying to fit
something better, like Wordpress,
into my head.
I wrote this code for my own use with no intention of releasing it, but
a few people have expressed an interest in using it, so here it is.
No longer maintained
Update (November 2013): Loathsxome never had more than a handful
of users, and only two who regularly contributed code
(Arnt Gulbrandsen and
myself). Arnt wrote
plusxome, and I wrote
something else using Mojolicious without even pretending to maintain
Loathsxome is still here. It works as well as it ever did, and none of
what's written below is untrue, but nobody uses it any more.
Inspector34 is a transparent web proxy that records requests and
responses for later playback and comparison. It is meant to help
with regression testing.
It consists of three programs:
i34-record is the proxy
server, which keeps a journal of HTTP requests and responses; and
i34-replay recreates the original requests from this
i34-diff compares responses from the original
and replayed sessions.
Inspector34 is written in Perl, and is distributed under a BSD-style
open source license. Feedback is welcome.
OpenSSL stores private keys in an
undocumented PEM format (the key data is DER-encoded and the result is
ASCII-armoured), which Cryptlib
does not support.
pemtrans reads an OpenSSL RSA private key and the corresponding signed
public key certificate, and writes a PKCS #15 keyset that Cryptlib can
use. The included manual page explains how to use the program.
Cryptlib requires (and respects) the KEYUSAGE attribute on certificates.
Some certificates do not contain this attribute, and pemtrans issues a
warning when it encounters them, because Cryptlib will probably reject
it when you try to use it for something, e.g. in a TLS server.
OpenSSL, by default, does not set KEYUSAGE. This can be fixed by
adding a line like the following to openssl.cnf before
generating the key and certificate (the attributes specified here are
enough for the result to be used in a TLS server):
keyUsage = cRLSign, digitalSignature, keyEncipherment, keyCertSign
Use, modification, and distribution of pemtrans is allowed without
any limitations. There is no warranty, express or implied.
Please send questions and comments to firstname.lastname@example.org.
This web page is about the most potent charm in my long-standing quest
for better time management: a program to help me keep track of exactly
where and how my time is spent.
Of course, I can't tell you how best to manage your own time, but just
as profiling is the best guide to code optimisation, I found it useful
(and sometimes very surprising) to begin by knowing exactly where time
goes. I've been very happy with the results, and I know of others who
approaches to the problem.
The idea is that you run a program (ts) every time you change what you
are doing. This sounds intrusive, and it does take a while to get used
to; but ts goes out of its way to make it easy, and it's almost second
nature to me now. I find the benefits worth this extra investment.
To run ts, you will need to have Perl installed along with the DBI and
DBD::Pg modules. You'll also need to provide a PostgreSQL database for
it to use, and you'll have to create the activities table from ts.sql.
The profiling data is stored in this very simple SQL table:
create table activities (
category text not NULL,
started timestamp(0) not NULL default current_timestamp,
check (stopped is NULL or stopped > started)
A row in this table describes a period of time spent on some activity.
The category is usually something like "work/mail" or "work/src", the
latter perhaps with a comment like "Fixing Jamfile". I find it useful to
organise my activities into a hierarchy, but you can decide for yourself
how you want to categorise things.
When you start a new activity ("ts work/src Fixing bug 42"), its started
time is set, and the stopped time of any activity that doesn't have one
(any activity that is in progress, in other words) is also set. You can
also specify the start and end times explicitly
("ts -s 12:00 -e 12:30 lunch").
My .zshrc contains "compctl -/ -W ~/.categories ts", and I've created
directories that reflect my categorisation under ~/.categories. Now I
can do things like "ts w<TAB>e<TAB>" instead of having to
type the category name ("work/email") in full.
Running ts without arguments explains how to use it.
Once the data is available, it's up to you to figure out how to use it
to your advantage ("How much time did I spend on the phone last week?"
"Let me generate a time sheet for work/some-client/code/" ...). Just
playing with the data in psql is often instructive.
The tsc program generates sc spreadsheets from the profiling data. It
presents, for a given time period (the last week by default), a list
of the categories matching some pattern (everything by default), and
a table (with totals) of how much time has been spent on them on each
of the days in the period under consideration. (You need a patched
version of sc to use tsc.)
I plan to write something that generates nicely-formatted time sheets
from this data, but I haven't gotten around to it yet. If you have an
interesting visualisation idea, please let me know.
Here's a tarball of the source code. It contains
the ts and tsc programs, the SQL table definition, a patch required for
sc to understand the tsc output, and some documentation. (You can also
download a patched version of sc.)
Please write to email@example.com if you have any questions, comments, or
suggestions. If this program is useful to you in some way, I'd love to
hear about it.
I've always uploaded my photographs into an "img" or a "misc" directory,
and handed out URLs like http://toroid.org/ams/misc/foo.jpeg to
people. This has worked pretty well.
Occasionally, however, I would upload a group of related photographs,
and, tiring quickly of cutting and pasting many URLs, end up writing
a little HTML page with links to the images in question.
Eventually, I tired of that too.
Update 2015-10-31: I don't actually use this code any more. It's
been replaced by something that works very similarly, but this version
is no longer maintained.
mod_leech is an Apache 1.3.x module which uses a MySQL database to
enforce per-user download and concurrent-login quotas.