Invention's Mother

I'm so very tired. Apparently coffee is not a long-term substitute for sleep after all. I ran a few successful tests of the distributor last night. The admin interface is coming together, and can manage the handful of configuration records. Messages are coming in and going out.

I nearly made a massive mistake in coding, by almost relying on the system date/time as a synchronization method among my unknown number of parallel instances.

I had made the assumption that within GAE, machine times would be synchronized to within (at least) the NTP ping drift of milliseconds. This is totally not the case. (One GAE server apparently managed to get 40 minutes out of synch, according to a Google engineer. Yeah.)

That's a shame, because properly synchronized clocks are hard to do, and not something we should be doing ourselves. It's slightly amusing that Google considers net access to be more important than knowing the time :-)

Memcache seems to be the preferred synchronization primitive, since it is basically a tiny networked database server with atomic operations. (Don't even consider the datastore for synchronization locks) And the use of named tasks in queues seems to be another pro tip - job names can't be re-used (for a week) after being allocated, so proper naming means idempotence. GAE will say "I can't schedule task 'send email 1029302' because it already did it!" but unnamed tasks will happen as many times as they are scheduled, and this can be more often than you would expect or want.

There is a very specific level of functionality needed in our application, and it wasn't decided on by committee or sales meetings or customer feedback. It's designed by necessity. We need this thing. We need it very, very badly. If we didn't need it so much, I wouldn't be taking time out of the primary project to do this.

There are a lot of bad reasons to develop software, such as chasing fame and fortune. Or even revenge. It rarely works. The best software comes out of need, from necessity.

Now that the prototype is complete, it will be put into service almost immediately. It will feed the other system we have been developing, the one we needed it for, and the two will grow up together. It will get stress tested, and attacked, and patched. That which does not kill it, will make it stronger.

Eventually I'll forget it's even there, just like all good plumbing.

Somewhere along the way, when it's stopped being exciting to use the application (like dancing through a minefield at the moment) it will be ready to unleash on the world, and we'll see how many other people's problems it can solve.

It is by caffeine alone I set my mind in motion.
It is by the beans of java that thoughts acquire speed,
the hand acquire shaking, the shaking is a warning.
It is by caffeine alone I set my mind in motion.