My latest posts can be found here:
Previous blog posts:
Additionally, some earlier writings:
2016/10/06 - Surprisingly Quick ...
In the 1990s I had a job at Liverpool University doing research into how we might make it possible for non-computing specialists to use parallel computers. Even today, over 20 years later, this is still an unsolved problem, and the machines now are designed to be easier to use. The machine I was using was a Parsys SuperNode with 96 T800 transputers, hooked together with a reconfigurable switch, cunningly designed so that any 4-regular network could be realised.
For the time it was quite an impressive piece of kit.
However, we wanted non-specialists to be able to use it, so it was decided that at a minimum we would need to have the NAG serial Fortran library available. We could worry about the idea of parallelising it later, but if the NAG library wasn't available at all, most serious numerical computing users wouldn't even consider the idea of looking at using it.
These days this is easy, but back then the machines I was using didn't have a topological sort, so I wrote the simplest thing I could think of in AWK, instrumented it, set it running, and went away to make a coffee. The idea was that when I got back I could see how far it had got, and then decide whether to leave it running, put it on a faster machine, or write something cleverer.
I got back. It had finished.
In fact, it had probably finished even before I had got out the door. By not bothering to take the time to write something clever I had saved myself huge amounts of time - the first thing I'd thought of was fast enough.
So what is the lesson to take away from this? Is it that we should do experiments first to make sure we really understand the situation? That's certainly one lesson, but we can learn that from pretty much anything, and I had learned it decades earlier. It's a simple rule of thumb, never "optimise" before you measure.
So perhaps the lesson is that machines are faster than we think? Well, sort of, but not really. Machines certainly are fast (by some measure) but the machine I'd been using wasn't that special.
Perhaps the lesson is that a topological sort isn't actually that hard to write? Well, a good and fast topological sort is hard to write, but more likely the data I had wasn't sufficiently complex.
No, the lesson I learned that day was this:
More, I would claim that even if they aren't exactly the perfect tool, they are a good place to start, a good tool to use to write that quick, first version, instrument it, and then let it run for a bit while you idly wonder whether it will be good enough.
Odds are, it's probably close.
I've decided no longer to include comments directly via the Disqus (or any other) system. Instead, I'd be more than delighted to get emails from people who wish to make comments or engage in discussion. Comments will then be integrated into the page as and when they are appropriate.
If the number of emails/comments gets too large to handle then I might return to a semi-automated system. That's looking increasingly unlikely.
Links on this page