Not bad for an evening’s work, and I now feel closer to the great Isaiah Mustafa, even if he hasn’t replied to me directly in a video.

The work I’ve done so far doesn’t really lend itself to being discussed succinctly, but it’s been pretty satisfying so far – I’ve come up with a particular procedure that was needed, which is quantitative whereas the procedure that it’ll end up replacing was only qualitative. I have a depressingly large list of things to do that I only ever seem to make modest headway on – but again, based on what I’ve seen of other people, this is fairly standard.

One of the most surprising things so far for me are the supervisions that I’m giving. The first few at the start of term did not go as well as I wanted them to: I wasn’t explaining things as lucidly as I knew they could be; I didn’t really feel like a supervisor in the supervisions. Then, one week, I found myself looking forward to it, and then after the fact I realised that I actually enjoyed it. I’m not sure what the difference was – whether I just got more comfortable with a novel situation, or something actually changed, I’m not sure. Either way, it’s excellent money for something that I now quite enjoy. Here’s hoping it continues.

I’d be lying if I said that everything went according to plan this term, but that’s life, really. What isn’t a lie is that, in spite of such things, I’ve still enjoyed myself immensely. I’m enjoying the differences between Clare and Churchill and, mystifyingly, I seem to have found more time to do social things, despite the fact that I’m (ostensibly) working more. I’ve made a relatively satisfying collection of new friends, though notably most of them are only doing M.Phils, which means that come next year, there will be a considerable exodus.

Still, that’s future Alex’s problem. I hear that guy is awesome, though, so I reckon he can probably deal with it.

Five lectures a day got a little tough to pay attention in towards the end of any given day, especially as most people were cruising along on less sleep than usual (or just skipping the odd judiciously chosen lecture entirely to catch up on sleep). Many were interesting, but a couple did drag on a little long; I suspect this was because the lectures frequently went to a full hour, but as undergraduates we’ve been conditioned to pay attention for 50 minutes, and then no more.

But really, it wasn’t about the lectures, it was about meeting new people. Indeed, we were told a couple of times to forget all the physics we’d been told, as it was likely to be irrelevant or otherwise shown to be somehow mistaken in a few years, and we should concentrate on networking. I went in with this intention, but we relatively quickly settled down into a group of about ten, as mentioned above, and modestly began calling ourselves ‘Team Awesome’. Lunchtimes and pre-dinnertimes were spent with a mixture of exploring St Andrews and playing party games, including ‘Godel Eats Currency’, the Cereal Box game and Pictionary.

Much of our evenings were spent inside the Raisin which was – purely coincidentally – the nearest pub to our accommodation. Pool tables, a jukebox and cheap drinks – what was not to like? The answer to this rhetorical question would be every other group in the pub’s taste in music. We rectified this on the final night, where we essentially hijacked the jukebox (and a pool table) purely for our own entertainment. That said, we didn’t spend all our time there. The second evening in St Andrews we went to the beach for what was nominally a bonfire, but should have really been called a campfire to reduce expectations. A bunch of us lay down on some conveniently flat rocks and stared at the sky. As we were all people about to start Astrophysics Ph.Ds, this was a common source of interest – and I was comprehensively schooled by a number of people on the contents of the sky, which was a novel experience.

On the Tuesday evening most people in our group went to see Inglorious Basterds, but I spent some time exploring the nooks and crannies of St Andrews with a couple of other people who weren’t keen on sitting in a dark room not talking to anyone. We trundled around for a few hours, and found quite a few small alleyways which warranted exploring, much to our delight. Equally delightful was an ice cream shop open at eight in the evening with more flavours available than you could possibly imagine. I managed to convince someone that, being in Scotland, they needed to try the Iron Bru flavour, but they regretted this soon after when they couldn’t finish the scoop!

A Ceilidh was held on the Wednesday, which was, as always, awesome fun. They allowed you to hire kilts, but I opted out of such frivolities and opted to bring my dinner jacket instead. As it transpired, I was one of the few guys there in black tie; other than the kilts, there were maybe two other guys there in proper attire – despite the fact that the kilts were present should have indicated otherwise. Still, if you’re overdressed you feel silly, but if you’re underdressed you look silly, so I was erring on the side of caution at least.

I think it’s a little difficult to pin down exactly what made the week such a great experience. The few things that I’ve written above don’t really convey just how much I enjoyed it, but I don’t know how to fill in what’s missing from the descriptions. I think part of it was that we had such a large common interest that conversation was easy despite our relative ignorance of each other. Of course, the fact that we got to make such incredibly nerdy astrophysics jokes that would have fallen flat elsewhere, but were met by guffaws of laughter there must have helped.

Team Awesome is hoping for a fairly compete reunion in a couple of weeks, when two members share the same birthday in Liverpool, which should be good fun one way or another. Beyond that, the word on the street is I actually have to start doing some work in order to claim I’m doing a Ph.D….

The second webcomic is Real Life Comics. The author, Greg Dean, is having people submit scripts each day this week for how to continue the week-long (and non-canon) adventure. Today, he deemed my script the best. I’ve been reading his comic for over six years now (I think), and to see my name at the bottom of the comic is more than a little bizarre. I’m half tempted to buy a print. Of course, I now have even more of an interest in how this storyline turns out!

A long-term goal of mine had been to build an arcade stick for a games console. It took around ten days of near-solid work, but it’s finally done and is a joy to use. It’s already getting a lot of use in Street Fighter IV (and I’ve only received a single piece of abuse on Live so far, which is less than I was expecting), and come Wednesday it’ll be getting a lot of love through the medium of Marvel vs. Capcom 2. The electronics was, ironically, the easy bit, despite the fact I was soldering to an existing PCB and setting up two 25 pin harnesses. It was all done in under three days and worked correctly first time. Cutting six pieces of wood out of a floorboard, drilling a few holes, and screwing them together? Took the remainder of the time. I hate working with wood. I’m very happy with how it turned out, though – certainly at the lowest point I wouldn’t have expected it to turn out this good. It ended up costing around the same as buying a Madcatz Street Fighter IV SE stick, but is at least future-proof – I can build other breakaway project boxes that will let it work on other consoles, if I am struck by the urge.

Much to my delight, there are enough people around without jobs just yet (mostly people who took a gap year and so have only just graduated along with me), so there’s more than enough support for fun times. Notable mentions so far are a black-tie dinner party, sailing on Ben’s boat (where we saw seals), the beach, and going out having dressed up in owl material (oooooooo indeed).

I also trekked into London with some excellent co-conspirators who were game enough to see Derren Brown’s Enigma show. We’re asked to ‘keep the mysteries mysterious, and the surprises surprising’, so in that spirit I won’t delve into too much detail. However, speaking loosely, I think I can safely say that we were all extremely impressed. We managed to deduce how some of the effects were done to our satisfaction, though many still eluded us. I believe I know loosely how the final effect was done (at least, I have a method that I would use if demanded to reproduce it at gunpoint), and I had a particular song stuck in my head for a couple of days after the show. What the most shocking thing about the show was just how impressive Derren putting people into a dissociative state is; there was one girl he kept putting under and pulling back, and she just went completely limp instantly, each time.

I’m conscious of the fact that compared to a lot of people I know, I’m not doing anything too extravagant with my prolonged time off. Phil’s off trekking South America, and I know a few other people who are travelling to similar extents. While doing something along those lines would be fun, I’m not pining for it to any great extent. I’m away for a week towards the end of next month, but it’s primarily Ph.D. related rather than outright fun – though I’m anticipating some of that as well; after all, all work and no play…

For a competition I wrote a program to ‘solve’ any Countdown numbers game (assuming a solution exists). The source is here, though it is very slow (a couple of minutes to exhaustively search a tree). Poking around the internet after I wrote it shows that there are much faster implementations of what is essentially the same algorithm, just with more shortcuts taken and a stricter adherence to the rules (it transpires that intermediate fractions are not allowed in Countdown, so by checking a % b before dividing you can save yourself a lot of time). It took me most of the day, but only because I don’t use Perl on a regular basis; I was initially unfamiliar with how it displays numbers and truncates their representations when asked to display them if there are a sufficient number of 0s after the decimal point. Frustrating, but good to know.

I’ve switched colleges, and while the change is something I wanted, it would have been easier to just stick with Churchill. There’d be fewer unknowns for three months from now, which would be comforting. While objectively I’m sure Clare is going to be great, my gut is unconvinced. Coupled with the fact that a lot of people I’ve become attached to over the last four years are going to be absent, I’m very much heading into the unknown again as I was four years ago. I managed well enough then, though, and fell into company that I’ve treasured since, so I should be able to cope.

This wasn’t intended to be terribly sentimental. I’ve already done that either in person or via email (to my shame) with those that I wanted to. A lot of photos were taken after the end of exams, some of which were rather apt and there are still more to be circulated. It was the first and only post-exams / May Week period that I’ve had without performing with the Fire Troupe at balls, I really enjoyed myself. Without the regimen of sleeping during the day, get up, rehearse, perform, sleep again, I’ve had plenty of time for fun – garden parties, 5-a-side football tournaments, musicals… all the trappings. And what fun it was too. I definitely have no regrets about doing it.

In english: I wondered just how much information you could send in a single tweet, and decided to find out.

Armed with the knowledge from an Information Theory course given by the singular David MacKay, I decided to see. So, it’s common knowledge that Arithmetic Coding is the best way to compress information, getting you to within a couple of bits of the Shannon Limit so long as you have an accurate modelling system. The ‘best’ bit of the compression comes from having suitably subtle and ingenious models of the text that you’re encoding; without such a model, you’re probably not going to see the benefits of Arithmetic Coding. However, I’m always up for a challenge – especially one set by someone with an Erdos Number of two.

Admittedly, I’d attempted to make an arithmetic encoder for a homework, which I almost achieved. However, I cheated by using the arbitrary precision maths module in PHP which would break after a suitably long string. As I presented it to him, I felt a pang of shame even before I explained… so reinvigorated with the fire of project procrastination, I thought I’d try and get one working properly. Admittedly, my first instinct was to just find one written in PHP, to get onto the meat of this project. But there didn’t seem to be one on the internet – until now! And it really does work properly, much to my amazement. Technically, it’s a range encoder, but the two are mathematically identical. As an added bonus, a range encoder isn’t protected by IBM patents, whereas an arithmetic coder is.

So, how do we transmit our bitstream over Twitter? Happily, Twitter supports UTF-8 characters, which can occupy up to four bytes each. For a four byte character, 21 are under control of the user. If we were to use just uppercase characters and space, for English this corresponds to about four bits per character we wish to send – so we could send up to 700 characters in such a case! We like punctuation though, so this means we can send fewer total characters, but it’s probably worth it. And so began my trial with Unicode.

1. First attempt: split the binary string into 21 bit blocks, pad the last one and send the corresponding UTF-8 characters. Unfortunately, not all 21 bit long binary strings map to valid characters – those above 0x10ffff are invalid. So drop down to 20 bits per UTF-8 character, and have the first bit as 0 – we’re now definitely going to map onto a valid unicode character, right?
2. Turns out, no. The points 0xD800 to 0xDFFF are UTF-16 surrogate pairs. So look for these, and if we would be trying to send one of these, send a three byte character instead (which will only be 16 bytes, but still a bargain).
3. Unicode has canonical decompositions – so the character á is canonically identical to the characters representing a´ in sequence, for example. These are defined to be identical, and no application should function differently when presented by one sequence or the other. So now check to see that a character we are trying to send is fully decomposed – if not, drop to a three byte character if we were four, and a two byte character if we were three. I’m not convinced that there’s a PHP library that does a complete job of this anywhere, so this is currently only a ‘probably’ step of the process – if a message fails to send, it’s probably failed here.
4. Line Feed / Carriage Return. Different OSes introduce the other – or not – after seeing one. So if the character we’re trying to send is actually in the bottom 128 of characters, only take seven bits. Set the 32s bit to 1, so we’re not using the awkward part of the ASCII table.

I then set about trying to send some information over twitter. The first test worked well. I decided to push it to the limit, however, and encoded nearly 600 characters as a string of 138 UTF-8 characters. Dropped my clipboard into Tweetie, and was informed that I was over the limit. Perplexed, I did some poking around and found that, in fact, Twitter isn’t sure what they mean when they say 140 characters. They think they probably mean bytes, not characters, but there is evidence to the contrary. No further updates as yet that I can see, but problems seem to be in at least partly due to how Ruby counts characters. Until this is fixed, however, the amount of information able to be sent using a single tweet is going to be limited, somewhat. They’re looking into it.

The upshot is that there is now a proof of concept of the compressor up along with the source. I’ve included the UTF-8 libraries with it which are from a couple of sources, but mostly MediaWiki. I based the range encoder on the pseudocode over at the wiki article, changing it to use binary rather than base 10. It’s entirely for use at your own risk.

Lastly, it would seem that I am not the first to have an idea along these lines, and some have even gone much, much further. Arguably too far. Truly, the internet is a wonderful enabler for people who want to do things simply to see if they can be done. I find it amusing how all of these seem to have sprung up in the last month!

Seeing a commentator speak in this way was novel, which was why it got the attention it did. Stewart was essentially forced into admitting this to defend his position on torture without resorting to double standards which Cliff May would have surely jumped on. Unfortunately, Stewart has now retracted what he said, saying it was stupid. I don’t believe that’s true; it’s a valid opinion that can be defended (as Stewart himself did to some extent in the original interview). From a (relative) outsider’s perspective, it seems like he’s being forced to say it simply because his original statement wasn’t popular. It’s unfortunate that Cliff May doesn’t have a chance to respond, because I think he would have relished the chance to be someone to score a (rare) point against Stewart.

Project deadline is looming, so as a bit of a shortcut for content, a selection of links that I’ve run across over the last couple of weeks and found worth enough of bookmarking. Not sure if I’ll do this consistently (it’s very easy to do), but it’s a relatively quick way to get a bulkier blog post with genuine content.

• Slagcoin Joystick Controller page – I’ve been fighting the urge to build an arcade stick for the Xbox 360 for a little while now; I can’t act on it while I’m at uni, but if the urge is still there when I get home over the summer, this certainly seems like a resource I’d want to tap.
• Textorizer – vectorize a picture using text strings – I don’t have a reason to use this yet, but it’s something that I could quite easily suddenly think of the perfect use for out of the blue.
• YouTube – 40 Inspirational Speeches in 2 Minutes – Sent to me as required watching before exams. Very entertaining.
• Making the World a Better Place, One Evil Mad Scientist at a Time – I’ve only skimmed it so far, but I’m already completely sold on this blog. Seems to link to exactly the sort of stuff that I’d enjoy making. They’re the source of the electric motor above, but the whole blog is solid gold.
• Ninja Ropes Extreme – Simple in its gameplay, yet epic in its scope – Ninja Ropes! I ran across this courtesy of the Dr Horrible Commentary – The Musical!
• Tinyarro.ws – Shortest URLs on Earth – Uses Unicode domains and characters to provide absolutely miniscule URLs. Unfortunately, a lot of Twitter (where it’d see a lot of use) clients don’t yet support it. And then there’s the whole debate about URL shorteners in the first place…
• How to make the simplest electric motor – Definitely making one of these when I get home, if not before.

My original suspicion was that it was due to the fact that iPods use SCSI over USB, but I did some reading. It turns out that many iPod hard drives use block sizes of 2048 bytes, compared to 512 bytes used by almost every other hard drive, everywhere, which the wbfs utility assumed. So a quick ‘find | xargs grep’  and a compile later, I was able to format the iPod. Unfortunately, I still can’t rip from the Wii, so I’ve been using another hard drive as an intermediary. Because the hard drive in the iPod is relatively slow, loading times are about the same as off the DVD – but no grinding. Whenever I hear it, I get flashbacks to my Dreamcast and PSO v2, with the ever-increasing loading times.

My Part III project is dragging itself along. It’s just managed to stagnate over Easter, despite my efforts to the contrary. I really need to get it rolling again. The computational element taking so long to do each run doesn’t really gel well with my working style, which is to sit down and just do the damned thing, but where I have six hour long enforced gaps, it’s tricky to find my way back once I’ve set it going. I have a reasonable draft for the write-up, up to the point where results are necessary, but after that it’s really just an expanse of blank. I’m back to university on Sunday, which will perhaps spur me on to action if I’ve not already managed to get back on the horse. Having to revise minor options for an exam later this month has hardly helped, either.

As part of my writeup procrastination, I found some interesting articles about common errors in scientific write-ups. I admit to committing some of them in the past, but I’m striving to eliminate them now.

• My week five blues seem to have arrived exactly a week late. This is unsurprising, as lectures didn’t start for me until week two, so they could be considered right on schedule. Unfortunately, everyone else has already conquered them, so I’m wallowing alone, somewhat.
• My Khan Machine seems to still be going strong. Two weeks after it was in the B3ta newsletter, it now has over 3500 graphs drawn, and still seems to be drawing around 50 visitors a day, after getting around 5000 hits in the first 24 hours. Pleasingly, people thought it worth submitting to Digg and Reddit, but it didn’t hit the front page on either, thankfully – I am only on free hosting, after all, and I don’t think their servers would have appreciated it!
• Astrometry.net is absolutely staggering, especially when you see it go to work on your own crude contributions.
• Having signed up for Twitter quite a while ago, I never used it. I’ve been forcing myself to (slowly) since around Christmas, and while my own contributions are comparatively infrequent, I enjoy those from the people I’m following. These are mostly celebrities of one form or another that I’m interested in for whatever reason. It’s bite-sized celebrity gossip, news and banter. I think Twitteriffic is going to be hanging around on my Mac for a while.