@madpilot makes

There comes a moment in every career where you realise that there is a whole world outside of what you do. Sure, you don’t have to have three PhDs to figure out the world of macrame is significantly different to Ruby on Rails hacking, but when was the last time you thought about process in, say banking software development? Or had a look at what is going on in the world of Operating System code? To an outsider, these are related industries – they both involve sitting in front a grey box for hours on end banging gibberish on to keys with embossed letters that are out of order, but to us, they are worlds apart.

A couple of days ago, something reminded me that I, and I assume a great number of other web developers have forgotten our brethren that are more at home with linked-lists than unordered-lists, and as a result, we are significantly reducing our ability to find the right tool for the right job.

This revelation was borne from the recent Twitter downtime. As always happens, hundreds of well-meaning developers tried to offer up solutions to fix Twitters yo-yo like uptime graph – there was the usual Rails bashing, some bewilderment over MySQL replication, and a general consensus that is shouldn’t be this hard, but one article really stood out: Scaling a Microblogging Service. Go and have a read if you want to know why scaling Twitter is REALLY HARD (maybe impossible),

For those of you who don’t mind a spoiler, the crux is that Twitter is designed on a platform more suited to content management, when really it is a messaging system.

Now, you go to any web developer on the planet and ask them to build you a Twitter-clone, and I bet you each design would be pretty similar. You would have a table for users, a table to hold friend references, and a table for messages, which would all be linked via some sort of foreign key relationship. The reason being is that for 90% of what we work on on a day to day basis, this makes the most sense. Generally, you have few authors, many readers and those readers all get basically the same information.

Twitter is is completely different. Everyone is a producer and everyone is a consumer. Every second there are hundreds if not THOUSANDS of writes from people leaving their 140 characters worth of thoughts. To make matters worse, every user get a different view, so every page load would be absolutely thrashing the Twitter database servers.

Twitter is a MESSAGING SYSTEM – not a CMS. It closer to a mailing list then a blog. So why did the developers not make this observation straight away? Well, as web developers, this is foreign territory. If Nokia decided that they were going to build Twitter, I bet things would look very different. Twitter and SMS are pretty closely related, and for someone dealing with short messages all day, that may have been the natural path for them to take.

You might counter this argument by saying that we are web developers – of course we think in terms of web. We do what we know, just like everyone else. Correct. But maybe we should stop and think a little before we dive in and start coding up prototypes using our favourite RAD tools. Maybe we should try to look for metaphors in other areas before taking our hammer to unsuspecting screws, bolts and watermelons.

Maybe we should take some lessons from the User Interface guys? They use metaphors all the time when they design. Without somebody realising that people like putting paper into manila folders, we would still be kicking around text terminals. Lucky for us, our metaphors don’t need to be as abstract as that. We only need to look at outside our little bubbles to see what developers from other industries are doing and we may too see a obvious solution that we would have otherwise missed.