Bloggy Hell

I recently read about the deployment model that Flickr uses. After picking my self up from the floor, it made me think about my own deployment methods, and I have started implementing a new system, which I though I would share.

First off all, let me describe the types of projects that I have to deploy:

1. Internal MadPilot or Personal projects - i.e. stuff hosted on the server at my house (which also doubles as my development server)
2. External projects that I can take a local copy of to work on
3. External projects that I am forced to work on “off-site” i.e on the clients server.

The deployment system I will describe here covers project type 1 and 2. Not much I can really do with 3, as I have no control over other people’s servers…

Most of the projects I work on, I’m the only developer, but the system described is designed to scale to multiple developers.

Step 1: Set up a subversion repository

I have my subversion repository structured as such: [svn_root]/MadPilot/Clients/[client_name]/[job_name]/

Because I do a lot of outsourced work, I use the job_name to denote different jobs for the same client. If the client is a one off (i.e. the client isn’t another web company) I will usually leave this off. The website code will be stored in the Website directory. This allows me to store documentation and other bits and bobs outside the development tree.

Step 2: Checkout the empty tree

Next I create development and test directories with in my webserver: usually of the form [web_root]/clients/[client_name]/[job_name]/dev/[developer_name] and /clients/[client_name]/[job_name]/test.

I then check out the empty subversion tree to all of the directories so that they become under source control.

Step 3: Setup the webserver

At this point I will set up a virtual host for the client of the form [developer].dev.[job_name].[client_name].clients.madpilot.com.au and test.[job_name].[client_name].clients.madpilot.com.au (Yes, this does end up with stupidly long URLS, but I prefer them to be descriptive. No one sees them other than the client anyway…)

Step 4: Import the initial tree

It is at this point I download the existing site, or start a fresh if it is a from-scratch job. I then add all the files to the repository.

Step 5: Setup any databases

I try to make sure that every developer has a separate database, so does the test system. Usually named [client_name_[job_name]_[development_name]

Using the system

When you start working on a project, you make sure you login to your working directory on the server (usually via SSH) and do a subversion check out. When you finish, you check your work back in. When the system is at the point of testing by the client, you check everything out to the test site.

Using subversion means that you have a versioned backup of all the code, and you can manage multiple developers and a test system at the same time.

Going to production

This can be the tricky bit. For projects on my server, I usually just do an export into the production directory. If the system was tested well enough, if should just work. Gotchas include paths to local files and permissions, but this should be documented anyway.

On external systems, this can be a little more difficult. At the moment, I export the code into a temporary directory, and manually FTP the files up. I hope to be able to automate this in some way, even if it will just reduce me typing.

Next thing to do is to write a nice web front end to subversion so I can do “one-click” deployment… Ahhh

So that is my plan - what does everyone else do?

Just did my honours thesis seminar. Was pretty under-prepared, but I think I swung it. Now, I just need to hand in my poster (It’s at the printers) and all that is left of my university career is one exam. Woot!

Six years of toil has culminated to one anti-climatic week. Still a relief though - psychologically, it has been a huge weight lifted, although things that I’ve said I would do “after my thesis” have just moved up a notch on my todo list.

I really want to get some work done on Taste.net.au soon (Hurry up Ruby books!), but I’m enjoying the freelancing at the moment (That’s right kids, I’m back on the market!)

And I’m looking for to my day off on Wednesday, because my beautiful girl, Giovanna is back from Sydney (Longest week ever!) and we are going spend the day together.

Things are lookin’ up.

I quick entry today, as I really should be working.

I came across a PHP MVC (Model/View/Controller) framework for PHP 5 - You can think of it as the Rails bit of Ruby on Rails. It is this sort of stuff that will continue to push PHP into the spotlight and allow it to compete with the big boys…

It is called Agavi PHP MVC Framework

Web 2.0 is about data exchange and classification. Taking a high-level look at software design process one of areas that is good in theory, that fails in practice is data exchange and classification.

Think about the last software project you worked on - would a new programmer be able to pick up the documentation and work out what is going on? Could they be able to find the documentation? Was there documentation at all?

Documentation is all about data exchange - I bet if you went through your email or project mailing lists you could piece together a decent amount of usable documentation (Implementation decisions, solutions, bug reports etc) - Can Web 2.0 provide the glue to ease the burden of documentation?

Imagine being able to tag bugs and design decisions so searching for a bug returns a link that points to the design documents and comments entered at check-in. This gives the developer background information quickly - and these docs are living and more easily maintainable. As a developer, you don’t have to change to documentation mode - it is already part of your work flow (Other than remembering to enter check-in comments - but you already do that don’t you?)

We don’t necessarly have to limit this to documentation either. Project managers could use it to gauge where the project is at, clients could actually report bugs with out having to learn an obscure bug reporting interface or software terminology. How about timesheets? Add an entry stating you started work at a particular time, and then enter another entry saying you finish at a particular time - tag it wit the project name, maybe what part of the project you were working on - Voila! Or meeting minutes - these often have action items interleaved - tag them and make them searchable. Tag them as completed when you are done.

But I think the best bit is that it wouldn’t require anything more than an email client or a blog-like web interface. EVERYTHING IN ONE PLACE! I know I personally hate having to log in to a different apps for bugs, and meeting minutes etc. We should be letting the software organise our data, and leveraging search.

I think I will be revisiting this one soon…

It is important to step back sometimes, and to look at what is going on. It is all too easy to become completely immersed in things in what is going on in your own life, that you forget how lucky you really are.

Today, I went to the launch of the Breast Surgery Gallery - a system that was developed by a uni mate of mine, Patrick MacQuillan as his honours project a couple of years ago. My honours supervisor (David Glance) was Patrick’s supervisor, and he asked me to do a quick re-design for their website before the launch today, which is why I was invited.

It was quite strange - as we walked into the offices, we noticed their was only two other guys there - the rest were woman. Now nothing was explicitly said, but it is pretty safe to say that a majority of the woman there (If not all) had been affected by breast cancer at some point in their lives.

As I was standing there, listening to the presentation, it hit me that these woman have had to deal with something that I could not even begin to fathom. Yet, they were here smiling - genuinely happy to see each other and to witness the official launch of a product that is actually helping people.

David, who has worked in large software companies for many years said something very profound (Maybe more so when you think about it) - he said: it was good to see a piece of software that is helping people.

We have all probably thought that a system we were writing was going to help someone, but not in the same way as this one did. The stuff that we write day-to-day will probably make someones work day more efficient, or allow them to shop more convienently - ironically the sort of stuff that in the whole scheme of things doesn’t really matter. This software was playing a roll in easing the pychological pain of life changing and life saving surgery, helping the suffers on the road to recovery.

It really isn’t often that software does that…