@madpilot makes

Yep, Software Engineering is dead

You know that feeling you get when something you’ve been taught to believe in gets discredited and because your belief was tenuous at best, the walls come tumbling down around you and then you have a huge weight lifted off you shoulders?

Pascal just posted this on the 220 mailing list. Amen. It’s something that I’m pretty sure I’ve been articulating for a long time. Whenever someone has asked me why software is hard, I always use this analogy:

If you ask a Civil Engineer to build you a bridge, it is easy to spec out. You know how far the bridge has to span, what sort of foundations you need, and as a result you can make a recommendation about what sort of bridge you need. The Engineer can build you a little model – you can look at the model and say “Yes! That is a bridge. That will do nicely”. They can mathematically model the bridge to make sure this doesn’t happen. They build the bridge and if it allows things to cross from one bank to the other, you have a success.

Unless you are building “Hello World”, a Software Engineer’s life isn’t so simple. You have different platforms, users, stakeholders, contexts – it gets exponentially harder with every feature that gets added. I once did a unit at Uni called Formal Methods which tried to mathematically model software. It was stupid. The code we modelled was like, nine lines long, and required a 32 page proof (I didn’t even get close). Stupid.

Of course, academics have been trying to shoehorn software into engineering for ever. In first year, they taught us UML which I guess is similar to architectural drawings or flow diagrams or something. I’m sure UML works really well when working with the waterfall model of software design, which has strong ties to old school, proper engineering. I couldn’t imagine having to go and update hundreds of UML documents every time a minor change was required. We are also taught in first year, that the waterfall model is pants in the real world, which by association makes UML nothing more than a nice thought experiment. (I’m still bemused by the number of Software firms that put it as a requirement for graduate Software Engineers – basically because coming up with job descriptions for inexperienced programmers is really hard).

Sure you can argue that testing is a software technique that we (should) use, but this is the exception to the rule. I guess the conclusion we need to come to is that Software isn’t an engineering problem – it’s a people problem. (Some may say, it’s a creative problem – that’s also true, but buy me a beer and I’ll explain that traditional engineering is too, so the argument doesn’t further my point). This in itself is a problem, as (gross generalisation ahead) boffins who like coding, tend not to deal with real people very well.

Further discussion on our internal list suggested that creating software products is the way to go. I think I want to agree with this – there are many examples of off-the-shelf products that are extremely popular: Microsoft Office, Adobe Photoshop etc. In these situations, the customer works with in the workflow of the software, and that seems to work. So do we as developers need to convince our clients that the feature they want may not be needed? Do our clients actually know what they need? Of course this view is not with out it’s flaws either – users will generally be working against the software, rather than with it. Is working with a sub-optimal solution better than battling with requirements and budget overruns?

I can’t help to think that there is something we are missing. It would seem there is a disconnect between what our clients want and what we can provide. If you look at the classic project triangle, your client wants to minimise price and time, and maximise good (I hope my English teacher isn’t reading this), where as we want to maximise all three. So the crucial “pick two” part flies out the window. Either we start sacrificing the good, re-negotiate the price, or try to stretch out the project to restore the balance – none of which makes for happy clients.

Well how about adding fat to the quote? In theory, this is fine – if a client sees value in an “inflated” (but more likey a realistic) price then everyone is happy right? Well, not really – software development is much like homework assignments: You start out with plenty of time, and the best intentions, and then end up pulling an all-nighter to get it finished – and you still only get a C at best. I suspect this is because it’s impossible to lock down requirements of an abstract problem. This isn’t only because of the difficulty in describing what we don’t understand, but because we don’t even know what half of the problems are going to be.

And this is our quandry – how can we estimate unknowns? Not just “we haven’t seen this before but it looks like X” unknowns, but “What the hell? How is that even possible?!” unknowns. Other areas of Engineering encounter these problems occasionally – we get them all the time. So, the solution (he says as if there is one) is to minimise the risks and/or consequences of these unknowns. Jobs that deal with people do this all the time. If you work in marketing, you can postulate all you like – you can’t be sure how a campaign will work until it does. Marketing is reactive.

When you make a change you can’t be sure what will happen. Sure, you can put an ad in the Yellow Pages year after year, because it has brought in on average Y leads per year – but there is no guarantee this year will be the same. It seems that the humanity-based sciences are happy with this, but quantitative-loving geeks don’t like that. Hell, binary is black and white, not Gray.

So, perhaps the key is to treat software as a living breathing thing. Agile programming and iterative development can help, but they are means to an end – they don’t work with out communication and understanding between people. We need to break down the barriers between provider and client – the question is: Is that even possible?

6 comments

  1. Pingback: the island of eleusis » Blog Archive » Software engineering is.. dead?
  2. Excellent post. Couldn't agree more - the engineering and manufacturing models don't, typically, work for software. Unfortunately, our customers are usually versed in engineering or manufacturing metaphors and cannot understand why software projects are such nebulous beasts to manage; both from a design and time-line point of view.
  3. Jack W. Reeves argues that programming is a design activity, not a manufacturing activity:



    http://www.developerdotstar.com/mag/articles/PDF/DevDotStar_Reeves_CodeAsDesign.pdf
  4. The Agile mentality is to work with sub-optimal solution and make incremental inprovements.



    Formal Methods isn't stupid, it's hard. I didn't get it either.



    In Oil and Gas engineering, it takes a few pages of maths to prove the requirements/design documents. In software engineering it'ld take too long.



    Whether software engineering is dead or not, the techniques are proven useful. Logic, problem solving, documentation are all engineering methods we use on a daily basis.
  5. hmmm...that idea of software craftsmanship is new and fits in well with what we do. But craftsmanship has a foundation in engineering. E.g. Ferrari.
  6. Hey Myles. Interesting read. My take is that it is not dead, it's needed more than ever. I've seen enormous benefits in the application of engineering principles to produce software and I've seen tragic results when not applied. The greater the complexity, risk, scope, budget, generally speaking the greater the overhead through the application of engineering principles but the greater the control. You work with the given constraints (including appetite for risk) and apply an engineering process that is fit for purpose. There's countless examples of (non software) engineering that's creative - flying to the moon is one that springs to mind. There's also many endeavours that are repeatable processes and engineering can be used to optimise them - some software development falls into this category. Pick the process to suit. I get the feeling that there is a association between engineering and the waterfall SDLC and that iterative and agile methods are seen as non-engineering. I don't agree with that - I think they are excellent examples of the right process for a particular application and embody sound engineering principles. Your assertion that we should "treat software as a living breathing thing" shows that your understanding of some typical contraints - that the user requirements will change over time, that the deployment environment will change, and you recognise the need to design a solution and a process to deliver that solution that considers those constraints. Good to see all that engineering didn't go to waste on you mate.

    :PM

Leave a comment