Saturday, October 31, 2009

Software Manifestos: A Matter Of Trust?

As software manifestos have started to proliferate these past months, I have started to wonder what could be the root cause for their creation. Why would thought leaders gather, assert a small set of values and shrink-wrap them as a manifesto, calling for others to sign it? My feeling is that these manifestos are the expression of a pushback on a particular aspect of software development that went insane.

Here is a little game: match the manifestos with the software insanities they push back on:

Big methodology and design up-front
Software craftsmanship manifesto
Army of flying monkeys testing

Agile manifesto
Snake-oil vendors and ivory tower architecture
QA manifesto
Reckless programmers and incompetent coders
SOA manifesto

(One manifesto I see missing here is the "recruiter manifesto", which should push back on inane keyword-driven head hunting schemes solely able to put the wrong people at the wrong spots)

If we dig deeper, we become tempted to ask why is our industry suffering from such insanities? What does make software different? Could it be because of complexity?

Complexity. Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.
Frederick P. Brooks, Jr., No Silver Bullet

The natural reaction to complexity is to try to escape it at all cost, even if it means wilfully practising self-deception. Hence silver bullets, hence snake oil vendors, hence all these methodologies, governance committees and ivory towers that are there to nurse the insecurity of higher levels of management by giving them the impression software creation is under control and, finally, out of the hands of programmers.

Of course, it doesn't work that way: years and millions of dollars later, reality comes knocking at the door, manifestos are getting written and everyone is sent back to the same fundamental question they've been trying so hard to avoid: how to build trust in software developers?

And that's of course a question for us, software developers. How can we build such a trust in us when so many forces are pushing towards the opposite?

Granted that software development is unpredictably complex and that this complexity reveals itself when the devil shows up (those pesky details), it is clear that the overall battle of trust is fought during each decision, when tackling each detail and writing each line of code.

I think we could learn a few lessons from the world of aviation, where trust in pilots has been built progressively and methodically. When you fly an airplane, you have plenty of decisions to make and losing any of these battles can end up very badly for everyone. So why are pilots trusted? Aren't they fully superseded by ATC anyway? Answer is no: even if ATC has authority, the PIC (Pilot In Command) has the last word because he is the one out there dealing with the ultimate reality of flight. Despite its authority, ATC doesn't micro manage the pilot: the pilot is in-command.

To have the privilege to be a PIC, you have to remain current and regularly prove that you can be trusted for your judgement based on your skills, experience and training.

If the acronyms didn't sound so bad, I would dare suggesting programmers should become DICs, ie Developers In Command. Though working under different forms of authority, DICs would be fully trusted for taking the final decisions in the daily battle of writing code. In this world, it wouldn't be an heresy to say that developers could build large and complex software systems from the ground up, without the need for snake oil, committees or big design.

When trust will be manifested, we won't need manifestos anymore.


Michael Vax said...

Thanks for the post David.
Having business trusting s/w developers will be really good indeed :-). And you are right by identifying complexity as the main cause of development issues. Most business leaders don't understand the complexity that developers need to deal with. To make matter worse this is a log term complexity that has ramifications well beyond initial release date.
In pilot example, it is very easy for business to assess success or failure of the flight. If pilot managed to get safely from point A to point B, he can be trusted to do it again.
In s/w world it is possible to successfully deliver first version of a product by doing crazy hours and no testing. It can and in most cases is announced as a success by management. We s/w professionals know that business will pay for that later but many enterprises are driven by short term goals.
As business managers don't see or understand complexity of s/w system it is difficult to demonstrate to them monetary value of good design and s/w practices.
So far, business models of s/w development are not well understood. I hope that sometime in future lean s/w development approach will be taught in MBA programs. S/W is such important part of modern economy and only handful of companies do it well.

David Dossot said...

Michael, thanks a lot for your insightful comments.

Indeed software is here to stay and one can only hope that our - still young - industry will evolve and realize what matters and what doesn't.

Pierre said...

Hi David,

Interesting position.
I take notice about 2 things:
- the PIC
- sw developers recognition by the business

Concerning your PIC example, my point of view is that developers wouldn't be the pilot but, in fact, they are the aircraft engineers, Dynamics Engineers, Flight regulators, nothing else...And a good PIC cannot flight safely without these guys.

Point 2, recognition by the business? trust between between Biz and Dev.? How?
Business has Macrovision, SW need to have Microvision.
I don't understand why these two parts of the same puzzle are searching to be the best one: non sense.
If you want trust: be trustful.
If you want to be a good Executive, don't be a sw developer: give True North and manage the strategy.

According, at last to your reflexion about Manifestos, I agree that too much manifesto is just noise in a coalition research.

For all the rest of the discussion, I agree with you: Scrum is a Way


Kind regards from the little Gran Duchy


David Dossot said...

Pierre, thanks for chiming in and sharing your thoughts.

The reconciliation of the macro and micro aspects of software development can appear daunting if it is tried to be done by committees.

I believe that any competently developed piece of software can fit the big picture without the need for this big picture to be pushed down the throats of developers.

As you suggest: armed with enough knowledge of the global strategy (the True North in your terms), competent developers should be trusted with establishing the tactics.