Sunday, May 10, 2009

Organic Distributed Systems

Migrating monolithic systems to distributed ones is probably one of the most exhilarating tasks in software development.

Monolithic systems, even if they engage in interconnected relationships, remain pretty much like silos (I like compare a network of monolithic systems to silos connected by monkey bridges). Reflecting on the work I have been doing in this domain for the past years, I came to realize how much an IT landscape of distributed systems ends up resembling a living organism.

Indeed, some proprieties of the organic world emerge in a system that evolves towards distribution.

Distributed systems are more resilient. Local issues in living organisms tend to remain local instead of endangering the whole system. This is achieved via redundancy, heterogeneity and a limited coupling between each part of an organism. Interestingly, the same applies to distributed software systems: if properly decoupled (interface-wise and time-wise), a particular system can be in pain without taking down the whole operation.

Distributed systems are harder to diagnose. Rare or complex diseases are uneasy to diagnose and often require many analysis to be performed. Distributed systems present the same challenge, complicating forensics tasks when something went haywire. Using tracing mechanisms, like correlation IDs, can simplify such diagnostics, the same way DNA-tracing can help figuring out the spread of a particular gene (or virus!).

Distributed systems can self-heal. Living organisms embark all sorts of self-healing mechanisms, from localized ones (cicatrization) to global ones (fever). Because each member of a distributed system focus on specific tasks and has reduced coupling to the rest, it has more freedom to perform recovery operations in isolation without needing the involvement of any other member. This said, it will still need an escalation mechanism in case the issue can not be treated locally.

There are surely other qualities distributed systems exhibit that make them look like living things. Do you think of any?

2 comments:

zepag said...

One of them being the vulnerability to pandemia ;). Though with 3 or 4 major hosters of distributed services in the world, it looks more like a colony of huge sponges than like a complete ecosystem!

Nevertheless (because of my genetics background), I've always looked at I.T. systems through this kind of prism.

In fact, the most similar thing between Distributed I.T. systems and ecosystems, is the illusion of control! As with ecosystems, we can react to changes, adapt to them, influence them (badly mostly :p) but not have a full grasp and control.

Once one has reached the point when he accepts this lack of control, he's ready to build things that react better because they don't always presume about what's going to happen. They adapt.

David Dossot said...

Yes, "illusion of control" versus self-adaptation is also a good analogy. I can hear governance sissies cringe in the distance.