Saturday, May 03, 2008

I don't know... yet!

In movies, computers know everything. Just ask the computer and you will get an accurate answer now. For us, developers who have to deliver the promises made by Hollywood, programming software that gives correct answers immediately is very easy. Unfortunately it is also very costly.

Giving the correct answer generally translates into querying the one source of truth of an application: the database. This is easy.

Sometimes the database is far away from the access layer the caller interacts with, whether it is a web tier or a service tier. No problem! It is very easy to ask the caller to hold his breath until a long chain of synchronous calls gather all the correct data needed and finally present it as a glorious reply. Usually the caller has enough breath to wait until it has to get some fresh air again (some call this a time out).

All this is fairly easy but unfortunately very costly. Contention to access the centralized source of blessed truth increases as more and more callers want to access it. If, for the sins of the application, it gets successful, the number of callers holding their breath while waiting will increase dramatically.

It is a time when the application would love to have callers with smaller lungs, so they could time out faster and free the threads they are holding while sitting idle. But they are not: the dozens of second they wait are an eternity for a server. Instead of being glued waiting for a remote service or database to deliver the ultimate answer, the application would like to afford replying:
I don't know... yet!

At least, this imprecise but immediate answer would allow the server to release threads almost as fast as they come. This would give the application the luxury of replying later, whenever possible...

Of course, such a paradigm shift is all but transparent for the caller: he must learn to deal with imprecision. He must embrace asynchronism. He must survive eventual consistency.

We now have the tools, from the web tier to the back end. Can we succeed in this paradigm shift?

I don't know... yet!