Sure enough, the fact that software industry is crippled by technologically-challenged project managers is mind-boggling. But I have been recently flabbergasted by the shameless disclaimer of such a manager who, in the middle of a crucial meeting directly concerning her project, announced: "Sorry, I am not technical". Hence all this discussion was sheer geeky mumbo-jumbo to her and she could not grasp that her project was going astray.
This is sad. Not even funny. Just plain sad. Direct management of software developers can not be that clueless.
But I have prepared my childish revenge. Next time a manager will ask for estimates, I will pretend not to understand and will feign the following excuse: "Sorry, I am not managerial".
Friday, June 27, 2008
Saturday, June 21, 2008
A Zephyr is blowing
My attention has been recently drawn to a new tool named Zephyr, which dubs itself as "The Next Generation Test Management Tool".
My very first impression is that the Zephyr team has done a great job putting on-line a complete workable demonstration environment. This is great to quickly delve into what the software is really about and getting assured that "it works on my machine", which is crucial nowadays as the corporate IT landscape is much more diverse than the traditional Windows + Internet Explorer desktops environment it used to be.
As far as platforms are concerned, Zephyr seems to run "on standard Windows desktop" (sic). I reckon they meant "Windows server", though I have seen production systems on desktops! Hence, I could not test the ease of install nor investigate the technologies used. Neither could I estimate its capacities to scale or work across a WAN.
The second impression is that the graphical interface is compelling, if not mind blowing. These guys did a great job of making arid forms filling an almost bearable task. Indeed the tool is brainy enough to avoid manual data copying and is able to pre-fill or filter data according to the context of the task.
Zephyr is also smartly aware of agile principles, as the notion of Scrum's sprint is hard wired into its dashboards. And dashboards is where Zephyr really shines. I love dashboards of all sorts, even complicated ones. But Zephyr's are truly awesome:
The dashboards and workspaces are tailored to the user profile, which makes navigation easier because you do not have to filter out a lot of non-relevant features. The tool does a great job at integrating and aggregating all sorts of QA related data, including the ones coming from defect tracking systems.
And I think this is where one of the challenges Zephyr will face resides. It currently connects to Bugzilla only, but there are many others out there. Moreover, companies have developed an habit of using their defect tracking system as a management tool for QA. How is Zephyr going to convert these users to this new platform? How disruptive for the practice would it be to move from the use, say, of JIRA to using Zephyr?
The thing that really truly bugs me about this tool is that it does not go further than the traditional "QA monkey" work, in which a human beings are presented a list of actions to perform and report the results thereof. That a tool with an agile penchant does not incite people to evolve their QA practices to more automation is flabbergasting. Where is the Selenium integration? Where is the FIT connector? Though manual QA will never be fully replaced, at least supporting a blend of automated and non-automated tasks would be a great first step. What the software industry really needs is lazy QA teams who mainly use their brains to work on automating their tasks!
To finish on a positive note, Zephyr takes integration seriously. It exposes JSON and REST APIs, the assurance that if you opt for this tool, you will not end up with yet another instant legacy application. This is something I would like to see more in so-called enterprise grade applications!
If you are looking for ways to improve your QA management, I can only recommend that you get Zephyr now, as the free 3 users licenses will allow you to see what this "Next Generation Test Management Tool" can do for you.
UPDATE 31-MAY-2009: Zephyr version 2.5 now comes with test automation features that include the ZBot technology.

ZBot allows you to execute testing scripts on remote machines and aggregate all results back in Zephyr. This is a great move, which addresses my concerns about the lack of support for automation in the previous releases.
My very first impression is that the Zephyr team has done a great job putting on-line a complete workable demonstration environment. This is great to quickly delve into what the software is really about and getting assured that "it works on my machine", which is crucial nowadays as the corporate IT landscape is much more diverse than the traditional Windows + Internet Explorer desktops environment it used to be.As far as platforms are concerned, Zephyr seems to run "on standard Windows desktop" (sic). I reckon they meant "Windows server", though I have seen production systems on desktops! Hence, I could not test the ease of install nor investigate the technologies used. Neither could I estimate its capacities to scale or work across a WAN.
The second impression is that the graphical interface is compelling, if not mind blowing. These guys did a great job of making arid forms filling an almost bearable task. Indeed the tool is brainy enough to avoid manual data copying and is able to pre-fill or filter data according to the context of the task.
Zephyr is also smartly aware of agile principles, as the notion of Scrum's sprint is hard wired into its dashboards. And dashboards is where Zephyr really shines. I love dashboards of all sorts, even complicated ones. But Zephyr's are truly awesome:
The dashboards and workspaces are tailored to the user profile, which makes navigation easier because you do not have to filter out a lot of non-relevant features. The tool does a great job at integrating and aggregating all sorts of QA related data, including the ones coming from defect tracking systems.And I think this is where one of the challenges Zephyr will face resides. It currently connects to Bugzilla only, but there are many others out there. Moreover, companies have developed an habit of using their defect tracking system as a management tool for QA. How is Zephyr going to convert these users to this new platform? How disruptive for the practice would it be to move from the use, say, of JIRA to using Zephyr?
The thing that really truly bugs me about this tool is that it does not go further than the traditional "QA monkey" work, in which a human beings are presented a list of actions to perform and report the results thereof. That a tool with an agile penchant does not incite people to evolve their QA practices to more automation is flabbergasting. Where is the Selenium integration? Where is the FIT connector? Though manual QA will never be fully replaced, at least supporting a blend of automated and non-automated tasks would be a great first step. What the software industry really needs is lazy QA teams who mainly use their brains to work on automating their tasks!
To finish on a positive note, Zephyr takes integration seriously. It exposes JSON and REST APIs, the assurance that if you opt for this tool, you will not end up with yet another instant legacy application. This is something I would like to see more in so-called enterprise grade applications!
If you are looking for ways to improve your QA management, I can only recommend that you get Zephyr now, as the free 3 users licenses will allow you to see what this "Next Generation Test Management Tool" can do for you.
UPDATE 31-MAY-2009: Zephyr version 2.5 now comes with test automation features that include the ZBot technology.

ZBot allows you to execute testing scripts on remote machines and aggregate all results back in Zephyr. This is a great move, which addresses my concerns about the lack of support for automation in the previous releases.
Sunday, June 15, 2008
Best. Game. Ever.
Ok, so I am an old timer and there are plenty of great games nowadays, so it is probably no the best game ever.
But Carrier Command is a true jewel of playability combined with a perfect mix of strategy and action. This is rare. Moreover, it the only game I have played 11 hours uninterrupted: it was 20 years ago and I still remember this all-night session!
Labels:
Fun
Monday, June 09, 2008
Article: Building Content Oriented Integration Solutions With Mule and JCR
The Server Side has just published my article presenting all the features of the JCR Transport for Mule in action!
It is called: Building Content Oriented Integration Solutions With Mule and JCR.
It is called: Building Content Oriented Integration Solutions With Mule and JCR.
Friday, June 06, 2008
Just Read: Dreaming in code

Dreaming in Code is a scary book, the kind of book that makes you wonder if it is really wise to keep pursuing the vain ambition of writing software. By telling the storing of Chandler, an open source project aimed at revolutionize Personal Information Management tools, the author takes us deep in the moving sands of software development.
The most daunting aspect of the book is the following: if the best developers in the world gathered together under the supervision of a level 5 leader (Mitch Kapor) struggle to build software like the rest of us mortals do our pesky daily jobs, then is there any hope?
Maybe hope is in the unintentional software we build while intending to build something else? We got Ruby on Rails, Blogger and Flickr that way, after all.
Labels:
Readings
Saturday, May 24, 2008
Heron, Gentil Heron
This is a follow-up for my last rant about how an unhappy Java developer I was on Leopard. I just had my first week of work with Hardy Heron running on my MacBook Pro and I am so glad I went through the few hours of setup and configuration: this bird really flies!
Gone are the feeling of clunkiness and resistance Leopard was giving me: things flow so easily in Ubuntu that I actually forget about the OS. Of course, I turned off all the fancy schmancy visual effects and activated just what is needed to jump between applications and desktops with a few key strokes.
I can now enjoy standard Java JDKs, too! It is a pretty good feeling to be back to year 2008.
In the migration, I have lost Entourage, which is in fact probably a blessing. I now use Thunderbird IMAP-connected to Exchange and have a Firefox tab opened to the Outlook Web Access 2007 (pre-web 2.0) interface to access my calendar.
The only application I will sure miss is the excellent Omnigraffle Pro. Any decent alternative on Linux?
So after this trial, I decided, Gentle heron, that I will not pull the feathers off your head.
Gone are the feeling of clunkiness and resistance Leopard was giving me: things flow so easily in Ubuntu that I actually forget about the OS. Of course, I turned off all the fancy schmancy visual effects and activated just what is needed to jump between applications and desktops with a few key strokes.
I can now enjoy standard Java JDKs, too! It is a pretty good feeling to be back to year 2008.
In the migration, I have lost Entourage, which is in fact probably a blessing. I now use Thunderbird IMAP-connected to Exchange and have a Firefox tab opened to the Outlook Web Access 2007 (pre-web 2.0) interface to access my calendar.
The only application I will sure miss is the excellent Omnigraffle Pro. Any decent alternative on Linux?
So after this trial, I decided, Gentle heron, that I will not pull the feathers off your head.
Labels:
Platform
Friday, May 23, 2008
Unit Tests: No Future?
Industry expert Andrew Binstock has just posted an entry in his blog titled "Is the popularity of unit tests waning?", where he discusses the staggering state of the practice of unit testing.
Andrew asks the crucial question of why did we end in this situation. I do not pretend to have the answer, but here are some patterns that I have observed, which, I think, could decrease the appeal of unit testing.
Bad unit testing practices
It is very easy to write fragile unit tests, for example by using stubs when mocks would be enough or by creating time sensitive tests that work intermittently or stop working after a while. Similarly, it is common to see integration-like tests sleep into the realm of unit testing, making these tests fragile, slow and dependent of external resources.
These bad practices tend to lead to a decrease of confidence in unit tests, which then can lead to a progressive reduction of their usage. For example, one programmer can decide he will not write any tests anymore for DAOs after struggling with poorly written tests.
Bad software design
There is a tight relationship between good code design and testability. I have already blogged about how increasing test coverage can lead to a better design: unfortunately, not all developers are ready to review their design to make their code more testable.
To be fair, our languages and frameworks often force us to write code that is hard or uninteresting to test. Who wants to write unit tests for infamous JavaBeans getters and setters? Not Allen Holub, for sure!
Other programmatic idioms like equals/hashcode often exhibit a high cyclomatic complexity: writing complete tests for those would be tedious, unless one use test generating products like Agitator, from late Agitar, or Jtest, from Parasoft.
Test benefits blindness
Management tend to be blind to the benefits of unit testing and very aware of its costs. A project I know have been deemed to have gone overboard with unit testing. At the same time, this project has the code base that is the most maintainable, flexible and fun to work with. It seems something prevents businesses to see the value added of solid test practices, as if it was all about some obscure geeky self satisfaction activities (as writing "perfect code" would be).
No Future?
Is there any hope, then? I think that, like for most of our problems on this little planet, education is the key. Whether we look at software engineers or business managers, there is still a great lack of education about unit testing, how it works and why it pays back.
Andrew asks the crucial question of why did we end in this situation. I do not pretend to have the answer, but here are some patterns that I have observed, which, I think, could decrease the appeal of unit testing.
Bad unit testing practices
It is very easy to write fragile unit tests, for example by using stubs when mocks would be enough or by creating time sensitive tests that work intermittently or stop working after a while. Similarly, it is common to see integration-like tests sleep into the realm of unit testing, making these tests fragile, slow and dependent of external resources.
These bad practices tend to lead to a decrease of confidence in unit tests, which then can lead to a progressive reduction of their usage. For example, one programmer can decide he will not write any tests anymore for DAOs after struggling with poorly written tests.
Bad software design
There is a tight relationship between good code design and testability. I have already blogged about how increasing test coverage can lead to a better design: unfortunately, not all developers are ready to review their design to make their code more testable.
To be fair, our languages and frameworks often force us to write code that is hard or uninteresting to test. Who wants to write unit tests for infamous JavaBeans getters and setters? Not Allen Holub, for sure!
Other programmatic idioms like equals/hashcode often exhibit a high cyclomatic complexity: writing complete tests for those would be tedious, unless one use test generating products like Agitator, from late Agitar, or Jtest, from Parasoft.
Test benefits blindness
Management tend to be blind to the benefits of unit testing and very aware of its costs. A project I know have been deemed to have gone overboard with unit testing. At the same time, this project has the code base that is the most maintainable, flexible and fun to work with. It seems something prevents businesses to see the value added of solid test practices, as if it was all about some obscure geeky self satisfaction activities (as writing "perfect code" would be).
No Future?
Is there any hope, then? I think that, like for most of our problems on this little planet, education is the key. Whether we look at software engineers or business managers, there is still a great lack of education about unit testing, how it works and why it pays back.
Labels:
Craftsmanship,
Testing
Wednesday, May 21, 2008
SVN? VoilaSVN!
If you are using Subversion as you source control management system, there is now a way to go further with it and turn it into a full fledged project and knowledge management system.
Indeed, Arcetis has just released the first version of VoilaSVN Enterprise Edition, which "provides the tools to successfully manage your projects, to coordinate all your resources and capitalise on the knowledge of your team".
VoilaSVN leverages GWT to deliver a smooth web interface, which is pretty nice for a tool on which you will have to enter and mine data.
There is also a free edition, if you have simpler needs. So give it a try and, voila!, see how far you can go with Subversion.
Indeed, Arcetis has just released the first version of VoilaSVN Enterprise Edition, which "provides the tools to successfully manage your projects, to coordinate all your resources and capitalise on the knowledge of your team".
VoilaSVN leverages GWT to deliver a smooth web interface, which is pretty nice for a tool on which you will have to enter and mine data.
There is also a free edition, if you have simpler needs. So give it a try and, voila!, see how far you can go with Subversion.
Labels:
Tools
Friday, May 16, 2008
Just Read: Release It! and ThoughtWorks Anthology

If you intend to write software that lasts and fares well during its journey, give yourself a hand and read this book. From actual situations, the author derives very concrete recommendations towards writing scalable and operable applications.
As far as I am concerned, reading this book has been an exhilarating experience. Though to a much smaller scale, I have experienced the same pains and came to similar conclusions than the author. Reading some pages entailed some sheer moments of excitement, very much like: "I have been saying the same!".
All in all, I am grateful that Michael Nygard has written such an authoritative book on this crucial matter for now no-one will be allowed to say "I did not know" anymore.

This book does a pretty good job at offering insights and real world feedback from top notch ThoughtWorkers on a variety of software development related subjects (coding, designing, building, QA). For a collection of essays from different authors, the overall uniformity has been pretty well maintained both on style and content.
As a software developer, I have found Neal Ford's "Polyglot Programming" and Jeff Bay's "Object Calisthenics" to be the most compelling pieces of this book.
At the end of the day, the real interest of this anthology resides in the fact that a small consulting firm has decided to share its sheer passion for software in a direct and hype-free manner. How many of the big ones out there could do the same?
Labels:
Readings
Saturday, May 03, 2008
I don't know... yet!
In movies, computers know everything. Just ask the computer and you will get an accurate answer now. For us, developers who have to deliver the promises made by Hollywood, programming software that gives correct answers immediately is very easy. Unfortunately it is also very costly.
Giving the correct answer generally translates into querying the one source of truth of an application: the database. This is easy.
Sometimes the database is far away from the access layer the caller interacts with, whether it is a web tier or a service tier. No problem! It is very easy to ask the caller to hold his breath until a long chain of synchronous calls gather all the correct data needed and finally present it as a glorious reply. Usually the caller has enough breath to wait until it has to get some fresh air again (some call this a time out).
All this is fairly easy but unfortunately very costly. Contention to access the centralized source of blessed truth increases as more and more callers want to access it. If, for the sins of the application, it gets successful, the number of callers holding their breath while waiting will increase dramatically.
It is a time when the application would love to have callers with smaller lungs, so they could time out faster and free the threads they are holding while sitting idle. But they are not: the dozens of second they wait are an eternity for a server. Instead of being glued waiting for a remote service or database to deliver the ultimate answer, the application would like to afford replying:
At least, this imprecise but immediate answer would allow the server to release threads almost as fast as they come. This would give the application the luxury of replying later, whenever possible...
Of course, such a paradigm shift is all but transparent for the caller: he must learn to deal with imprecision. He must embrace asynchronism. He must survive eventual consistency.
We now have the tools, from the web tier to the back end. Can we succeed in this paradigm shift?
I don't know... yet!
Giving the correct answer generally translates into querying the one source of truth of an application: the database. This is easy.
Sometimes the database is far away from the access layer the caller interacts with, whether it is a web tier or a service tier. No problem! It is very easy to ask the caller to hold his breath until a long chain of synchronous calls gather all the correct data needed and finally present it as a glorious reply. Usually the caller has enough breath to wait until it has to get some fresh air again (some call this a time out).
All this is fairly easy but unfortunately very costly. Contention to access the centralized source of blessed truth increases as more and more callers want to access it. If, for the sins of the application, it gets successful, the number of callers holding their breath while waiting will increase dramatically.
It is a time when the application would love to have callers with smaller lungs, so they could time out faster and free the threads they are holding while sitting idle. But they are not: the dozens of second they wait are an eternity for a server. Instead of being glued waiting for a remote service or database to deliver the ultimate answer, the application would like to afford replying:
I don't know... yet!
At least, this imprecise but immediate answer would allow the server to release threads almost as fast as they come. This would give the application the luxury of replying later, whenever possible...
Of course, such a paradigm shift is all but transparent for the caller: he must learn to deal with imprecision. He must embrace asynchronism. He must survive eventual consistency.
We now have the tools, from the web tier to the back end. Can we succeed in this paradigm shift?
I don't know... yet!
Labels:
Craftsmanship
Saturday, April 26, 2008
Not Dash Bored
I love dashboards. You probably got that from my previous post. If I had enough screens, I would be surrounded by dashboards of all kinds: work tasks lists, continuous integration server control panel, project metrics sites and server monitors.
Maybe this comes from the time when I was handling another kind of dashboard, from which my life was directly depending!
Dashboards are great because they present a synthetic view of a situation in a form that is visually expressive and does not require a lot of concentrated attention to capture relevant and crucial information.
The most recent dashboard I built exposes particular aspects of several instances of Mule ESB I have under my control. Through JMX, Mule exposes a wealth of statistics about the different components of a particular instance. Here is a small portion of the HTML console that displays this information:

This is too much information the brain, or at least my brain, can digest efficiently and quickly enough. Hence I created a simplified view that represents the variations of message routing statistics on a selection of components:

The components are simply selected by name: I decided to prefix the important ones with "process" and "dispatch" and derived from there a simple selection pattern. I use different colors to show different states:
Of course, this does not compare to the professional grade monitoring tools you can buy from MuleSource, but is already handy for deployments of limited scope and criticality.
I think the most interesting aspect of this dashboard is how quickly you can develop the ability to recognize a normal behavior pattern from a faulty one. It is pretty much like reading the matrix undecoded... Now how could you get bored from such a board!
UPDATE 03-MAY-2008: This dashboard is now available on MuleForge.
Maybe this comes from the time when I was handling another kind of dashboard, from which my life was directly depending!
Dashboards are great because they present a synthetic view of a situation in a form that is visually expressive and does not require a lot of concentrated attention to capture relevant and crucial information.
The most recent dashboard I built exposes particular aspects of several instances of Mule ESB I have under my control. Through JMX, Mule exposes a wealth of statistics about the different components of a particular instance. Here is a small portion of the HTML console that displays this information:

This is too much information the brain, or at least my brain, can digest efficiently and quickly enough. Hence I created a simplified view that represents the variations of message routing statistics on a selection of components:

The components are simply selected by name: I decided to prefix the important ones with "process" and "dispatch" and derived from there a simple selection pattern. I use different colors to show different states:
- Green: no activity,
- Yellow: at least one message went through,
- Orange: the backing event queue has been resized up,
- Red: an error has been routed,
- Gray: no delta available (first call of the dashboard),
- Black: component statistics unavailable.
Of course, this does not compare to the professional grade monitoring tools you can buy from MuleSource, but is already handy for deployments of limited scope and criticality.
I think the most interesting aspect of this dashboard is how quickly you can develop the ability to recognize a normal behavior pattern from a faulty one. It is pretty much like reading the matrix undecoded... Now how could you get bored from such a board!
UPDATE 03-MAY-2008: This dashboard is now available on MuleForge.
Sunday, April 20, 2008
Building Value
This week, one of my colleague (a guy named Josh) was all grumpy about the time he spent adding documentation in his projects Maven sites so operations could deploy his application properly. This made me reflect on how great is this tool not only for building software but value in general.
By giving developers the opportunity to document in the same environment as where they code and by embedding the HTML rendition of this documentation in the project site alongside all the other technical reports, Maven presents to the stakeholders an overview of the value built by a project.
Value is a vague term, so let me be more specific:
This said, thanks to continuous integration and the dashboard plugin, I believe it is possible to catch the interest of a broader audience because it is now possible to display trends instead of static values: management understands trends.
For example, a flat test coverage value is meaningless but a trend that shows it increases means that quality, thus value, does the same. Similarly, comparing projects based on their metrics is a non-sense, while comparing their trends makes sense.
Did you have any good experience sharing Maven sites to management? Did they get the feeling that value was being built?
By giving developers the opportunity to document in the same environment as where they code and by embedding the HTML rendition of this documentation in the project site alongside all the other technical reports, Maven presents to the stakeholders an overview of the value built by a project.
Value is a vague term, so let me be more specific:
- Intrinsic value: technical metrics, like the ones coming from static bug analysis, package dependencies, test coverage or code style compliance, represent the core value of the code in term of quality, flexibility and maintainability.
- Business value: results from acceptance testing tools like Selenium or FitNesse represent the capacity of the application to satisfy business requirements.
- Corporate value: on top of the auto-generated technical documentation from the code base, all the extra documentation that is added, whether it is installation guides, monitoring procedures or deployment diagrams, brings value to the company as a whole, from operation teams to new recruits.
This said, thanks to continuous integration and the dashboard plugin, I believe it is possible to catch the interest of a broader audience because it is now possible to display trends instead of static values: management understands trends.
For example, a flat test coverage value is meaningless but a trend that shows it increases means that quality, thus value, does the same. Similarly, comparing projects based on their metrics is a non-sense, while comparing their trends makes sense.
Did you have any good experience sharing Maven sites to management? Did they get the feeling that value was being built?
Labels:
Craftsmanship,
Tools
Saturday, April 12, 2008
Abstraction First
When designing services, the common wisdom is to opt for a contract-first approach, instead of an implementation-first one. There is no question that this is a valid approach but I think the emphasis should be put on the necessity to design a good abstraction first.
Consider this: technical implementations, especially in strongly typed languages, often result into a pollution of the client model by the service model. Interfaces or stubs used by a client to perform remote invocations can very easily become mixed with its own domain.
This creates a tension between a service provider and its consumers, as they often tend to drive the contract too much on their side because they consider the local crystallization of the contract as a part of their model. Exacerbated, this tendency can result in tight coupling between both parties.
Let me give you an old but typical example from my tumultuous past.
A little more than a decade ago, I worked on a corporate centralized contact management system. It was supposed to serve contact details (persons, organizations, addresses and whatnot) to all the applications in use in the company, including secretaries' word processors. Admittedly an interesting project, it in fact quickly turned to be a death march. I soon learned this was the sixth attempt of such an endeavor and that people wanted to see how a n00b like me would fare.
I then realized that each department wanted very different things out of this system and their views would not be reconcilable. I ended shipping a version that was only usable by the secretaries, which I think was a smart move as you must always be good to them!
The next n00b was assigned the creation of version 7 of the contact manager, built on what I did with the goal to generalize it to all departments. Of course it failed and the project disappeared for ever. Maybe the mythical number 7 was to be reached before the whole stuff could have been killed.
At that time, if I would have known better, I think the best I could have done would have been to advocate for the deployment of an LDAP provider. Indeed a directory server accessed via this protocol does not try to be everything for everybody and does not present a contract that any application would consider using in its own domain model. Yet it offers a simple and powerful abstraction that an application can use to query directory information and then tie them with its own object model.
Let me quote Uncle Bob:
To me this sounds almost like a caricature, where the most prominent features are made so obvious and visible that there is no doubt left about what really matters. A good abstraction for a service should then translate in clear intents and well defined boundaries, which would guide the creation of valuable contracts.
Finally, for all these existing services that want to invade your application domain, there are fortunately ways to remain insulated. For example you could:
Consider this: technical implementations, especially in strongly typed languages, often result into a pollution of the client model by the service model. Interfaces or stubs used by a client to perform remote invocations can very easily become mixed with its own domain.
This creates a tension between a service provider and its consumers, as they often tend to drive the contract too much on their side because they consider the local crystallization of the contract as a part of their model. Exacerbated, this tendency can result in tight coupling between both parties.
Let me give you an old but typical example from my tumultuous past.
A little more than a decade ago, I worked on a corporate centralized contact management system. It was supposed to serve contact details (persons, organizations, addresses and whatnot) to all the applications in use in the company, including secretaries' word processors. Admittedly an interesting project, it in fact quickly turned to be a death march. I soon learned this was the sixth attempt of such an endeavor and that people wanted to see how a n00b like me would fare.
I then realized that each department wanted very different things out of this system and their views would not be reconcilable. I ended shipping a version that was only usable by the secretaries, which I think was a smart move as you must always be good to them!
The next n00b was assigned the creation of version 7 of the contact manager, built on what I did with the goal to generalize it to all departments. Of course it failed and the project disappeared for ever. Maybe the mythical number 7 was to be reached before the whole stuff could have been killed.
At that time, if I would have known better, I think the best I could have done would have been to advocate for the deployment of an LDAP provider. Indeed a directory server accessed via this protocol does not try to be everything for everybody and does not present a contract that any application would consider using in its own domain model. Yet it offers a simple and powerful abstraction that an application can use to query directory information and then tie them with its own object model.
Let me quote Uncle Bob:
"Abstraction is the elimination of the irrelevant and the amplification of the essential"
To me this sounds almost like a caricature, where the most prominent features are made so obvious and visible that there is no doubt left about what really matters. A good abstraction for a service should then translate in clear intents and well defined boundaries, which would guide the creation of valuable contracts.
Finally, for all these existing services that want to invade your application domain, there are fortunately ways to remain insulated. For example you could:
- use dynamic language scriptlets to perform remote invocations and set values on your local domain,
- use Dozer to tie client stubs and your objects,
- consider invocation responses as raw XML and extract values out of them with XPath.
Labels:
Craftsmanship
Saturday, April 05, 2008
Fruit and coffea? I don't think so...
Last year, a guy named Marc Fleury ranted about how Macintosh was not a suited platform for software development. At the time I thought he just hated it because he is French, and French people hate everything. Let me quote him:
Nowadays, I am the one who daily (hourly) complain about how clunky and inappropriate is Mac OS X for Java developers. People around me roll their eyes and probably assume I just hate it because I am French too, and French people hate everything.
But, boy, since I started to use this platform professionally, how much do I find Mac to be a hindrance to my development activities! How often the OS gets in my way and pulls me out of the state of flow I am in...
Here is a non-exhaustive list of my grievances, so you can decide if it is mere ranting or if there are some valid reasons for my grumbling:
I will not mention Eclipse that dies unexpectedly on Leopard while it was stable on Tiger (yes I have configured the JVM memory parameters, thank you). I will not mention the disgrace that is Entourage, because it is a Microsoft product and Apple can not be blamed for it. And I will not mention that my MacBook Pro hard drive fried just after a year (bye-bye guarantee), which is the first time something like this happened to me for the past ten years that I have been working on laptops.
Nuf' said! So what are my platforms of choice for Java development? In order of preference: Kubuntu, Windows XP and... Mac OS X. But at home, I am very happy with the little white Mac Book we use for browsing, e-mailing and managing photos. For this kind of home activity, having an OS that shows off is acceptable. For professional usage, the less the OS gets in your face, the best it is.
"a mac is like a bimbo". It looks good and shiny from a distance, you think you really want to try it. But once you do, after 2 weeks of "doing it" you are bored, bored to tears. Tired of everything, the pretty animation stuff, the big tatas, the transparent look and feel, the big tatas, the genie bullshit animation, the big tatas, the stuff that is different from windows "just to be different", the big tatas and the empty brains.
Nowadays, I am the one who daily (hourly) complain about how clunky and inappropriate is Mac OS X for Java developers. People around me roll their eyes and probably assume I just hate it because I am French too, and French people hate everything.
But, boy, since I started to use this platform professionally, how much do I find Mac to be a hindrance to my development activities! How often the OS gets in my way and pulls me out of the state of flow I am in...
Here is a non-exhaustive list of my grievances, so you can decide if it is mere ranting or if there are some valid reasons for my grumbling:
- Bitter Java: official JDK support is way behind other OSes. For example, the Apple JDK 6 was beta when I was on Tiger and disappeared on Leopard. Sure I could follow the great work of Landon Fuller with Open JDK, but I honestly do not have time to invest in such endeavors.
- Keyboard support is a joke: even if I am using QuickSilver, I constantly have to grab the mouse to click this, highlight that or dismiss a pop-up. The last one really drives me nuts: why is it that the escape key does not always cancel a pop-up dialog?
- Focus messy: there is always an application that steals the focus out of my working window. Maybe this is the fault of badly behaving applications, but it is the first OS where this happens to me so often that I get annoyed by it. Is this OS making it easier for applications to become focus-rude?
- Prompt to freeze: the UI freezes very easily, to the point I can not even switch applications or invoke the task killer. It seems that a badly behaving application can very easily mess-up the whole user interface, hence the whole OS.
- Hidden BSD: it is hard to say this but as far as the overall stability is concerned, OS X reminds me of Windows ME sans Blue Screen of Death. I have to do hard reboot at least once a week, whether the screen saver locks me out for ever or the UI freezes to the point I can not do anything except sitting on the power button. I have not seen this in any OS for almost a decade. And having this kind of behavior on an almost newly re-installed machine, with minimal applications running, is a plain disappointment.
- Finder sucks big time: for an OS that is supposed to be all about user experience, I find the Finder to be a complete disgrace. Try to create a new directory: it never ends up where you want it. Try to shift-select files with the keyboard: going up and down performs some counter-intuitive file selections. Then use the mouse to drag the selected files: they might end-up where you drop them, or not. Instead of adding a new view in Leopard (cover flow), if only Apple could have fixed the existing ones so they become really usable (who uses the insane column view?).
- Flaky AirPort: I am constantly losing connectivity with my Wifi router. Maybe my cheap D-Link router is the issue, but then why my other non-Mac machines have no problem maintaining their connections up and running for hours?
I will not mention Eclipse that dies unexpectedly on Leopard while it was stable on Tiger (yes I have configured the JVM memory parameters, thank you). I will not mention the disgrace that is Entourage, because it is a Microsoft product and Apple can not be blamed for it. And I will not mention that my MacBook Pro hard drive fried just after a year (bye-bye guarantee), which is the first time something like this happened to me for the past ten years that I have been working on laptops.
Nuf' said! So what are my platforms of choice for Java development? In order of preference: Kubuntu, Windows XP and... Mac OS X. But at home, I am very happy with the little white Mac Book we use for browsing, e-mailing and managing photos. For this kind of home activity, having an OS that shows off is acceptable. For professional usage, the less the OS gets in your face, the best it is.
Labels:
Platform
Wednesday, April 02, 2008
Healthy Health Checks?
Here is a little story that happened to a friend of mine a few years ago. Users of his web application started to complain about the system being broken and not responding anymore. The curious thing was that the operation team was not aware of any issue. After checking with them, it appeared that the monitor they had in place for this web application was simply checking if an HTTP response was received. Any response. Even a 500 one!
This sounds naive and ridiculous but setting up application monitoring is a subject that is a little more hairy than it appears at first glance. Consider another more recent case that came to my attention: in this case, the application was still replying positively to its health check monitor but was not functioning properly, as it was unable to access required file system resources. Again, the end users were affected while the monitoring was happily receiving correct responses from the application.
So how can we, software developers, create health checks that operations can rely on?
Taking the canonical multi-tiered web application as an example, the following schema shows an health check that is too shallow to be useful (in red) and one that exercises the full layer depth (in green).
While it is clear that the shallow approach brings little value, as far as end user quality of service is concerned, why do not we always shoot for the deep approach then?
Well, if you consider how a serious load balancer appliance (like BIG-IP) works, you will realize that if performs health checks very regularly (by default every 5 seconds) in order to have the most up to date view of the sanity of the members of the pools it handles. Bearing this mind, if an health check request would exercise the full depth of an application, you would have a permanent load added to your system, which would increase the strain on your diverse resources, down to the database itself. With a farm of n servers, the cumulated strain induced by the health check requests on all the members of it would start to be non negligible on any shared resource.
My take on this would be the following: create an internal watchdog that evaluates the sanity of the application at a reasonable pace and report the current state of this watchdog when a monitor requests a health check from the application.
As shown in the above schema, the watchdog life cycle is uncoupled from the health check one, which allows to reduce strain on the underlying resources while allowing the monitoring environment to become aware of an application issue almost as soon as the application realizes it itself (because the monitor polling frequency will be kept high).
What is your own experience in this field and what is the path you have followed in order to build dependable health checks?
This sounds naive and ridiculous but setting up application monitoring is a subject that is a little more hairy than it appears at first glance. Consider another more recent case that came to my attention: in this case, the application was still replying positively to its health check monitor but was not functioning properly, as it was unable to access required file system resources. Again, the end users were affected while the monitoring was happily receiving correct responses from the application.
So how can we, software developers, create health checks that operations can rely on?
Taking the canonical multi-tiered web application as an example, the following schema shows an health check that is too shallow to be useful (in red) and one that exercises the full layer depth (in green).
While it is clear that the shallow approach brings little value, as far as end user quality of service is concerned, why do not we always shoot for the deep approach then?Well, if you consider how a serious load balancer appliance (like BIG-IP) works, you will realize that if performs health checks very regularly (by default every 5 seconds) in order to have the most up to date view of the sanity of the members of the pools it handles. Bearing this mind, if an health check request would exercise the full depth of an application, you would have a permanent load added to your system, which would increase the strain on your diverse resources, down to the database itself. With a farm of n servers, the cumulated strain induced by the health check requests on all the members of it would start to be non negligible on any shared resource.
My take on this would be the following: create an internal watchdog that evaluates the sanity of the application at a reasonable pace and report the current state of this watchdog when a monitor requests a health check from the application.
As shown in the above schema, the watchdog life cycle is uncoupled from the health check one, which allows to reduce strain on the underlying resources while allowing the monitoring environment to become aware of an application issue almost as soon as the application realizes it itself (because the monitor polling frequency will be kept high).What is your own experience in this field and what is the path you have followed in order to build dependable health checks?
Labels:
Craftsmanship,
Platform,
Tools
Saturday, March 29, 2008
Dumb XML
Beautiful Code is a book that can not let oneself indifferent. While reading it, I have been really annoyed by a statement made by one of the authors:
Of course, for trivial XML blocks that will never contain any special character nor vary in form too much, using a StringBuilder (not StringBuffer by the way, most of the time it is unnecessary to use this synchronized version) is more than enough. Of course, you can use helper class to encoding all strings and escape all entities.
But if you go further than the trivial use cases or if the data you integrate in your XML document comes from an uncontrolled source (like a database connection or another application layer), use a proper library for building XML.
XML is simple, but do not dumb it down to simplistic.
What I always tell people is that XML documents are just big text strings. Therefore, it's usually easier to just write one out using StringBuffer rather than trying to build a DOM (Document Object Model) or using a special XML generator library.I fully disagree with this position, because I have seen time and again the adverse results of such a simplistic approach to XML generation:
- Malformed XML: basic string construction does not handle general entities escaping, element name validity and correctly balanced tags. I once had to deal with an XML document that was so broken I initially thought it was SGML. It was neither and I ended using regular expressions instead of SAX to parse it.
- Invalid XML: applications that generate XML should be polite enough to validate the data they produce before sharing it with other applications. At the time DTDs were current, I followed the practice to add the external declaration only if the document was tested to be valid, like a proofing stamp. Of course, I also had to deal with an application that never considered important to output XML that complied with its own schema!
- Bad encoding: I realize that many developers live and work in a place where ASCII-7 is enough to represent all the characters they need. But the rest of the world cares for accents and other language particularities. Hence again: basic string building gives no guarantee in term of correct representation of Unicode characters.
Of course, for trivial XML blocks that will never contain any special character nor vary in form too much, using a StringBuilder (not StringBuffer by the way, most of the time it is unnecessary to use this synchronized version) is more than enough. Of course, you can use helper class to encoding all strings and escape all entities.
But if you go further than the trivial use cases or if the data you integrate in your XML document comes from an uncontrolled source (like a database connection or another application layer), use a proper library for building XML.
XML is simple, but do not dumb it down to simplistic.
Labels:
Craftsmanship
Thursday, March 27, 2008
Bugs Of Opportunity
Yesterday night, I fixed a trivial bug in NxBRE. Doing so, I have spent almost 5 times more time writing tests to assert the current behavior and the expected one when the bug would be killed.
This reminds me of an earlier reflection on how being test infected changed my reaction to incoming bugs. Prior to become green light addicted, bugs were my enemy and were received a treatment depending on my mood:
They are now an opportunity to improve and increase test coverage, regardless of the mood I happen to be in:
But, as you have certainly noticed it, grumbling is still part of the process!
This reminds me of an earlier reflection on how being test infected changed my reaction to incoming bugs. Prior to become green light addicted, bugs were my enemy and were received a treatment depending on my mood:
They are now an opportunity to improve and increase test coverage, regardless of the mood I happen to be in:
But, as you have certainly noticed it, grumbling is still part of the process!
Labels:
Craftsmanship,
My OSS,
Testing
Wednesday, March 19, 2008
Fighting The Good Fight
An article published yesterday in the Wall Street Journal (Pleasing Google's Tech-Savvy Staff), made me reflect on the fight for corporate standardization of technologies, a fight in which I have been pretty involved in the past 4 years.
This battle happens at many different levels: operating systems ; database, application and web servers ; development platforms, tools, frameworks and libraries. Is it worth fighting it?
First, let us bear in mind that there are some strong rationale in unifying the different technologies in an IT landscape. Here are a few:
I will not discuss the advantages of single server operating systems strategies: despite the clear advantage in term of resource management, the raise of virtualization platforms have somewhat made a multi-OS environment an easier possibility. Narrowing the discussion to developers, who have the tough job of taking decisions in a world where every day brings a new and promising tool, what is the actual risk of letting them make choices?
My experience in the matter taught me that:
This battle happens at many different levels: operating systems ; database, application and web servers ; development platforms, tools, frameworks and libraries. Is it worth fighting it?
First, let us bear in mind that there are some strong rationale in unifying the different technologies in an IT landscape. Here are a few:
- Limited and simplified licensing and support contracts negotiations,
- Facilitated software maintenance and operations,
- Improved interoperability and potential for re-use.
- Golden hammerism: Forcing the use of technologies into scenarios where they are not adequate,
- Team entrenchment: any technological choice usually satisfies as much people it displeases,
- Kool-aid intoxication: single vendor dependency often reduces options and opportunities for using better fitted approaches.
I will not discuss the advantages of single server operating systems strategies: despite the clear advantage in term of resource management, the raise of virtualization platforms have somewhat made a multi-OS environment an easier possibility. Narrowing the discussion to developers, who have the tough job of taking decisions in a world where every day brings a new and promising tool, what is the actual risk of letting them make choices?
My experience in the matter taught me that:
- No project gets doomed by its technological choices: I have seen more harm done by the abuse or misuse of a particular framework, than by the framework itself. And yes, this also holds true for projects condemned to use Entity Beans.
- Applications designed on the same platform do not inter-operate by the sole fact of being developed and run on the same platform.
- Similarly, re-use does not happen by unifying technologies (except if you develop for portals and want to re-use widgets without resorting to HTML scraping).
- Paid-for support for open source projects is seldom useful (while "quick-start" consulting gigs are valuable).
- Practice quality over dependencies uniformity: more than the usage of a common set of libraries, improving code readability, test coverage, application design and build practices have the biggest impact on the maintainability and evolutility of a particular application.
- Loose coupling over platform uniformity: an IT landscape greatly benefits from systems that have clear contracts between each other, interact in well defined and contained manners and can gracefully survive if their neighbors have temporarily fallen in digital limbos.
- Operation friendliness over environment uniformity: applications that are developed while having operations in mind have a happier life in this world. Targeting a particular server or database or OS does not automatically translate into an application that will be easily handled by the production team of a company.
Labels:
Craftsmanship,
Google,
Platform,
Tools
Just Read: Beautiful Code

Beautiful Code is probably the most unequal software book I have ever read, both in term of style and content. Some chapters are very formal and academic, while others are more relaxed and down to earth. Some chapters really offer food for thought in the matter of software development, while others are arid displays of obscure code with no lesson to gather from. If all the royalties were not given to Amnesty International, I would have felt totally frustrated by this book. At least, I have the feeling to have served a good cause with my money.
Labels:
Readings
Saturday, March 15, 2008
Unchaining Backward Chaining
NxBRE's Flow Engine is currently under work: I have added a simple backward chaining scheduler that can execute sets in order to produce a specified goal in the rule context. Not all rule bases qualify for it: no construct should exist outside of any set to be usable in this context.
This will allow to support Reaction RuleML as an input, alongside the current proprietary syntaxes, hence to be able to consume the output of Acumen's RuleManager, an excellent tool for BRE related development on the .NET platform.
Stay tuned for the upcoming new release. In the meantime, the most adventurous can already try the backward chaining engine by checking out the latest version out of SVN!
This will allow to support Reaction RuleML as an input, alongside the current proprietary syntaxes, hence to be able to consume the output of Acumen's RuleManager, an excellent tool for BRE related development on the .NET platform.
Stay tuned for the upcoming new release. In the meantime, the most adventurous can already try the backward chaining engine by checking out the latest version out of SVN!
Labels:
My OSS
Subscribe to:
Posts (Atom)
