Article: Building Content Oriented Integration Solutions With Mule and JCR

The Server Side has just published my article presenting all the features of the JCR Transport for Mule in action!

It is called: Building Content Oriented Integration Solutions With Mule and JCR.

Just Read: Dreaming in code

Dreaming in Code is a scary book, the kind of book that makes you wonder if it is really wise to keep pursuing the vain ambition of writing software. By telling the storing of Chandler, an open source project aimed at revolutionize Personal Information Management tools, the author takes us deep in the moving sands of software development.

The most daunting aspect of the book is the following: if the best developers in the world gathered together under the supervision of a level 5 leader (Mitch Kapor) struggle to build software like the rest of us mortals do our pesky daily jobs, then is there any hope?

Maybe hope is in the unintentional software we build while intending to build something else? We got Ruby on Rails, Blogger and Flickr that way, after all.

Heron, Gentil Heron

This is a follow-up for my last rant about how an unhappy Java developer I was on Leopard. I just had my first week of work with Hardy Heron running on my MacBook Pro and I am so glad I went through the few hours of setup and configuration: this bird really flies!

Gone are the feeling of clunkiness and resistance Leopard was giving me: things flow so easily in Ubuntu that I actually forget about the OS. Of course, I turned off all the fancy schmancy visual effects and activated just what is needed to jump between applications and desktops with a few key strokes.

I can now enjoy standard Java JDKs, too! It is a pretty good feeling to be back to year 2008.

In the migration, I have lost Entourage, which is in fact probably a blessing. I now use Thunderbird IMAP-connected to Exchange and have a Firefox tab opened to the Outlook Web Access 2007 (pre-web 2.0) interface to access my calendar.

The only application I will sure miss is the excellent Omnigraffle Pro. Any decent alternative on Linux?

So after this trial, I decided, Gentle heron, that I will not pull the feathers off your head.

Unit Tests: No Future?

Industry expert Andrew Binstock has just posted an entry in his blog titled "Is the popularity of unit tests waning?", where he discusses the staggering state of the practice of unit testing.

Andrew asks the crucial question of why did we end in this situation. I do not pretend to have the answer, but here are some patterns that I have observed, which, I think, could decrease the appeal of unit testing.

Bad unit testing practices

It is very easy to write fragile unit tests, for example by using stubs when mocks would be enough or by creating time sensitive tests that work intermittently or stop working after a while. Similarly, it is common to see integration-like tests sleep into the realm of unit testing, making these tests fragile, slow and dependent of external resources.

These bad practices tend to lead to a decrease of confidence in unit tests, which then can lead to a progressive reduction of their usage. For example, one programmer can decide he will not write any tests anymore for DAOs after struggling with poorly written tests.


Bad software design

There is a tight relationship between good code design and testability. I have already blogged about how increasing test coverage can lead to a better design: unfortunately, not all developers are ready to review their design to make their code more testable.

To be fair, our languages and frameworks often force us to write code that is hard or uninteresting to test. Who wants to write unit tests for infamous JavaBeans getters and setters? Not Allen Holub, for sure!

Other programmatic idioms like equals/hashcode often exhibit a high cyclomatic complexity: writing complete tests for those would be tedious, unless one use test generating products like Agitator, from late Agitar, or Jtest, from Parasoft.


Test benefits blindness

Management tend to be blind to the benefits of unit testing and very aware of its costs. A project I know have been deemed to have gone overboard with unit testing. At the same time, this project has the code base that is the most maintainable, flexible and fun to work with. It seems something prevents businesses to see the value added of solid test practices, as if it was all about some obscure geeky self satisfaction activities (as writing "perfect code" would be).


No Future?

Is there any hope, then? I think that, like for most of our problems on this little planet, education is the key. Whether we look at software engineers or business managers, there is still a great lack of education about unit testing, how it works and why it pays back.

SVN? VoilaSVN!

If you are using Subversion as you source control management system, there is now a way to go further with it and turn it into a full fledged project and knowledge management system.

Indeed, Arcetis has just released the first version of VoilaSVN Enterprise Edition, which "provides the tools to successfully manage your projects, to coordinate all your resources and capitalise on the knowledge of your team".

VoilaSVN leverages GWT to deliver a smooth web interface, which is pretty nice for a tool on which you will have to enter and mine data.

There is also a free edition, if you have simpler needs. So give it a try and, voila!, see how far you can go with Subversion.

Just Read: Release It! and ThoughtWorks Anthology

If you intend to write software that lasts and fares well during its journey, give yourself a hand and read this book. From actual situations, the author derives very concrete recommendations towards writing scalable and operable applications.

As far as I am concerned, reading this book has been an exhilarating experience. Though to a much smaller scale, I have experienced the same pains and came to similar conclusions than the author. Reading some pages entailed some sheer moments of excitement, very much like: "I have been saying the same!".

All in all, I am grateful that Michael Nygard has written such an authoritative book on this crucial matter for now no-one will be allowed to say "I did not know" anymore.



This book does a pretty good job at offering insights and real world feedback from top notch ThoughtWorkers on a variety of software development related subjects (coding, designing, building, QA). For a collection of essays from different authors, the overall uniformity has been pretty well maintained both on style and content.

As a software developer, I have found Neal Ford's "Polyglot Programming" and Jeff Bay's "Object Calisthenics" to be the most compelling pieces of this book.

At the end of the day, the real interest of this anthology resides in the fact that a small consulting firm has decided to share its sheer passion for software in a direct and hype-free manner. How many of the big ones out there could do the same?

I don't know... yet!

In movies, computers know everything. Just ask the computer and you will get an accurate answer now. For us, developers who have to deliver the promises made by Hollywood, programming software that gives correct answers immediately is very easy. Unfortunately it is also very costly.

Giving the correct answer generally translates into querying the one source of truth of an application: the database. This is easy.

Sometimes the database is far away from the access layer the caller interacts with, whether it is a web tier or a service tier. No problem! It is very easy to ask the caller to hold his breath until a long chain of synchronous calls gather all the correct data needed and finally present it as a glorious reply. Usually the caller has enough breath to wait until it has to get some fresh air again (some call this a time out).

All this is fairly easy but unfortunately very costly. Contention to access the centralized source of blessed truth increases as more and more callers want to access it. If, for the sins of the application, it gets successful, the number of callers holding their breath while waiting will increase dramatically.

It is a time when the application would love to have callers with smaller lungs, so they could time out faster and free the threads they are holding while sitting idle. But they are not: the dozens of second they wait are an eternity for a server. Instead of being glued waiting for a remote service or database to deliver the ultimate answer, the application would like to afford replying:

I don't know... yet!

At least, this imprecise but immediate answer would allow the server to release threads almost as fast as they come. This would give the application the luxury of replying later, whenever possible...

Of course, such a paradigm shift is all but transparent for the caller: he must learn to deal with imprecision. He must embrace asynchronism. He must survive eventual consistency.

We now have the tools, from the web tier to the back end. Can we succeed in this paradigm shift?

I don't know... yet!

Not Dash Bored

I love dashboards. You probably got that from my previous post. If I had enough screens, I would be surrounded by dashboards of all kinds: work tasks lists, continuous integration server control panel, project metrics sites and server monitors.

Maybe this comes from the time when I was handling another kind of dashboard, from which my life was directly depending!

(Yes, I am a pre-glass cockpit flying dinosaur)

Dashboards are great because they present a synthetic view of a situation in a form that is visually expressive and does not require a lot of concentrated attention to capture relevant and crucial information.

The most recent dashboard I built exposes particular aspects of several instances of Mule ESB I have under my control. Through JMX, Mule exposes a wealth of statistics about the different components of a particular instance. Here is a small portion of the HTML console that displays this information:


This is too much information the brain, or at least my brain, can digest efficiently and quickly enough. Hence I created a simplified view that represents the variations of message routing statistics on a selection of components:


The components are simply selected by name: I decided to prefix the important ones with "process" and "dispatch" and derived from there a simple selection pattern. I use different colors to show different states:

• Green: no activity,
  
• Yellow: at least one message went through,
• Orange: the backing event queue has been resized up,
• Red: an error has been routed,
• Gray: no delta available (first call of the dashboard),
• Black: component statistics unavailable.
  

Extra symbols represent a non active state of a component, like paused or stopped. The dashboard itself is the aggregation (frames, yuck!) of the HTML output of a specific component deployed alongside the other ones.

Of course, this does not compare to the professional grade monitoring tools you can buy from MuleSource, but is already handy for deployments of limited scope and criticality.

I think the most interesting aspect of this dashboard is how quickly you can develop the ability to recognize a normal behavior pattern from a faulty one. It is pretty much like reading the matrix undecoded... Now how could you get bored from such a board!

UPDATE 03-MAY-2008: This dashboard is now available on MuleForge.

Building Value

This week, one of my colleague (a guy named Josh) was all grumpy about the time he spent adding documentation in his projects Maven sites so operations could deploy his application properly. This made me reflect on how great is this tool not only for building software but value in general.

By giving developers the opportunity to document in the same environment as where they code and by embedding the HTML rendition of this documentation in the project site alongside all the other technical reports, Maven presents to the stakeholders an overview of the value built by a project.

Value is a vague term, so let me be more specific:

• Intrinsic value: technical metrics, like the ones coming from static bug analysis, package dependencies, test coverage or code style compliance, represent the core value of the code in term of quality, flexibility and maintainability.
• Business value: results from acceptance testing tools like Selenium or FitNesse represent the capacity of the application to satisfy business requirements.
• Corporate value: on top of the auto-generated technical documentation from the code base, all the extra documentation that is added, whether it is installation guides, monitoring procedures or deployment diagrams, brings value to the company as a whole, from operation teams to new recruits.

With all this goodness available, I am still baffled by the limited number of managers who pay attention to the reports generated by Maven. I imagine that they might be too technical for "generic" managers who are in the business of software as if they would be in the one of gravel and stone delivery. Moreover, the usual metrics for software development is generally focused only of features delivery and deadlines.

This said, thanks to continuous integration and the dashboard plugin, I believe it is possible to catch the interest of a broader audience because it is now possible to display trends instead of static values: management understands trends.

For example, a flat test coverage value is meaningless but a trend that shows it increases means that quality, thus value, does the same. Similarly, comparing projects based on their metrics is a non-sense, while comparing their trends makes sense.

Did you have any good experience sharing Maven sites to management? Did they get the feeling that value was being built?

Abstraction First

When designing services, the common wisdom is to opt for a contract-first approach, instead of an implementation-first one. There is no question that this is a valid approach but I think the emphasis should be put on the necessity to design a good abstraction first.

Consider this: technical implementations, especially in strongly typed languages, often result into a pollution of the client model by the service model. Interfaces or stubs used by a client to perform remote invocations can very easily become mixed with its own domain.

This creates a tension between a service provider and its consumers, as they often tend to drive the contract too much on their side because they consider the local crystallization of the contract as a part of their model. Exacerbated, this tendency can result in tight coupling between both parties.

Let me give you an old but typical example from my tumultuous past.

A little more than a decade ago, I worked on a corporate centralized contact management system. It was supposed to serve contact details (persons, organizations, addresses and whatnot) to all the applications in use in the company, including secretaries' word processors. Admittedly an interesting project, it in fact quickly turned to be a death march. I soon learned this was the sixth attempt of such an endeavor and that people wanted to see how a n00b like me would fare.

I then realized that each department wanted very different things out of this system and their views would not be reconcilable. I ended shipping a version that was only usable by the secretaries, which I think was a smart move as you must always be good to them!

The next n00b was assigned the creation of version 7 of the contact manager, built on what I did with the goal to generalize it to all departments. Of course it failed and the project disappeared for ever. Maybe the mythical number 7 was to be reached before the whole stuff could have been killed.

At that time, if I would have known better, I think the best I could have done would have been to advocate for the deployment of an LDAP provider. Indeed a directory server accessed via this protocol does not try to be everything for everybody and does not present a contract that any application would consider using in its own domain model. Yet it offers a simple and powerful abstraction that an application can use to query directory information and then tie them with its own object model.

Let me quote Uncle Bob:

"Abstraction is the elimination of the irrelevant and the amplification of the essential"

To me this sounds almost like a caricature, where the most prominent features are made so obvious and visible that there is no doubt left about what really matters. A good abstraction for a service should then translate in clear intents and well defined boundaries, which would guide the creation of valuable contracts.

Finally, for all these existing services that want to invade your application domain, there are fortunately ways to remain insulated. For example you could:

• use dynamic language scriptlets to perform remote invocations and set values on your local domain,
• use Dozer to tie client stubs and your objects,
  
• consider invocation responses as raw XML and extract values out of them with XPath.
  

Fruit and coffea? I don't think so...

Last year, a guy named Marc Fleury ranted about how Macintosh was not a suited platform for software development. At the time I thought he just hated it because he is French, and French people hate everything. Let me quote him:

"a mac is like a bimbo". It looks good and shiny from a distance, you think you really want to try it. But once you do, after 2 weeks of "doing it" you are bored, bored to tears. Tired of everything, the pretty animation stuff, the big tatas, the transparent look and feel, the big tatas, the genie bullshit animation, the big tatas, the stuff that is different from windows "just to be different", the big tatas and the empty brains.

Nowadays, I am the one who daily (hourly) complain about how clunky and inappropriate is Mac OS X for Java developers. People around me roll their eyes and probably assume I just hate it because I am French too, and French people hate everything.

But, boy, since I started to use this platform professionally, how much do I find Mac to be a hindrance to my development activities! How often the OS gets in my way and pulls me out of the state of flow I am in...

Here is a non-exhaustive list of my grievances, so you can decide if it is mere ranting or if there are some valid reasons for my grumbling:

• Bitter Java: official JDK support is way behind other OSes. For example, the Apple JDK 6 was beta when I was on Tiger and disappeared on Leopard. Sure I could follow the great work of Landon Fuller with Open JDK, but I honestly do not have time to invest in such endeavors.
• Keyboard support is a joke: even if I am using QuickSilver, I constantly have to grab the mouse to click this, highlight that or dismiss a pop-up. The last one really drives me nuts: why is it that the escape key does not always cancel a pop-up dialog?
• Focus messy: there is always an application that steals the focus out of my working window. Maybe this is the fault of badly behaving applications, but it is the first OS where this happens to me so often that I get annoyed by it. Is this OS making it easier for applications to become focus-rude?
• Prompt to freeze: the UI freezes very easily, to the point I can not even switch applications or invoke the task killer. It seems that a badly behaving application can very easily mess-up the whole user interface, hence the whole OS.
• Hidden BSD: it is hard to say this but as far as the overall stability is concerned, OS X reminds me of Windows ME sans Blue Screen of Death. I have to do hard reboot at least once a week, whether the screen saver locks me out for ever or the UI freezes to the point I can not do anything except sitting on the power button. I have not seen this in any OS for almost a decade. And having this kind of behavior on an almost newly re-installed machine, with minimal applications running, is a plain disappointment.
• Finder sucks big time: for an OS that is supposed to be all about user experience, I find the Finder to be a complete disgrace. Try to create a new directory: it never ends up where you want it. Try to shift-select files with the keyboard: going up and down performs some counter-intuitive file selections. Then use the mouse to drag the selected files: they might end-up where you drop them, or not. Instead of adding a new view in Leopard (cover flow), if only Apple could have fixed the existing ones so they become really usable (who uses the insane column view?).
• Flaky AirPort: I am constantly losing connectivity with my Wifi router. Maybe my cheap D-Link router is the issue, but then why my other non-Mac machines have no problem maintaining their connections up and running for hours?

I will not mention Eclipse that dies unexpectedly on Leopard while it was stable on Tiger (yes I have configured the JVM memory parameters, thank you). I will not mention the disgrace that is Entourage, because it is a Microsoft product and Apple can not be blamed for it. And I will not mention that my MacBook Pro hard drive fried just after a year (bye-bye guarantee), which is the first time something like this happened to me for the past ten years that I have been working on laptops.

Nuf' said! So what are my platforms of choice for Java development? In order of preference: Kubuntu, Windows XP and... Mac OS X. But at home, I am very happy with the little white Mac Book we use for browsing, e-mailing and managing photos. For this kind of home activity, having an OS that shows off is acceptable. For professional usage, the less the OS gets in your face, the best it is.

Healthy Health Checks?

Here is a little story that happened to a friend of mine a few years ago. Users of his web application started to complain about the system being broken and not responding anymore. The curious thing was that the operation team was not aware of any issue. After checking with them, it appeared that the monitor they had in place for this web application was simply checking if an HTTP response was received. Any response. Even a 500 one!

This sounds naive and ridiculous but setting up application monitoring is a subject that is a little more hairy than it appears at first glance. Consider another more recent case that came to my attention: in this case, the application was still replying positively to its health check monitor but was not functioning properly, as it was unable to access required file system resources. Again, the end users were affected while the monitoring was happily receiving correct responses from the application.

So how can we, software developers, create health checks that operations can rely on?

Taking the canonical multi-tiered web application as an example, the following schema shows an health check that is too shallow to be useful (in red) and one that exercises the full layer depth (in green).
While it is clear that the shallow approach brings little value, as far as end user quality of service is concerned, why do not we always shoot for the deep approach then?

Well, if you consider how a serious load balancer appliance (like BIG-IP) works, you will realize that if performs health checks very regularly (by default every 5 seconds) in order to have the most up to date view of the sanity of the members of the pools it handles. Bearing this mind, if an health check request would exercise the full depth of an application, you would have a permanent load added to your system, which would increase the strain on your diverse resources, down to the database itself. With a farm of n servers, the cumulated strain induced by the health check requests on all the members of it would start to be non negligible on any shared resource.

My take on this would be the following: create an internal watchdog that evaluates the sanity of the application at a reasonable pace and report the current state of this watchdog when a monitor requests a health check from the application.

As shown in the above schema, the watchdog life cycle is uncoupled from the health check one, which allows to reduce strain on the underlying resources while allowing the monitoring environment to become aware of an application issue almost as soon as the application realizes it itself (because the monitor polling frequency will be kept high).

What is your own experience in this field and what is the path you have followed in order to build dependable health checks?

Dumb XML

Beautiful Code is a book that can not let oneself indifferent. While reading it, I have been really annoyed by a statement made by one of the authors:

What I always tell people is that XML documents are just big text strings. Therefore, it's usually easier to just write one out using StringBuffer rather than trying to build a DOM (Document Object Model) or using a special XML generator library.

I fully disagree with this position, because I have seen time and again the adverse results of such a simplistic approach to XML generation:

• Malformed XML: basic string construction does not handle general entities escaping, element name validity and correctly balanced tags. I once had to deal with an XML document that was so broken I initially thought it was SGML. It was neither and I ended using regular expressions instead of SAX to parse it.
• Invalid XML: applications that generate XML should be polite enough to validate the data they produce before sharing it with other applications. At the time DTDs were current, I followed the practice to add the external declaration only if the document was tested to be valid, like a proofing stamp. Of course, I also had to deal with an application that never considered important to output XML that complied with its own schema!
• Bad encoding: I realize that many developers live and work in a place where ASCII-7 is enough to represent all the characters they need. But the rest of the world cares for accents and other language particularities. Hence again: basic string building gives no guarantee in term of correct representation of Unicode characters.
  

Of course, for trivial XML blocks that will never contain any special character nor vary in form too much, using a StringBuilder (not StringBuffer by the way, most of the time it is unnecessary to use this synchronized version) is more than enough. Of course, you can use helper class to encoding all strings and escape all entities.

But if you go further than the trivial use cases or if the data you integrate in your XML document comes from an uncontrolled source (like a database connection or another application layer), use a proper library for building XML.

XML is simple, but do not dumb it down to simplistic.

Bugs Of Opportunity

Yesterday night, I fixed a trivial bug in NxBRE. Doing so, I have spent almost 5 times more time writing tests to assert the current behavior and the expected one when the bug would be killed.

This reminds me of an earlier reflection on how being test infected changed my reaction to incoming bugs. Prior to become green light addicted, bugs were my enemy and were received a treatment depending on my mood:

They are now an opportunity to improve and increase test coverage, regardless of the mood I happen to be in:
But, as you have certainly noticed it, grumbling is still part of the process!

Fighting The Good Fight

An article published yesterday in the Wall Street Journal (Pleasing Google's Tech-Savvy Staff), made me reflect on the fight for corporate standardization of technologies, a fight in which I have been pretty involved in the past 4 years.

This battle happens at many different levels: operating systems ; database, application and web servers ; development platforms, tools, frameworks and libraries. Is it worth fighting it?

First, let us bear in mind that there are some strong rationale in unifying the different technologies in an IT landscape. Here are a few:

• Limited and simplified licensing and support contracts negotiations,
• Facilitated software maintenance and operations,
• Improved interoperability and potential for re-use.
  

But there are risks too, like:

• Golden hammerism: Forcing the use of technologies into scenarios where they are not adequate,
• Team entrenchment: any technological choice usually satisfies as much people it displeases,
  
• Kool-aid intoxication: single vendor dependency often reduces options and opportunities for using better fitted approaches.
  

Is the Google way, where people are left free - and responsible - for their technological choices a viable approach? Can it even work anywhere else, where the density of geniuses is much lower?

I will not discuss the advantages of single server operating systems strategies: despite the clear advantage in term of resource management, the raise of virtualization platforms have somewhat made a multi-OS environment an easier possibility. Narrowing the discussion to developers, who have the tough job of taking decisions in a world where every day brings a new and promising tool, what is the actual risk of letting them make choices?

My experience in the matter taught me that:

• No project gets doomed by its technological choices: I have seen more harm done by the abuse or misuse of a particular framework, than by the framework itself. And yes, this also holds true for projects condemned to use Entity Beans.
• Applications designed on the same platform do not inter-operate by the sole fact of being developed and run on the same platform.
• Similarly, re-use does not happen by unifying technologies (except if you develop for portals and want to re-use widgets without resorting to HTML scraping).
  
• Paid-for support for open source projects is seldom useful (while "quick-start" consulting gigs are valuable).
  

So, rather than the quixotic pursuit of the perfect unified IT environment, what is the good fight worth sweating and bleeding for? I believe it is worth fighting for the following:

• Practice quality over dependencies uniformity: more than the usage of a common set of libraries, improving code readability, test coverage, application design and build practices have the biggest impact on the maintainability and evolutility of a particular application.
• Loose coupling over platform uniformity: an IT landscape greatly benefits from systems that have clear contracts between each other, interact in well defined and contained manners and can gracefully survive if their neighbors have temporarily fallen in digital limbos.
• Operation friendliness over environment uniformity: applications that are developed while having operations in mind have a happier life in this world. Targeting a particular server or database or OS does not automatically translate into an application that will be easily handled by the production team of a company.
  

Let us fight the good fight!

Just Read: Beautiful Code

Beautiful Code is probably the most unequal software book I have ever read, both in term of style and content. Some chapters are very formal and academic, while others are more relaxed and down to earth. Some chapters really offer food for thought in the matter of software development, while others are arid displays of obscure code with no lesson to gather from. If all the royalties were not given to Amnesty International, I would have felt totally frustrated by this book. At least, I have the feeling to have served a good cause with my money.

Unchaining Backward Chaining

NxBRE's Flow Engine is currently under work: I have added a simple backward chaining scheduler that can execute sets in order to produce a specified goal in the rule context. Not all rule bases qualify for it: no construct should exist outside of any set to be usable in this context.

This will allow to support Reaction RuleML as an input, alongside the current proprietary syntaxes, hence to be able to consume the output of Acumen's RuleManager, an excellent tool for BRE related development on the .NET platform.

Stay tuned for the upcoming new release. In the meantime, the most adventurous can already try the backward chaining engine by checking out the latest version out of SVN!

Was @SD West 08

SD West 2008 is over and my brain hurts. But batteries are reloaded: this is what happens when you come close to the luminaries of our industry. And it is a great feeling.

The CMP event team did a great job both expanding the conference with new sessions and filtering out the vendor kool-aid sessions that sometimes managed to enter the schedule.

Kudos and a big thank you to the organizing team!

@SD West 08: Highlights of Day Five

Ten Ways to Improve Your Code (Neal Ford)

Even if I do not fly airplanes anymore (for now?), I try to stay informed about what happens in the pilots' world. One aspect of it that has always impressed me is the importance given to constantly improving one's practice. Software development should not be different. This is why I like this kind of session, as there is always something to improve somewhere!

Neal presented ten ways to walk this path of improvement. Here is a very short version of these ways, consult the slides on-line for the full version:

1. TDD for its design benefits (including DI).
2. Static analysis (byte-code & source analysis).
3. Good citizenship (encapsulation, invariants preservation from construction to mutation, cautious usage of singleton).
4. YAGNI (no speculative development, no more ivory-towerish frameworks pleaeaease).
5. Occam's razor (make the difference between essential & accidental complexity).
6. Question authority (including established so-called standards, rebuke anti-patterns).
7. SLAP (single level of abstraction principle: for this, refer to yesterday's "Clean Code").
8. Polyglot programming (leverage languages targeted at specific problems)
9. Learn the nuances of Java (discover the hidden JDK gems!).
10. Anti-objects (too inspired by the real world and solving problems in a reverse manner).
    

Neal also presented 10 corporate development smells that would be hilarious if they were not tragic! Discover them now. For point 4 (the inane debate about stored procedures), you can also read my take on it in this discussion.

To finish I think it is worth quoting Neal's answer to client who try to find excuses for not making things better:

"Your problem is not more complex or so different than everybody else's!"

Responsible Web Design (Scott Fegette)

As if anything in software development could be responsible (software liabilities anyone?), Scott re-stated the need of staying abreast of current standards and best practices during his very open and non-dogmatic session. He also reminded us where we come from and all the progress that has been made along the way.

Here are a few key points:

• Thinking semantically (avoiding layout-specific markup as much as possible), with microformats for example.
• Properly managing CSS styles.
• Opting for un-obstrusive JavaScript (enough of these links that break when JS is disabled!).
• Staying current with the standards and practices.
  

I will not claim I followed everything in this session, even though Scott did a great job to demonstrate his points with actual web sites and code samples.


Memory Leaks in Java Applications (Gregg Sporar)

My worst memory leak, despite forgetting anniversaries, happened in 2002 when using Xalan. Each XSL transformation was leaving a bunch of not collectible objects in memory, until the JVM had enough and died. Since then, I am worried about all sorts of leakage (and since I am getting older, this should not surprise you), but I still like XSL ;-)

Gregg is obsessed with memory leaks too, but he works for Sun and knows the problem like the palm of his hand. The goal he had for us in his class was:

"To understand the different types of tools and techniques available for finding memory leaks."

He then went through a thorough review and demonstration of different techniques:

• Post-mortem inspection: analyzing a thread dump after the JVM crashed with proper tooling.
• Instrumentation: to perform live analysis of what is going on in the JVM, mainly by analyzing the trend in object generation count.
• A combination of both: to capture dumps on running code.
  

He also explained the lack of user-friendly tooling for analyzing class-loader related memory leaks (that can lead to the infamous permanent generation error). To learn further about this interesting and uneasy subject, I encourage you to read the two articles he co-authored for ST&P in April and May 2007, from which he extracted the material for his talk.


Mock Objects / Mock Turtles: The Role of Patterns in TDD (Scott Bain)

After reminding us the youth of our industry, Scott made this very interesting statement:

"Software Development is both an intellectual and a practical profession."

I find this interesting because I tend to over-emphasis the intellectual part of the job when I digress about the nature of our profession. So, yes, there is also a practical dimension to it and Scott argued that testing is a driving force for this concretization.

So how do design patterns play a role in unit testing? On top of allowing us to use a simple name to communicate a lot of information and context, which is extremely valuable, did the GoF give us best practices for testing these patterns? Unfortunately not. But all hope is not lost, as Scott explained as he went on detailing applicable test strategies for the most prominent patterns (strategy, decorator, façade). This is a work in progress that can be contributed to on Net Objectives web site.

So what is the relationship with turtles? Because some patterns force you to know a lot about the chain of objects behind the scene (turtles all the way down) to be able to test them ; and that placing a mock at the right place can alleviate this issue. It is for this astutely located mock that Scott coined the term mock turtle (well in fact, he recycled the term).

@SD West 08: Highlights of Day Four

HTTP for Web Developers (Jason Hunter)

After his yesterday's talk about caching, Jason detailed the core of the HTTP magic. Indeed modern web development tools very often abstract HTTP out of the development paradigm. You end up with developers who talk about controls on forms as if they were developing Access applications (do not laugh, it happened to me).

HTTP is a wild world: with browsers and servers interpreting the standard in their own manner (often leading to bastardized additions to the standard itself), developers need to know what is happening under the hood.

I will not detail all what Jason talked about but I am always amazed by the amount of extra knowledge you can get when an expert revisits the basics! A highly recommended exercise for anyone who does things with the webernet.


Clean Code: Functions in Java (Robert C. Martin)

Woohoo! A new class from Uncle Bob! It is of course impossible to properly summarize an hour and half of such a magistral session. The main concept that this talk detailed is the following:

"Making code more readable allows us to write code faster, because we end-up reading code twenty times more than writing code when we are in the process of writing code."

So how to achieve this? The general idea is that a function should be an executive summary of what is happening below it: it must be short and should not mix concepts from all layers of the application, in order to remain understandable. This must lead to a recursive functional decomposition and the creation of functions with meaningful and descriptive names all the way down through the abstraction layers. And by the way: if you find hard to find a good name for a method, it is probably because it does too much things.

Here is short summary of the talk:

• Write small function, and if possible, even write smaller ones.
• In a method: do one thing to preserve cohesion, i.e. one level of abstraction.
• Three is the absolute maximum number of arguments: think about how two arguments are already confusing (right order?).
• No side effects: a method should have no strange temporal coupling, it should always have the same semantics whenever it is called.
• Throw exceptions instead of returning error codes because they get mixed with the natural outcome of the method and force ugly calling code.

Someone in the audience asked a question that is dear to me: "there is nothing new here, why keeping repeating it?" Uncle Bob replied: "because we do not believe it". This sounds like the classic "Shema Yisreal", with prophets and priests repeating the same things again and again because the people was unfaithful. We need to hear again and again to internalize things so we practice them. Writing code as if reading it matters is not natural but we can reach this goal thanks to method extraction refactoring.

Ouch. I will spend the rest of my life refactoring my own code.


Parallel or Perish!! - Are you Ready? (James Reinders)

After assuring us that multi-core are not a temporary trick to gain performance that will be deprecated later by faster single core processors, James made very clear the urgency and necessity of a mindset shift towards parallel programming, especially for client side developers.

He then gave us several tips that you can read in an article he wrote for DDJ a while ago.

I have really appreciated his remark on how to consider leveraging multicore parallelism in light of an increased work load, and not only by looking after accelerating existing code.

I was surprised by James remark about the lack of parallelism abstraction in Java: since version 1.5, the JDK offers a wealth a concurrency oriented high-level constructs (like collections, conditions, mutexes, futures and whatnot...). He might refer to the thread management themselves, but again I find that executors are offering an interesting way to splitting work around threads. But he confessed his main focus is on C/C++ though!

I am in fact more surprised by the number of Java developers who do not have clear (or at least basic) guidelines about how they write code that runs concurrently without coughing. As James said: we all have to learn and think about parallel development. Coming from Intel, this shows how much learning we can expect in the coming years!

In the meantime, thanks to either smart scheduling or pure randomness, this keynote was followed by two sessions on Java concurrency and parallelism: so I had an immediate opportunity to keep climbing the learning curve!


Thousands of Threads and Blocking I/O (Paul Tyma)

I was really looking forward this session and was not disappointed. The breadth and depth of the material Paul shared with the audience was worth attending. He did a great job debunking some myths about synchronous vs. asynchronous server models, all backed with hard facts. Here are a few of them:

• Java asynchronous NIO has higher throughput than Java IO (false)
• Thread context switching is expensive (false)
• Synchronization is expensive (false, usually)
• Thread per connection servers cannot scale (false)

Visit Paul's blog for more details and even the slides of the presentation. All in all, this talk confirmed my quasi-gutsy feeling about concurrent development: let the threads flow and be smart about their touch points!


Anti-Patterns in Software Projects: Human Factor (Rob Daigneau)

Writing code as a very mental process, hence is subjected to our human nature, with all its up and down sides. Rob gave a great presentation about how human factor affects software development and how can leaders of all sorts act to make the workplace a better place. This of course spawned a lot of lively discussions, as everybody has so much to say about what happens in their own life in software development.

It is hard to summarize all what Rob said but I really liked his emphasis on passion and how essential it is to let it burn in developers (without let them burning it, which is destructive). I also appreciate one of his final word: except if we write life-critical software, well, it does not really matter that much so better having fun while working. Sounds like a good advice to me.


Developer Bowl

Last year Developer Bowl was all about Googlers and their incredible supremacy in term of computer science knowledge. With Google being in no shortage of big brains, I walked in the theater with the expectation to get a fair deal of deja-vu...

This year was as entertaining as last year's edition. The questions, which were partially submitted by DDJ's readers, were much less oriented on the fundamentals of computer science and more on history and anecdotes. So this year Google did not pass the first round and IBM won versus Intel!

Oh well, it is all rigged anyway ;-)