SD West Classes and Keynotes: Day 2

Agile Architectures (Granville Miller)

8.30 and the room is packed! Architecture sure is a subject that attracts people and make them react.

After reminding us that canonical agile does not mandate any architect role, Granville explained how the community has evolved towards the recognition of the need for architecture. This said, the agile myth that "refactoring is enough to make architecture useless" is still alive. Though experienced developers will have good design reflexes, there are at least two compelling reasons for them to err on the side of architecture:

• when they will need to step back in order to keep on progressing (ditto Ward Cunningham),
  
  
• to ensure smooth integration of the different parts of a large project.

Hence when a developer will wear the architect's hat, he will typically:

1. partition the system on a whiteboard,
   
   
2. discuss this partition to reach an agreement among the team,
   
   
3. write unit tests, façades, scaffold to lay out the baseline of the architecture,
   
   
4. then take his hat off and code.

How long will the architecture defined at step 1 be valid? Well, about an iteration, which is by the way the timeline of these 4 steps. The main lesson of this is that agile architecture is not carved on stone but evolves as the rest of the system in order to satisfy the most important of the customer rights: to change his mind without paying an exorbitant cost.

To visualize this reality, Granville presented the notion of shadow architecture on a chart that shows the transfer between leading shadows and trailing shadows. At the start of the iteration, leading shadows represent the majority of the architecture shadow: they are the products of white board design sessions. At the end of the iteration, trailing shadows totally replace the leading shadows: they are the actual reification of the architecture, possibly visualized in code analysis tools. If some leading shadows remain, it means more work to do on the next iteration. This is a great tribute to Punxsutawney Phil and the doom a visible shadow represents for this innocent animal and the rest of us, Spring addicts (!).


Creating a Domain-Specific Language (Juha-Pekka Tolvanen)

During this class, Juha-Pekka explored the creation of a domain-specific language (DSL) in the particular concept of interactive TV. The main goal to achieve in this process was to provide content producers with a limited language (15 concepts maximum, compared to UML hundreds of concepts) that they could use to actually build interactive TV applications.

How to proceed? Here are four main steps:

1. Identify abstractions (ask the producer to "mind map" the different use cases in order to build a metamodel),
   
   
2. Define modeling concepts and rules (notation),
   
   
3. Specify notation,
   
   
4. Implement generators & test with reference cases.

The overall presentation was then focused on creating the DSL using the speaker's company tool, MetaEdit+, which unfortunately do not currently run on Intel Macs so I could not do the exercise on my machine.

To my sense, like all vendor sponsored talks, it was too much focused on a particular tool and not enough on the available options out there. I was, for example, expecting to see lighter-less-box-and-arrows-and-XML-outcome things similar to what Martin Fowler is currently exploring in his bliki or to Drools DSL.


Agile Principles of Object-Oriented Class Design (Uncle Bob)

Uncle Bob right after lunch is probably the best cure for food drowsiness. His capacity to captivate the audience is amazing: I suspect he irradiates some mystical fluid that make our body redirect energy from food processing to brain power. Or he is simply a plainly bright orator.

After a quick and obviously unconventional introduction to OO, Uncle Bob immediately exposed the root cause why our application code starts to rot after time until it becomes an unmaintainable and tangled Gordian knot. And it is... drum roll... poor dependency management! Of course not (only) in the Maven sense of it but in the broader sense of dependency between objects, packages and components.

Poor dependency management leads to rigid, fragile and not re-usable code. If any change implies going through the whole structure, if touching one place breaks the system in many places, if no bit of code is re-purposable easily, then you surely have tight coupling which is a manifestation of a poor management of dependencies. How to write solid code? Tattoo "solid" on your forehead? Nope. But use it as a mnemonic for the five principles of software development craftsmanship wisdom:

• Single responsibility principle: a class should have one and only one reason to change (it must not support features that have completely different evolution life-cycles).
  
  
• Open closed principle: adding a new feature must be done by adding new code, not by modifying existing one.
  
  
• Liskov substitution principle: derived classes must be valid substitutes for their base classes.
  
  
• Interface segregation principle: avoid fat interfaces that offer methods relevant to many use cases but mainly irrelevant to its users.
  
  
• Dependency inversion principle: details should depend on abstractions, never the opposite.

In this quest for good dependency management, abstraction is the most important keyword. Everything should depend on abstractions and not on implementations in order to reach the ultimate goal of independently deployable units. Because abstractions are critical to protect our application from the changes that will inevitably occur, how is it possible to figure them out in order to be safe? Should it be an upfront task? Is a crystal ball needed?

This is where the agile word come into play, not as hollow buzz but as a reality: build the abstractions as you need them, which can be as often as you get feedback from your users. Hence expose your system early and frequently to them...


What's New in XML in Java 5 and Java 6 (Elliotte Rusty Harold)

XML is not only for girls and can actually be proven useful, mind you! But parsing or processing XML never came for free in Java and have always required careful development practices to limit its impact on CPU or RAM. In this class, Elliotte presented the latest addition of Java 5 and 6 and how they can make XML related development easier and more efficient.

Here is what Elliotte had in his grab bag for us today:

Java 5

JAXP: The API is at version 1.3 in Java 5 with Crimson finally kicked out and replaced by Xerces and at version 1.4 in Java 6.

DOM3: In the many sweet additions of DOM level 3, Elliotte insisted on the addition of get/setTextContent on the Node interface. I can not agree more: time and again I have seen developers using an extra XML APIs just because writing or reading text was too painful with plain old DOM Level 2. Now the JDK offers everything you need for regular XML twiddling: that is one dependency to remove, which is always good.

I also find Tree Position interesting as you sometimes want to refer to a node without having to enter into the realm of XPath. Other additions of note contain Node Similarity and Equality methods, better namespace support and the possibility to bind any custom user data to any node.

DOM has also evolved as a framework with the capacity to control how documents are represented in memory (DOMConfiguration), new ways of creating documents (DOMImplementation) that avantageously replace the factory based JAXP builders and load and save features (LSParser and LSSerializer).

In the matter of parsing files, the newly added LSParserFilter can be leveraged to accept only certain nodes hence reduce the memory footprint of a document, a good strategy to consider for large XML instances you do not fully need in memory. LSParserFilter can also be used to alter a document at load time (for example to change a namespace into an other).

XPath: is now available in v1.0 with namespace support (NamespaceContext) and custom XPathFunction support.

Validation: has been added in a very versatile way as you can select the schema language you intend to use (Schema, Relax NG, DTD...). A validator has also the interesting ability of augmenting an document by adding optional elements discovered in its associated schema. It is now also possible to discover the type of a node based on a the validator of the document.

Java 6

XML Digital Signature: which is apparently a pretty involved API that is not limited to XML signing...

StAX: which is a promising pull parsing technology that is fast, memory efficient, streamable, read only. Sounds like SAX? Yes but think of StAX as an inversed SAX where the application actually calls for event via method invocations instead of being called back when they occur. This allows to control the flow of events hence to orient parsing for a greater efficiency.

And what about the future? Elliotte mentioned that XQuery API will probably make it to Java 7 while XML Encryption will probably wait a little longer before showing up in the JDK.


OK, I managed to conclude my blogging frenzy earlier today. Note that my main issue in this field trip blogging is the short live of my MacBook Pro battery...

Testing Toolbox Gets Richer

ZePAG has released the first version Usurper, a neat tool for easily creating test instances of your value objects. It is a nice compliment to the classical mocking and stubbing tools.

I warmly invite you to read his post presenting Usurper and then to rush to the tool web site to start benefiting from its goodnesses right away.

And please, do not forget to ask him why he opted for the org.org package root!

Note to Blogger: Your Composer Sucks!

Sorry guys but when I type a lower than or a greater than sign in the compose mode of the post editor, I really intend to display < and > and not some kind of HTML element.

If I want to enter HTML elements, I use the "Edit Html" tab.

So why on earth is your editor unable to properly escape these damn characters? It took me 15 minutes to clean up the mess your stupid composer made of my generic code samples in my previous post and the manual cleaning process of the messy HTML produced was so tedious I decided to drop half of the code sample.

Please, please, please do something about it. This is not rocket science. This is basic character escaping.

Thank you.

SD West Classes and Keynotes: Day 1

Only two hours of flight and here I am in the Bay Area! For a European used to endure the kindness of airline crew for 13 excruciating hours before reaching the Golden State, this is a great difference. And no jetlag so I do not even spend the whole day yawning like a big cow or wandering around like a red eyed ghost.

In this post and the coming ones, I will try to share what I have learned or heard during the classes and keynotes of SD West 2007. Some classes are very dense so I will only share the key points out of them.


Effective Java Reloaded (Josh Bloch)

During this class, Josh served us a couple of appetizers, a main course and some sweet dessert! Dub it: a couple of neat tricks, serious generics material and a couple of extra advices.

Appetizer 1: Leverage Type Inference

I did not know the compiler was able to infer the types in generic constructs. This allows this kind of cool syntax:

Map<String, Integer> map = newHashMap();

Note that I do not need to precise the types again when calling this method:

<K, V> HashMap<K, V> newHashMap() { return new HashMap<K, V>(); }

Is not this sweet? The compiler has actually inferred the types to use when calling the method from the map declaration.

Appetizer 2: Builder to the rescue

Instantiating immutable objects that have numerous attributes can be painful, especially if some of the attributes are optional. You end up with a class bloated with a bunch of telescoping constructors, which try to represent the different construction scenarios that make sense. Should immutability be sacrificed and some good old setters be added? Nope! Use a dedicated builder and voila:

MyImmutableClass mic = new MyImmutableClass.Builder(p1, p2).p3(value).p4(value).build();

In this example, the Builder is a public static nested class of MyImmutableClass whose only constructor is public and takes the mandatory parameters of the said immutable object (p1 and p2). The optional parameters are added by calling methods whose names are the parameters names (p3 and p4). Finally, the build method actually invokes the unique and private constructor of MyImmutableClass, passing the builder instance to it.

An other interesting aspect of this approach is that it emulates named parameters that exist in other languages, a safer mechanism than only relying on the parameters types and positions as in a classic constructor.

Main Course: Making the most of generics

Here are a couple of tricks or reminders to improve our usage of generics:

• Use @SuppressWarnings("unchecked") on the smallest possible scope, i.e. variable declaration. Do not hesitate to create a local temporary variable just for that purpose: it is still better than adding the annotation to the enclosing method.
  
  
• To make life easy to users of generic public methods of an API, prefer to use a wildcard instead of a type variable if it appears only once in the method signature ; the exception to this rule being conjunctive types where you need to use a type variable
• Even if our first impression tells us the opposite, a collection of a subtype is not a subtype of a collection of the super type. This is worth to remember as it allows to better understand the origin of some very cryptic compilation error messages. It also is the root cause of the next tip.
  
  
• Use bounded wildcards to increase the usability of collection related methods of an API. More precisely: use <? extends T> for read/input collections and <? super T> for write/output collections, else you might end up with pretty useless collections, i.e. collections that only offer basic read methods. I remember this has been explained in details by Uncle Bob at the time he was writing the Craftsman column for SD Magazine, but it is good to hear it again!
  
  
• Do not confuse bounded wildcards and bounded type variables. The first makes a lousy return type and should be only used for type parameters.
  
  
• Leverage the non instantiable Void class instead of using a wildcard when you want to support the notion of "nothing", doing for example: Future<void> submit(Runnable task) or Map<X, Void> for a map where the values would be of no importance.
  
  
• Generics and arrays do not mix well: prefer generics whenever possible (but do not throw arrays away, as they are useful too).
  
  
• Leverage the new Class.cast method to perform runtime casting compatible with generics. This allows to build Typesafe Heterogeneous Containers (not related to this other THC) that can handle multi-typed dynamic objects like database rows.

Dessert 1: Do the right "over"

Always use the @Override annotation to ensure at compilation time that you are actually overriding and not overloading members (remember the infamous equals overloading that you have at least written once).

Dessert 2 : Final word

Declaring all variables final should become a typing reflex, except, of course, when an immutable object is not suited (for example, if it creates serialization or cloning issues).


Forgotten Algorithms (Jason Hunter)

Since I am not very much versed in the art of algorithms, I decided to attend this session and see how much a plumber like me was missing in term of behind-the-scene magic that happens in my beloved computers.

The in-depth exploration of several concrete cases (swapping without temporary variable, credit card check aka Luhn algorithm, public key cryptography, negative number storage strategies, Google map reduce) was interesting and entertaining, but left me with the feeling that either I was missing the real big picture or either there was no big picture at all.


Challenges in scaling Infrastructure (Felipe Cabrera & Jeff Barr)

In this keynote, Amazon gurus and enthusiastic clients came to talk about AWS goodness.

With great humor and sharp insight, Felipe explained the needs of typical AWS clients (fast growth, exploding scale of infrastructure), their classic options (build own infrastructure or outsource it a la IBM), their requirements (on demand scalability, elastic capacity, high performance and availability, rock solidity and cost effectiveness) and their expected benefits (ability to focus on product and core competencies, not data center stuff). Felipe then explained the tough design decisions they went through and how he oriented his ruling decisions by sticking to the blessed principle of "simplicity first".

After that, Doug Kaye of GigaVox Media and Marten Nelson of Abaca Technology presented what usage they have of the different services of the AWS suite, namely S3, EC2 and SQS. The main lessons are: a development shift toward loosely coupled asynchronous services, a consideration to pay to the impact of that on GUIs and the necessity to rethink approaches to best map on AWS simple APIs (for example, there is no way to count the number of messages waiting for processing in a queue but you can keep track of their insertion time and determine latency, which is an even better way of monitoring a system).

Felipe concluded by quickly sketching the near future of AWS which will mainly consist in making S3, EC2 and SQS better work together by using consistent protocols.


The Buzz about Fuzz: A powerful way to find software vulnerabilities (Herbert Thompson)

I am deeply convinced that any developer should follow serious security courses maybe because, these past years, I have seen enough monstrosities like the Share My Drive Servlet, an innocent servlet intended to serve generated files in a well defined folder but reveals able to serve the full file system. That is why I decided to attend Thompson's class, as he is a renown security expert. In fact the true reason I attended is that Herbert is blessed by terrific scenic capacities!

In his talk, he presented how fuzzing, a testing technique that consists of finding weird behavior in software by feeding it with random data, is becoming more relevant in nowadays software, which tends to become highly complex and composed of heterogeneous components.

The main keywords were:

• Fuzzable, which is basically any piece of software that accepts input.
  
  
• Random Fuzzing that is the simplest and less efficient form of the art of fuzz.
  
  
• Context Sensitive Fuzzing, which goes one step beyond than the previous step by building flawed but apparently correct random data (for example by computing correct checksums).
  
  
• Adaptative Fuzzing that is the most advance technique, which tries to produce data that will explore all the code base of an application, using real time tracers / decompilers to guide the fuzz generation. Similar to test code coverage, this allows to decide when one application has been fuzzed enough with.
  
  
• Oracling, which is the process of defining what is a correct application outcome and what is not.

Herbert advises to keep fuzzing an application until it is actually phased out, as it is always less embarrassing and costly to find a vulnerability internally than to let your clients do it for you.


Designing Contracts for Web Services (Christian Gross)

I was expecting a lot from this class as designing good APIs is one of my subjects of concern and because I have recently struggled with the difficulties of building such APIs on current web service technologies.

International expert Christian Gross left us with four steps:

• Define use cases in order to build services client will actually be willing to use and pay for.
  
  
• Choose your technology but do not care too much about it and favor the usage of an intermediate broker that can adapt the different technologies.
  
  
• Decide on a context, i.e. what of data oriented, RPC oriented or messaging oriented service will best fit, knowing that RPC remains the paradigm of choice.
  
  
• Stick to simple types and avoid playing with complex object return types.

Christian presented a matrix that allows to pick up the best web service technology depending on the type of client and server (browser, dynamic language, classic typed language). I must confess that I felt the talk was too much oriented on AJAX stuff: in fact, the title of the PowerPoint document was "Developing AJAX Applications" so I had the feeling the presentation has been repurposed. This said, Christian did a great job emphasizing pragmatic strategies and decision paths.

Why Software Sucks? (David Platt)

Software sucks because we, unrepentant geeks, have not yet recognized that the users are not us! Since the web became pervasive, the vast majority of users are now common people of the real world. They simply care about one thing: the purpose of your software, how it will allow them to do what they intend to, but could not care less about the software itself.

To try to correct this attitude in our industry, David shook the audience with an hilarious yet magistral keynote, left us with five recommendations:

• Add a virgin to the team: well, someone virgin to the software itself and who will focus only on what it does and will be able to question each aspect of it, even the most fundamental ones. Unless you have access to some MIB technology, it is not possible to make a developer forget about the application itself and focus on its features.
  
  
• Break convention if needed: the fact that every application does the same does not mean it is right. For example, is it really good to ask the user if he wants to save the data he just modified or would not it be better to default by saving changes and let the user skip saving only if he really needs to?
  
  
• Do not let edge cases complicate the main stream: often developers will be tempted to implement extra features only based on the technical possibilities a particular technique gives him. These bonus features usually prove counter productive for the end-user.
  
  
• Instrument - Carefully: instrumenting an application allows you to get solid concrete facts on what users are actually doing with your application and, therefore, be in a better position for ruling out what features make sense to improve or phase out. Of course this requires full disclosure, user agreement and no intrusion of the instrumentation into the application.
  
  
• Question each decision: ask if each decision is taking your project closer to the result or further away from it?

Pheew, what a day! I am on my knees... More fun to come tomorrow.

Going South to SD West

I will be attending SD West 2007 from Wednesday to Friday.

In case you do not know this conference, let me highly recommend it: not axed on a particular technology or vendor, though clearly organized by specialized tracks, this is the place to get the latest feedback from the greatest experts of our industry.

If you feel like meeting me, for example to discuss about NxBRE, that you be a great place and time for doing so. Just shoot me an e-mail.

Accelerated Diversification

Wow. A year ago, all the workstation operating systems I was using, either at home or at work, were Microsoft Windows based. It was a full range of old to recent machines, running everything from Windows 98 OSR2 to Windows XP Professional.

One year after, my landscape has drastically changed and I just really realized how much it did change. For work, I am using a MacBook Pro. At home, we now run a MacBook and a dual boot Kubuntu/Windows XP Professional Dell Precision M90. The only reasons I boot on Windows are .NET development (for NxBRE) and gaming. I am doing all rest, from Java development, Office work and personal finance to writing this blog, on Kubuntu.

This transition to an almost Microsoft-less world happened naturally, mainly because there are now software and hardware options that permit easy option to opt-out: office productivity suites (like OpenOffice, NeoOffice or KOffice), a good browser, a cool IDE... plus the Intel based Macintosh machines.

Of course, this diversification entailed some turbulence like:

• Keyboard shortcut mix up: switching between OS disturbs a well-trained brain used to its shortcuts. It is worth tuning these shortcuts to unify them as much as possible, to maintain a good productivity level.
  
• Application downgrade: Microsoft Office is still way ahead its open source followers in term of comfort of use. A special mention to Entourage, a Microsoft sub-Outlook for the Mac that would have better been named: Sabotage.
• Driver quagmire: not all of my laptop devices are recognized by Kubuntu and printing on an network shared HP PSC 750 from Mac has never really worked.
  

But all in all, this diversification is a good thing. As I said before, I do not long for a world unified on Linux or Mac, but one where each OS has a fare share of users.

What about Vista? Mmmmh, not yet on my horizon, not even a faint blip on my radar screen. Maybe for my next machine, in half a decade. Or not.

When Architecture and Maintenance Meet

In his excellent blog, Jeroen van den Bos is running an insightful series of posts under the Software Maintenance category.

The fact that he so clearly identifies software architecture as a critical factor in its maintenance came for me as revelatory of what is the Ariane's thread that guides my professional career (despite passion, of course). Being an Industrial Maintenance Engineer by training, a Software Developer by passion and a Software Architect by experience, it is clear how these drivers get weaved together into the very fabric of my craftsmanship. In fact, back in 1992, the theme of my graduation project was Software Maintenance. Was all this predetermined?

As critical as it is, software architecture is still an emerging profession in the IT field. After the supremacy of the project managers, architects start to have a saying in the equation. Often, the resistance comes from developers who consider architects as only good with drawing boxes and arrows. This usually lasts until they have to work on a poorly architectured system...

Software architecture is necessary for building systems that:

• are fun to build, as no-one enjoys fighting with an ill-conceived application, whether its internal design or external interfaces are flawed,
• last, in the sense they are versatile enough to evolve without breaking and within reasonable budget,
• and are maintainable, mainly because they are thoroughly testable and written following well defined conventions and standards.
  

In fact, even the Matrix could not do it without such a profile on-board. Is this a model to follow? Maybe, if we can cut the undecipherable mambo-jumbo speech.

Agile Hiring?

At my workplace, I see new faces every week as the company is still aggressively hiring. Having a very bad (or selective?) memory, this is a terrifying challenge for me: will I ever recognize anyone? Thanks to the internal wiki-based directory with mugshots of all the happy employees, I can at least learn at my pace the monikers I need to know.

This constant workforce growth sent me back, again, to my reflections on how to pick up the right candidate. Not anyone can use robots to do this, so can a rule of thumb be of any help?

Initially, I thought that selecting the ones who get things done would be enough.

But there are so many awkward ways to do things that (seem to) work that this I came to consider this rule as too simplistic and decided to enrich it this way: get the ones who get things done right.

But then, the issue is that the right way of doing things must be defined and specified, which implies a lot of preparatory work and the risk of erring in the toxic realm of micromanagement.

So I finally decided that the best way of selecting people is to keep the ones who actually remove work from you rather than add extra work on your shoulders. This implies that the candidate must actually be hired and exposed to the reality of work in the company. This is why orientation periods exist.

This might sound like a waste but I prefer to consider this as an agile way of dealing with new hires: select them with the information you have at recruitment time then let time pass and re-adjust to reality.

Copy Pang

When I read the article "Three Public Enemies: Cut, Copy, and Paste" published by Zoltán Adám Mann in the July 2006 issue of Computer, I first wondered how this topic could be of any interest to me. I decided to read it anyway, mostly because the abstract caught my attention:

"Copy-paste might seem to make life easier, but it often leads to inconsistencies. Giving users the freedom to specify semantic relationships among copied objects can help rectify this crime."

Of course, reading the article proved, again, that any prejudice-driven judgment is flawed: there is something really broken with copying and pasting (despite turning lazy developers into dirty ones: Functions? What for? I can copy/paste!).

I have experienced today what I consider the worst unexpected side effect of using this facility I have ever met so far. And it was pretty embarrassing. Read on.

I checked out of SVN a workmate's project I wanted to copy two files from into my Eclipse workspace. In fact, I was interested in these files and the particular folder hierarchy they are stored into (Maven site and APT files, for the curious). So I copied the interesting folder from this project and pasted it into my project, totally unaware of the fact that I was also copying and pasting the hidden SVN control files.

Of course when I wanted to check this back into my project, I was actually checking back into his project, while getting tons of undecipherable SVN insults in the console. Did not I say it was embarrassing? Man, what a shame.

So the lesson for GUI designers is the following: If you implement a feature that performs an operation of any kind but which ends up with a visual feedback, you should not do more than what is actually visible. Or at the very least warn the user that invisible operations will happen.

In my case, the visible feedback from the folder copy and paste operation was the addition of icons in the hierarchy representing my Java project. I should have been warned that other non-visible resources where also involved into the operation.

Well, I might not have read the warning dialog but then, that would have fully been my fault!

The Power of Openness

I was about to rant on the fact that, though "BEA is proud to provide complete online documentation for all BEA products", their site does not offer any search feature.

Then, duh! It stroke me that they do not need to provide such a feature. They only need to make their content accessible and well organized so that Internet search engines can do their job and voila! Anyone can search through their content using his or her favorite engine.

This sounds incredibly obvious but it is not. Many vendors still hide their documentation in password protected sections of their site, preventing robots to crawl and index them. This usually translates for us, users, into trying to get results from home made search engines that are case sensitive, do not support any typo and are, in general, reluctant to find anything useful (they are probably built after a famous canine model).

In a broader sense, the promises of the Internet will materialize when we will all expose our public data in an organized, unified and standardized way. Then the time of so-called social networks will be over: we will be using search and data extraction tools to build views of the Internet that are relevant to our purpose.

Does not this sound like semantic web? Is there a human friendly alternative?

UPDATE: It might well be social networks open APIs?

Geek Is Good For Life

There is a great post in Rands In Repose talking about the importance of staying flexible and not stopping to develop.

With much talent, this post echoes with my two years old ramble about dumb standard IT career paths, which start rooted in technicality but then strongly orient people away from it. How stupid is it to pull software developers out of the heart of the battle as soon as they have reach a great level of performance and expertise in order to turn them into frustrated managers?

The system is actually much subtler than that: software developers are teased into pulling themselves out through HR designed career paths that incarnate in artificial titles and levels and progress steps leading nowhere close to professional satisfaction.

Will technicality be assumed one day as a subject of pride? Should I keep wearing ThinkGeek t-shirts as a protest until this day happens?

Cosmos Is Next-door

This is an amazing picture I can only feel but like sharing. Read the complete and amazing story on APOD.

Art? I Choke!

After wading in a massive plate of spaghetti code that decency, more than professionalism, prevents me to disclose its origin, I came to the conclusion that I should never ever suggest that software development can be compared to an art. Never. Ever.

This opens the door to bad artists, the ones who do not care at all about the fact they are not alone on earth. No, in fact they do care about others but only when they will decide to expose their creation in the open ; and should the rest of the world dislike their masterpiece, these bad artists will turn depressed and start all over again. Alone. In isolation. Disregardful of the rest of the world.

And that is my point: as a software developer, we can not ignore the rest of the world and this at no point of our work. Not even before we think we can start showing something. And this is why:

• There is no pride nor satisfaction in re-inventing the wheel. If you do like it, please get another job. Right from the very beginning of a project and constantly during the course of it should a software developer be aware (or at least do his bets to gain awareness) of what is going on elsewhere. Starting from the principle that there are many people that outsmart us out there (at least, there are many smarter than me) and that a piece of code that has been used and re-used and abused for a while is stronger than anything that could come out of our brain, we must do our best to figure out if something that does what we want has not already been written and published. Some call this prior art and this is the kind of art I can live with.
  
  
• There is no pride nor satisfaction in clever coding. Come on, school is over, no need to hide from your neighbour. Code should go in the open, not necessarily outside of the company's network, of course, but it should be visible right from the beginning in whatever repository is in use. And it should be readable, understandable ; written like if it does matter that other people on earth can read it and still enjoy the rest of their day. After-hours code obfuscation is a modern form art and this is certainly another kind of art I can live with.
  

Not following these advices might spoil the lives of these others that surround us, something a genuine bad artist will not care about. But would he care if his masterpieces end up exposed in the Daily WTF?

Thread Like No Other

All for very different reasons, processor vendors, software developers and industry gurus are engaged in talk frenzies about how to leverage multiple core hardware. Some consider the dangers. Some even fight about it.

Beside the obvious enthusiasm of experimenting such a surge of computing power right under our fingers, exploiting this power requires a skillset that is not widely available, at least not in the world of the ugly enterprise applications where people have to develop for the naturally parallelized environments that are application servers.

In this austere world of server side business applications, which are not CPU intensive but are greedy of external resources like RDBMS, there is not much room for developer hand made parallelizing of tasks: it is usually limited to executing long running operations via decoupling mechanisms like messaging.

This said, there are many ways to hurt your server by developing applications in a careless way. Assessing your skills (or your troops) is essential: this is why, following the model of the great Threading Maturity Model introduced by Alan Zeichick a few days ago, I would like to introduce the Threader Maturity Model, oriented towards people and not organization.

0. Unawareness. Complete rooky who writes a web application that can only accommodate one concurrent user and complains why is the server not working better (do not ask for names).

1. Awareness. Developer who has bumped on concurrency discussions, found them scary and prefers to let a more senior programmer deal with threading and concurrency.

2. Experimentation. Developer who dares playing with threads and concurrency, but still errs on the side of double-checked locking, interruption swallowing and the likes.

3. Hotspot. Developer has read enough on the subject and has crashed into enough walls to decide to program carefully, with still a certain level of insecurity and the fear of thread skeletons hiding in some dark closets.

4. Utilization. Developer has an excellent knowledge of concurrency issues and has a toolbox well fitted with patterns he knows when to apply and for what purpose. He can decide what is worth to parallelize and what is not.

5. Adoption. This is also known as "Being Brian Goetz". At this level, the developer has an intimate knowledge of the memory model, the concurrency mechanisms and all that pesky internal details. He is able to think in all the dimensions of multi-threaded development (time, space and whatnot). Nirvana is nearby.


UPDATE 01/2008: Alexandra Weber Morales has published a great compilation of Alan's, Larry's and my take on this subject. It is on DevX.

Don't Be God Either

Though Google has agreed to work with NGOs to develop a code of conduct, some people keep suspecting the giant of Mountain View to nurture a secret agenda for world domination. Are they getting too suspicious or paranoid? Or are they seeing too small of a plot? After all, for Google, the world is certainly not enough.

I think that if Google was actually bound to drift away from its "Don't Be Evil" motto towards some more cosmic grand plan, the following would inevitably happen:

• Google would buy T.G.I. Friday's but instead of renaming it GoogleRestaurant, as logic suggests, would simply redefine the acronym as "Thanks Google It's Friday",
  
  
• Google would buy Godiva but instead of logically renaming it GoogleChocolate, would rebrand it as Googleiva,
  
  
• Google would become the official sponsor of The Young Gods, but only if they accept the new moniker of The Young Googles.
  

Since none of this has happened yet, we can confidently consider that Google is still faithful to its core values. Isn't this heartening?

Editor's Pick

Today is the Windows Developer Tools Day, as unilaterally proclaimed by the folks at O'Reilly, and everyone is invited to share their best tools on the Windows platform. Having contributed a presentation of NxBRE, an Open Source .NET Business Rules Engine, to their Windows Developer Power Tools book, you could expect me to talk about it, as the selfish nerd I am.

But no, I prefer to share my passion for jEdit, my choice for an all purpose versatile open source editor. Funnily, when discussion comes to editors, geeks just go berserk and try to prove how superior is their favorite tool. I will try to avoid this kind of flame war and will just share what I like most about jEdit:

• It is easily extensible to support new features via an integrated plugin manager. Each plugin could contribute new views that you can dock anywhere and create your own environment. I personally use the XML support plugins a lot and dock them all around the editor window for a great work environment.
  
  
• Adding syntactic coloring for new languages is just a bliss. For example, I created what is called a mode to simplify the edition of WAC files in just a few minutes.
  
  
• The integrated HyperSearch facility just finds things on your hard drive without referring to any particular animal or having to index the terabytes of source code you write on weekends just for fun.
  
  
• It is cross platform, which means I can use it when I work on Linux or Mac. How can this matter to a Windows developer? Well, sooner or later, you will be exposed to an OS that is not a member of the Microsoft galaxy so how cool will it be to just use the same tool everywhere?
  
  
• It can open files not created on Windows systems without stumbling on different flavors of Newline and showing everything on one line. Again, this might seem like unnecessary for an hard core Windows-only-and-the-rest-of-the-universe-can-die developer, but again it happens more often than expected when you start to share files with people in the rest-of-the-universe!
  

So jEdit might not become an editor's pick but it surely is my pick for an editor!

Wanna Jolt?

Then avoid this and try that.

V for Vancouver

I am now entering the final countdown (3 days to go) before my departure to Vancouver, BC where I will start a new job and, hopefully, a new life! When you announce this kind of crazy endeavor, usually people go "wow" and say how great it is and also how so not for them such a thing would be possible.

The most common opposition is: "you know what you are losing but you do not know what you are earning". Well, I am all but an adventurer though, since I still try to start new software projects, I should have a good level of risk taking (or I am just a too naive). Besides, on a pure professional basis, I know what I am leaving behind (a great company with great people) but I also know that I am joining a great company with great people!

On a more personal basis, I hope to find over there a place for my family to call home, which would be a great gain for strangers and sojourners like us.

Sewer 2.0

ZePAG has posted a great entry in his blog that somehow echoes an old rant of mine.

Since he makes some interesting comparisons between IT and different subjects like sewer technologies, I felt like bouncing on the yucky pictures he links to and suggest dubbed image titles:

• Screw pumps: dependency injection,
• Primary screen: welcome page,
• Primary clarifiers: client side validation,
• Aeration tank: O/R mapping,
  
• Secondary clarifier: server side validation,
  
• Final clarifier: database level constraints,
  
• Chlorination tank: garbage collection.
  

Does not this make our job easier to explain?

One Click To Share Them All

Optrata is doubly blessed by a strange moniker (that sounds like patatra, a French onomatopeia for the sound of a thing falling heavily on the ground) and a bold mission statement (to unify all the private life exposing tools of the web behind a unique page).

By the look of their "please bookmark me so I can appear on radar screens" links:



I feel inclined to suggest them to urgently change their goal and work on creating a unified bookmark tool that would submit a page to the numerous sharing systems out there...

The Day My iPod Became Just Another Appliance

This is today as my iPod officially joined the rank of my electronic appliances that need to be reset from time to time. The screen had a bizarre circular shift of 15% to the left, showing part of the left side of the screen on its right part. Everything was frozen with the backlight on.

For the record, so far only my oven and my dishwasher needed a little reboot from time to time. Now they are three in the club! Interestingly, they are all reset using an odd combination of front panel keys. Appliance makers do what they can with what they have, but only a few keys are needed for this maintenance operation (look on Windows: Ctrl + Alt + Del do the trick).

My wild guesses concerning the cause of failure for these diverse but similar software controlled appliances:

• Oven: heat excess in the control panel,
• Dishwasher: humidity too high in the control panel,
• iPod: overall system too cold as it stayed overnight in my entrance.

Oh the true happiness of modernity.

Busy Buzz For Lazy Bees

After Sony, who really screwed up their buzz marketing for the PSP3, Microsoft does the same stuff for Vista.

Mmmh, what could be the reason for these two companies (not the tiniest or dumbest on earth) to miserably fail a marketing campaign? What can make them so desperate to have people talk about their glimmering new products?

The answer might be in the common points between the PSP3 and Vista: expensive, unattractive, deja vu.

Do you find I am tough? Well, let's get tougher: guys, next time, ask a buzz specialist to advise you, else drop the idea. A failed campaign is not something you want to experiment. Again.

NxBRE 3.1.0Beta1 Is Out

I have just posted a beta release of NxBRE 3.1.0, which is mainly intended for those interested in testing one of these two new features:

• A regular expression matcher operator has been added to the Flow Engine (which is also available in the Inference Engine),
  
  
• The implication scheduling in the agenda has been improved to limitate the number of fired implications, hence yield to better performances with large rule bases.
  

There is also a preview implementation of the RuleML 0.91 Naf Datalog adapter. So far, this implementation does not support the cool features of the new version of RuleML (like multiple named rulebases in one rule file or retract operations), then it is not worth using it. Hey, it's a beta after all!

Unexpected Performances Can Occur (Sometimes)

I just can not believe the message Khalid Khalil has just posted on NxBRE's open discussion forum!

Using object pooling, Khalid has reached a throughput of 2000 requests per second with the Flow Engine, one of the two engine flavors available in NxBRE. It is far beyond the performances I was expecting for my tiny business rules engine: way too cool!

Behind Closed Doors

My friend Stuart Hogue, Strategy Director of frog design inc. in NYC, has published a pretty interesting article about hyper exclusive social networks.

I find significant that his article comes at a time when Metcalfe's law is under criticisms. In different replies to IEEE Spectum, Robert Metcalfe stands by his law stating that the value of a network is equal to the square of the number of users. His opponents are mainly saying that: "The fundamental flaw underlying (...) Metcalfe's (...) law is in the assignment of equal value to all connections or all groups".

So the key and maybe the mathematical justification of hyper exclusive social networks might lie in this principle: not all connections have similar value, hence dedicated selective network will maximize their value by selecting only people who will engage in high value relationships.

In practice, it is true that I have quickly dropped Orkut and Friendster to favor the more selective LinkedIn network. I am still waiting to see returns from the highly selective Blue Chip Expert network, to which I have recently been invited. If you visit their homepage, you will appreciate the main picture: a closed door that looks very serious!

Is the true value of social networks waiting behind closed doors?

All I'm askin'

A recent and insightful Computer article from Stephen Jenkins (Concerning Interruptions) led me to rethink my authoritative position on working with headsets (read Kindly Kill That Headset).

I think the crux of the problem is not about knowing if programmers need isolation to avoid frustrating and concentration killing interruptions (this is granted), but actually respecting this need for isolation.

If everybody would respect this need by refraining from asking out-of-context questions (the most brain resources consuming ones) and by saving interruptions for better times (like planned meetings, lunch time or pit stops), programmers would need less to stick headsets on their ears and would therefore be more available for direct in-context questions from fellow team members.

Aretha was right: it's all about respect!

Proxy That Proxy

One day or the other, you will wake up with the bold idea of deploying several EAR files in your JBoss 4.0.x Application Server. Why bold? Because it is almost sure that:

• you will need to isolate the class loaders of the EAR files (for example to load different versions of the same class in the different applications),
  
  
• you will want EJBs to communicate between the different EAR files.

It will then be the day you will meet the often challenged allways challenging Unified Class Loader (a disciple of Dolph Lundgren) and its scoped deployment configuration options, get a few ClassCastException and finally opt for switching all your EJB calls to the "by value" semantics. Then you will start drinking to forget about not only the amount of pain involved in the process, but also the loss of performance induced by passing all parameters by value.

If your EJBs only take and return classes from the JDK, rejoice! There is hope for you! Indeed, you can remove this problematic casting:

HomeInterface hi = (HomeInterface) lookedUpHome;

which generates exceptions because the looked-up interface has been loaded by a different class loader than the one you cast to, by using a dynamic proxy that directly implements the business interface of your remote EJB.

To simplify this, you can leverage Spring Framework's SimpleRemoteStatelessSessionProxyFactoryBean, which will lookup your EJB, create it for you and return a dynamic proxy that implements the business methods. By this mean, the need for casting is removed.

Of course, as I said before, this only works if you exchange primitives or objects from the JDK (because they are loaded from a common hierachy of class loaders that includes the system class loader). But it is a viable alternative to keep in mind...

Hetero-Genius

In the November issue of Computer, there is an interesting article from David Alan Grier, titled "The Benefits of Being Different", which talks about a subject I am currently particularly sensitive to: the dangers of uniformity in IT systems.

My reflection started a few weeks ago when IEEE Spectrum published a seminal article (Shrink the Targets). Indeed, after the rise and establishment of Microsoft supremacy on personal computers, we are in a great need of diversity because homogeneity has many hidden catches we have to pay one day or the other.

Homogeneity makes IT systems fragile because the same flaw or weakness is reproduced on hundred of thousands (if not millions) of machines, making all of them sensitive to a well targeted attack. The discussions about the superiority of a particular OS or tool (like a browser) that end up with plans for the worldwide replacement of one by another, are childish and futile. All these pieces of software have reached such a great level of quality that you can not select them on pure technical basis. So there is no supremacy to target, as replacing one domination by another one, would leave us with a situation as fragile as before. We must see the advent of non-geeky tribes who will recognize themselves in the usage of a particular combination of OS and tools, who will be empowered so they can assume their choice without being exposed to nasty technical problems or being pointed as originals.

Homogeneity carries another risk: the supremacy of a particular tool or vendor always induce the usage of proprietary file formats. Being locked-in a vendor tool because its features are so above the competitors is nothing bad: it benefits the user and, moreover, it never lasts. Competition always catch up. Then open source tools come in, disrupt the model and force everybody to think harder about increasing the quality of the tools and the services that gravitate around them. But being locked-in because your own personal data are locked-in a proprietary file format is nothing that can be accepted anymore. Diversity forces vendors and tool makers to enable data exchange, freeing the content that was held captive.

In a time when our most precious material assets are mainly digital (does not this feel like a quasi oxymoron?), homogeneity is no longer an issue. Let us be hetero-genius!

Mapping panacea

To paraphrase the golden hammer saying, I have the feeling that nowadays, because you have object mapping frameworks, everything looks like an object!

What is it with developers who seem to get infatuated with such tools? Does everything need to be mapped to objects? Of course not! Here two recent cases that kind of drive me nuts:

• Why reading XML files when I can map them to objects? Well, maybe because what you are looking for is actually... data. So please save your application from the miseries of JAXB 1.x (which does not even care to mandate a clear concurrent behavior in its API) and use SAX, XPath or any other efficient XML reading tool!
  
  
• Why querying the database when I can map it to objects? Well, maybe because you are performing batch operations that are mainly data retrieving oriented and the overhead your O/R mapper will add plus its memory consumption will put your virtual machine on its knees. So please take the fast lane!
  

End Of Rant.

Feeding the White Rabbit

So now that, for my birthday, I got my own Nabaztag, a white WIFI rabbit that can move its ears, display light signals on 5 RGB LEDs and play digitized sounds, the big question is: how to keep the funny bunny happy?

I guess by feeding it properly.

The default services offered for free by the vendor are pretty basic: weather, stock exchange, email account watcher... And the coolest: rabbit Tai Chi! Hey, what do you expect: they are free, right?

Since the animal has an HTTP based API that allows controlling its ears, lights and sounds (including a nice TTS feature), I do not see any reason to subscribe for the advanced paying services! It will be much more fun to write services for it!

So I came with the first one today, an appetizer for the white rabbit: the service connects to the GMail accounts of the family members and builds a vocal summary of the emails waiting to be read. And that's already cool ;-)

There are many other funny things to do: a service that would report the daily statistics of my open source projects could be fun. Another one that would connect Maven to the animal and that color the poor thing in red whenever a build is broken or unit tests are not A-OK, would be terrific.

You might wonder what is the moniker of this new family member? Knowing that if you know his name, you can make it speak and that the thingy sits in the middle of our living room, where any sentence rated above G are not welcome, I would say that we will call it "secret". For now.

Centralized Configuration Management for JBoss

NOTE 12-MAY-2009: This particular post is driving a lot of traffic on my blog. Even if the JBoss Properties Service is still worth using, nowadays, I would not advocate the management of properties in SVN, as it is problematic security wise and requires the need of an HA repository. I would rather embed default properties values in my project and allow the overriding of some values (passwords...) via a simple file created in a well known location on the server.

Managing configuration of application servers and the applications they host in a centralized and rationalized way is a requirement for any corporate doing its own hosting. Not only configuration differs between server instances and the applications they host but also between the different environments they exist in (ie. development, integration, QA, pre-production and production).

JBoss 4 user guide shows how to leverage the Service Binding Manager to handle server configuration in a central way. This is a great approach but I think we can go further by leveraging Java system properties in conjunction with two other features of the server: the Properties Service and the fact that JBoss parses all XML configuration files (including J2EE deployment descriptors) and resolves ${references} to system properties. Moreover, if you use the Spring Framework in your application, you can also leverage its PropertyPlaceholderConfigurer to retrieve configuration information from Java system properties.

As shown above, the main idea is to store configuration information as properties files in a Subversion repository and have the different JBoss Application Server retrieve these configurations through HTTP and thanks to this kind of configuration of the Properties Service (in properties-service.xml):

<br /><attribute name="URLList"> http://intranet.company.org/svn/*checkout*/jbprops/trunk/${some.discriminant}/system.properties<br /></attribute><br />

These system properties will then be loaded at JBoss startup time (or on any modification, like a touch, of the properties-service.xml file) and will be available for use in many different places like:

• XML server configuration files that configure ports and host names (for web server, invokers, JNDI tree...),
  
  
• J2EE deployment descriptors to avoid hardcoding of parameters (like user name and password used to connect an MDB to a remote JMS destination),
  
  
• application parameters provided they are read from Java system properties (which is easy and nice to do if you use Spring).
  

Of course, there should be a way to find the right configuration in the SVN repository. As you surely have noticed it in the previous configuration fragment, since the Properties Service is configured by an XML that is parsed, it is possible to use a system property to configure it (now this is eating your own dog food!). The system property used to select the proper configuration can be either ad hoc (a custom one added in the startup script or run.conf of the particular JBoss instance) or standardized (for example, if each JBoss instance is run with a specific user, the system property can then be user.name).

Another interesting feature of the Properties Service to leverage is the fact you can provide a list of URLs: all the properties will be loaded from these URLs, the latest ones having the precedence on the first ones. This can be used to organize configuration in a hierarchy, with global settings, server specific settings and application level settings loaded from a common place and mixed together in the Java system properties. Here is an example of such a configuration:

<br /><attribute name="URLList"><br />http://intranet.company.org/svn/*checkout*/jbprops/trunk/${some.discriminant}/system.properties,<br />http://intranet.company.org/svn/*checkout*/jbprops/trunk/serverA/system.properties,<br />http://intranet.company.org/svn/*checkout*/jbprops/trunk/global/system.properties<br /></attribute><br />

As you can see, not only is JBoss a versatile application server, but it is also an extremely configurable one that can be easily tuned to fit the needs of large companies with complex deployment constraints.

New Position: File Name Manager at Oracle Corp.

Goals:

• Rename Oracle JDBC driver file names so they actually differ between different versions,
  
  
• If possible, improve the manifest files so there is more difference than the "Implementation-Time" attribute between two different versions.
  

Required skills:

• A true desire to help developer teams and production departments making sense of the different driver releases.
  

Disclaimer for the slow brained: This is a hoax. I took the decision to create this position simply because someone had to do something.

To fully enjoy this post, go to these pages (9i, 10g & 10gR2) and download the different versions of the JDBC driver: you can not miss them, they are all named ojdbc14.jar. You can then try your skills by figuring out sensible names for these files!

A 16 Months Iteration

I have finally been able to release a first alpha version of jinFORM, 16 months after the project started. The title is voluntarily provocative: it was not an iteration but a slow maturation!

Why did it take so long? As explained in jinFORM news, part of the time has been spent waiting for a presentation server able to render some much needed features of XForms 1.1. Initially, Chiba was targeted as the presentation server but testing and experiments have finally pushed the project to use Orbeon Presentation Server, which offers a more stable XHTML + JavaScript rendering, a better compliance to standards (XForms, XML Pipelines...) and the much needed XForms 1.1 features mentioned before.

So, what is jinFORM 0.1.0 worth? Not much for production: you can only fill simple forms with repeating sections, but without computation or logic. The only submission that is currently supported is for saving the XML instance built by the form behind the scene.

Then, was it worth the effort and why should anyone care about this project? Well, anyone with a slight interest in the intimates of Microsoft Infopath file format (aka XSN) should be interested to see how jinFORM transforms this proprietary form definition to XForms.

Because an XSN file is simply a CAB stuffed with XML, XSL and extra files, described with a documented manifest, groking its content is fairly easy. Doing anything with it is another thing. Take the XSL for view rendering (usually named view1.xsl) that is generated by Infopath. Because this XSL relies on implicit MSXML objects for supporting many extension function calls, jinFORM has to pre-chew it and implement similar functions in order for anything to work.

When designing this XSL, Microsoft, of course, had anything but elegance or portability in mind. It is cluttered with prescriptive commands like xsl:for-each, traps like testing for the existence of a function and using another one and oddities like using a function to retrieve the current document on which this XSL applies. Surprising approach from Microsoft? After they love functions and procedural programming: why would they embrace the template concept that is underlying XSL transformation?

There are still many things to add in jinFORM to make it really usable but it is a good first step. Foundations have been laid, now let's build on it!

Vive le ROI!

Please excuse my French but I could not resist the pun!

Agile proponents, including me, often use an improved return on investment (ROI) as an argument for avoiding traditional software development methodologies in favor of agility. The scope considered for this ROI usually concerns the financial investment made for the particular project and the result the stakeholders are expecting from it.

There is another ROI that I think is important to consider and that is also heavily impacted by the adoption of agile practices: the intellectual return on investment.

Indeed, developers involve a lot of themselves when working on a software project. The most obvious investment is the time spent at their desks but there is also a tremendous part of the job that is done after hours, outside of the office.

Ever zombie drove from work while mentally exploring the arcane of a new system's architecture? Ever had the light bulb that turns on while revisiting the latest written code under your shower? If yes, you probably see what I mean when I say that software development is pretty mind invasive.

Pecuniary compensation is often considered as the just compensation for this invasion: I pretend this is not enough. Money only buys time. The intellectual investment must be compensated by a specific ROI. What could be this specific ROI?

From my experience:

satisfaction in the quality of work

a sustainable elevated productivity and

an increased confidence in the system that is under construction

 

is a combination that forms a good intellectual ROI.

 

Knowing that this is the kind of conditions that agile methodologies can create, it can be used as an argument to sell them to programmers, who can sometimes get convinced of the benefits for their company (or the project stakeholders or their managers) but are not always convinced of the benefits for themselves.

 

The Roots of a Bias

A few weeks ago, Kubuntu released an upgrade that crashed the OS: this was a major goof and they promised they will never do it again. When my machine refused to boot after the upgrade, I frowned, grumbled, downgraded the faulty package and kept going until a new update restores the situation a few hours later.

Now, when I work on Microsoft Windows, I almost instantly go fuming and ballistic for the smallest glitch. Should the Explorer freeze for ages because I clicked on a network resource, I see red. Should I have to defragment the mess that has become my drive after a few months of work, there is smell of a burning martyr in the air.

Could I be somewhat biased against Windows? This would be a subject of shame because any prejudice is a reduction of reality that leads to unfair behavior. Yuck! Nothing to be proud of...

What could be the roots of such a bias? I do not particularly care about Windows supremacy on the desktop computers (except that such a global uniformity is a security problem), nor about the personal fortune of such Microsoft big hat. In fact, Windows XP Professional is probably the best Microsoft operating system I have ever used, so where is this wrath coming from?

Doing a little introspection and exploring the depths of my psyche, I think I have found out the event that triggered this anger: it was in the mid-90s so this is surely a case of unfinished business!

My first application that went into production was a modem hub for the Belgian branches of a pharmaceutical distributor. The application was written in C and, the most important factor, developed and executed on OS/2. Coming from Atari TOS, I really fell in love with OS/2!

Then I finished school and started to work in the professional world, dub it, the world of Windows for Workgroups 3.11. Coming from a world of sessions, processes and threads, WfW seemed to me like a castle of cards waiting for the faintest breath of wind to fall down in pieces. And it was. Rebooting was a big part of a developer's activity of this time.

When Windows 95 started to be announced, I was certain the industry will make the right decision based on the misery of WfW and move to OS/2 Warp. Yes, I was so naive to believe that technical soundness and robustness would help reverting the trend towards Windows to what I considered a better professional operating system.

History proved me deeply wrong and left me frustrated for the many years it took Microsoft to build an operating system as stable and usable as OS/2. I think this old frustration that has turned into bitterness is the root of my bias. I need to work on it and restore pacified relationships with Windows. And I will keep using Kubuntu as my main operating system, just in case... A relapse is always possible!

Gnosis of Diagnosis

For a few days, I have been procrastinating writing about the ill distribution of debugging skills among developers, then I have read a letter sent to Computer and published in the September 2006 issue under the evocative title of "Deskilling Software Engineering".

The author states that "diagnosing a problem remains an art and requires skill". I like the fact that he evokes art because there is truly a part of instinct in the difficult task of figuring out why a system fails or refuses to do what it is supposed to. Instinct is hard to define and quantify, so what else a developer would need to be successful when chasing bugs?

From my experience, I have retained what I consider the three topmost traits that helps in the art of diagnosis:

• Systematic thinking: software being complex systems, they take a lot of various inputs (configuration, data, user entries...) and turn them into various outputs, with usually one being the desired one. Modifying too much inputs at the same time does not allow to figure out the impact of each modification on the output. Programmers without rigor or a varnish of scientific method will often try to modify many parameters at the same time, which will ruin any attempt to empirically figure out what is causing the issue.
  
  
• Pugnacity and thoroughness: some problems are hard to tackle, especially the ones that require a long setup time before being able to test what is happening (think of rebooting a server to release a particular resource before testing). Others require to be able to read through extensive traces without panicking! I have seen many non-rookie developers unable to read a stack trace and I understand why: it is intimidating and requires an effort of will to delve into.
  
  
• Hands-on experience: as the saying goes "once bitten twice shy", nothing replaces years of experience, dub it, the numerous blows and slaps you get creating then solving bug after bug. With the current trend of devaluing hands-on experience in favor of managerial skills, hourly rates for programmers are a clear signal that there is nothing good in keeping writing code. Hence when someone gets good at its job of writing code, he is kindly asked to stop doing it and moved to a more desirable (and rewarded) position, leaving projects with rookies who will have to go through years of blows and slaps to become efficient bug killers.
  

Off course, these traits can help in other software development activities as well. But their lack always impairs any diagnosis endeavor.

The price of common sense

In "Agile Software Development with Scrum", Ken Schwaber stated that:

 

"Scrum demands the liberal application of common sense".

 

I think this applies to any agile methodology: common sense is a driving force for any practice that relies on empirical management and self-organizing teams.

 

Now, I have just read "Unsystematic Engineering", an IEEE Spectrum article where Robert W. Lucky dares saying out loud what everybody knows:

 

"(...) systems engineering is often based on experience and common sense, and we know where common sense fits in the hierarchy of things that justify a high salary."

 

Okay, we all work for the beauty and love of our craftsmanship and not for the bottom line, but is not this pattern of under-valuing the qualities we have learnt to be the right ones, something that gets harder and harder to accept?

 

So, what should be the true price of common sense? Could not it be evaluated by measuring the cost of projects that failed because of a lack of it?

 

Dependable Dependencies

With the advent of human friendly build platforms like Maven 2 or Buildix, I am expecting to see improvements in the way programmers manage their dependencies. Of course, I might be too candid but proper tools can give enough incentive for developers to consider dealing with their dependencies in a less hip-shooter style and therefore benefit from more predictable, reproducible and automated build sequences.

Classical patterns of classpath quagmire I have noticed include:

• Deployment of libraries needed only at compile time: this is the usual rookie error, where you typically end up with things like servlet.jar in WEB-INF/lib. A little explanation is usually enough for this to not happen again.
  
  
• Deployment of libraries needed only at unit test time: less harmful than the previous error, it is simply disturbing to have applications deployed with unit test and mock libraries merrily packaged altogether.
  
  
• Deployment of platform libraries in excess or in different versions: this is typically the case for applications that are designed to be deployed on "any" J2EE compliant server. Very often, it will contain libraries that are already present in the server, sometimes in different versions, which will lead to hard to track issues. This is usually done just in case your particular server does not support a particular version of a J2EE sub-component, and, unless you run on the exact same server the vendor is using, you will typically end-up shaving a dozen of useless jar files off the libraries' folder!

To my sense, when it comes to dependencies, a NoClassDefFoundError is much better than a ClassCastException, or, in other words, too little is better than too much. It is indeed easier to figure out what are the missing libraries rather than what are the version conflicting ones.

Anyway, as build tools help us to formalize what is the role of each of the library used by our projects (compile? test? deploy?), we can expect to see better handled and dependable dependencies around us in the near future.

A Simple Pattern for Prevayler

While implementing Prevayler in a cool project I will detail in a near future, I of course met the need of doing it in a way that will allow me to unit test my prevalent system independently from the persistence mechanism. This would allow me to exercise all the code implied in the business logic of my domain without actually bootstrapping the prevalence engine.

I came to the simple pattern shown below:

The IDomainManager interface exposes all the domain related methods. The MemoryDomainManager implements this interface and contains all the serializable data: it is the actual root of the prevalent system and the only object to thoroughly unit test.

The PrevalentDomainManager is merely an empty shell that delegates all calls to the MemoryDomainManager, wrapped in Prevayler transactions. It also contains the basic code to bootstrap the persistence engine.

Note that the MemoryDomainManager object does not implement any synchronization mechanism: Prevayler, thanks to its support for transactions, blissfully takes care of this.

Having unified MemoryDomainManager and PrevalentDomainManager behind the IDomainManager interface naturally allows to unit test all the components relying on the domain objects without using actual persistence and to deploy the productive application by designing it to use the prevalent implementation of the said interface.

So, how about dropping your prejudices and jump head first into the jolly world of Prevayler?

________
* For those of you who wonder, Prevayler is a purely object oriented persistence framework that offers a solid alternative to the now classic "RDBMS+O/R mapping framework" pair. It received a well deserved Productivity Award in 2004.

Surfing the expansion wave

Software development is a very demanding professional field of activity. Anyone who caresses the desire to make a living of it should be very aware of this in order to take a well informed decision before plunging into this business. It is sometimes best to decline the red pill and take the blue one.

I vividly remember my 1989 student project leader who, after introducing the exhilarating objectives of the Celesstin research project that was done in cooperation with IBM and Dassault, opened the door and took the time to explain the challenges of software development: there would be no shame to leave the room before the project started, but after, it would be a world of pain for everyone. I wish all computer science teachers could do such an introductory speech: many people would have a better fitted job and would stop making passionate developers mad (on the other hand, what would then happen to the Daily WTF?).

For me, the most demanding aspect of our job is to stay current with what is going on out there. Software development, unlike most of jobs, has a constantly growing body of knowledge. All the difficulty is to stay somewhere close to the border of this constantly expanding universe. This is an exhausting and sometimes frustrating endeavor:

• exhausting because on top of our daily paid-for activities, we have to explore this growing universe, very often using our free time and what is left of our brain power after a long day of work to read, download, test, wander and sometimes wade in terra incognita that others have pioneered for us,
  
  
• frustrating because it expands in all directions and it is therefore virtually impossible to stay close to the border all around this sphere of knowledge: everyday brings its wealth of new technologies, new releases, new frameworks, new tools, new methodologies, new fundamental discoveries... We can only chose an area we feel comfortable with and surf the expansion wave as much as we can.
  

Once you have accepted that every morning you will wake up, the universe will have expanded a little and the border will have moved further so it will be another day of learning and discovery, life will be easier. No more dreams of "being at the top" but, instead, the simple path of the craftsman who learns and improves everyday and everyday realizes that the amount of things to learn and practices to improve has grown a littler bigger than the day before.

Now what is the fuel for doing this, if not passion?

Coverage Generation or Generations of Coverage?

While I was thinking of this post, I came along the "32 & 16 Years Ago" section of the July 2006 issue of Computer and was amazed by what I read in the entry titled "Software Testing". It started like this: "The objective of (...) probe insertion mechanisms (...) is not to find errors; rather it is to quantitatively assess how thoroughly a program has been exercised by a set of test cases" (edited by Computer).

This is dated... July 1976! Code coverage in 1976! Golly! What a blunt reminder that we are, at most, building on the shoulders of giants... Humility is definitively the only path that is safe to walk.

Anyway, my current concern with code coverage was to fix the 100% target that I was talking about in my "Programming for Agility" post and that I borrowed from Uncle Bob himself. This is a nice high level target but can it really be reached? I do not have much doubts for code I am writing with my little busy hands but what with generated code? Especially code that contains many conditional branches like the one Eclipse generates for equals() and hashCode() methods?

@Override

public boolean equals(Object obj) {
if (this == obj)
 return true;

if (obj == null)
 return false;

if (getClass() != obj.getClass())
 return false;

final DefaultTodoListItem other = (DefaultTodoListItem) obj;

if (id != other.id)
 return false;

return true;
}

I came with three possible strategies, including one for which I already apologize:

1. Exclude generated code from test coverage estimation: for having written this one, I have already slapped my hand as a due punishment* ! Generated code does not mean intrinsically safe code: it is live code and is subjected to all sorts of possible errors you can imagine.
   
   
2. Use test code generators: great tools like Agitar Agitator can very conveniently generate test cases for all sorts of extreme values that will most probably explore all the possible execution paths of the aforementioned generated methods. These tests would complement nicely the ones written by the developers.
   
   
3. Reduce the number of needed tests by enforcing class invariants: this can be done by working with immutable objects and check for invalid or exceptional conditions at construction time. This also allows to pre-calculate values like hashCode() and toString() rendering. The tests will then target user written code and will exercise construction of invalid objects instead of testing the calls of all methods possibly sensitive to an invalid state object.

Please do share other possible approaches or practices you think of or daily use in this matter. Even if we are the second generation of code coverage buffs, we might still have something useful to say!

____________________
* An efficient self-punishment technique I learnt from J.B. Rainsberger during SD West 2005.