History Repeating

In the nineties, I used to work as a freelancer, typically in small one-man projects. At that time, Perl was by far my favourite programming language. Some relicts can still be found. Perl was the proper tool for the job: I had the oversight. I knew any single line of code. If changes were required, then I knew exactly what to do.

When I started to work in larger projects, with two or more members, I felt the weak sides of Perl immediately. Even as the lead developer, I began to lose control. Even if I knew how sources had been, I didn't know exactly, how they were. For example, I could no longer change a methods signature without the risk to break things. The larger the project, the more overwhelming the problem. Java with its type safety and strictness was much more appropriate. I was again able to respond quickly to changed requirements. (Today, I really wonder how large PHP projects are managed and how much time is lost when dealing with these things.)

Of course, that wasn't the end of the story. Code is more than Java, Perl, or any other programming language. I began to become aware of that in my first XML projects, around 1999. The initial version was always based on DOM. The application programmers were fumbling around from node to node, reproducing algorithms again and again. Of course, DOM (or SAX, or pull parsing) was (and still is), the proper tool for generic XML processing. But it is extremely unusable for application programming: As soon as the data model changes, you are lost. Once more, the concept works for small projects. But in a large project, things are quite different: There's absolutely no reliable way to find all locations in the code, that need to be changed.

My reply was the predecessor to JaxMe 1, an early Java/XML binding tool. The binding compiler converts the DTD, or XML schema, into Java beans. By using these beans, the Java compiler controls your use of the data model. If the data model changes, then the compiler tells you exactly what breaks. Ideally, the application programmers will be able to implement about 80% of the projects code by working on the beans and the project becomes manageable again. Of course, the JaxMe predecessor, JaxMe 1, and even JaxMe 2 are very limited tools, compared to modern and full blown Java/XML binding suites like JAXB. But that's not the point, in contrary: It's quite telling, that even my self-made tool could boost a project. The fact, that there have been so many similar projects (Castor, Zeus, XML Beans, to name a few), also goes to show the concepts value.

Two, or three years later, I felt a dejavu: For the first time, I was working in a large SQL/JDBC project. It was the same story again: As soon as we had implemented the first 50 or so queries, some of them quite complex, we felt again, that the growth began to become uncontrolled. An action as simple as changing a column type from boolean to integer could break the project and result in hours of bug tracking. As Hibernate, OJB, and other object-relational mappers (not to mention JDO) weren't available or not really recommendable at the time, I began to implement my own nonsense again. It was sufficient as the base of two large projects. A similar tool, not implemented by me, but by another very capable programmer (Hello, Winfried, should you read this. :-), was the foundation of a third project. The success in relation to the simplicity proves again, how important mapping of loosely coupled entities (tables) to structures controlled by the Java compiler can be.

Well, history's repeating. Nowadays, I am working in a large project again, feeling my own unability to keep the projects parts together: This time the entities are called registry objects, classifications, or associations, they are stored in a registry and accessed via JAXR. The project is getting larger, the pain caused by data model changes is growing. This time, we do not even have a schema language, not to mention a binding compiler, that translates structures into accessors.

But maybe, we can learn from history. I promise that I search for other peoples solution much longer this time. :-)