Fast and Furious

I'm sure I have goofy speech patterns and favourite catch phrases of my own but I also have a talent for picking these colloquialisms out of someone else's speech as well. I have discovered over the years that the best time to capture these beauties within an IT environment is during the inevitable "status meeting". It's almost like a freestyle rap battle where two or more contenders try to outdo each other for status meeting supremacy. Fortunately for me, the agile process inspired daily standup meetings on my new project have become a veritable gold mine for these things. We have a few people on the project who are black-belt kung-fu masters of the colloquialism spinning these out with speed and grace... Here's a sample of the ones I've captured so far:

up to speed	high level design
make it real	snapshot
visionary thing	anecdotal evidence
identify the need	lay the groundwork
cycles available	disparate user experience
bottom feeder	look and feel
production capacity	nebulous
context	numerous permutations
drivers	shock effect
more or less	impact
grease the skids	fresh set of eyes
steer going forward	engage you
take forward	externalize
put the spin on it	socializing it
draft rollout plan	show stopper
iterative process	mitigate risk
right hand man	extra pair of hands
duck is cooked	messaging protocol
key artifacts	validation effect
verbage	sooner rather than later
put lipstick on it	technical components
bring to the table	consider the options
test bed	full meal deal
iterative cycle	lock it down in stone
instant gratification

Misplaced Priorities

I've worked on quite a few software projects in my career and I always wonder at the people who get hung up on their favorite features regardless of importance to the actual users of the system.

A few years back I worked with a developer who became analyst. At some point in one of his past projects he had created a feature for allowing a user to press some key combinations to "hot jump" from screen to screen. Although he never really elaborated I suspect this feature was more than just the typical shortcut keys you see in most desktop applications. But that's not the point. He became convinced this was THE feature to add to our app. Over the next few weeks he became fixated on "hot jumps". We kept hearing about "hot jumps" over and over. This was going to be the "most used feature" in the system.

But the other developers and I didn't agree with him... We felt "hot jumps" had absolutely no value unless the destinations of these hot jumps actually worked! And that was the problem and the point of this blog entry. We had tight deadlines, and a lot of actual business functionality to develop long before features like "hot jumps" should ever have entered anyone's mind. I'm not saying "hot jumps" would have been completely worthless, but at the time, it was undoubtedly a misplaced priority.

So it's with some surprise (and frustration) that I find myself in a similar situation today, only it's not "hot jumps", it's directory structures, eclipse project files, and (lord help me) Ant scripts. Under normal circumstances I would definitely say these things all have some importance, but what about actually developing the application? We're three weeks into development with five developers but under the direction of our "leader", I find myself spending my entire day renaming things and begging the "gods of Ant" to actually let me get this damn build.xml file right so it will do what I want. And gentle reader this is not just your regular run of the mill ant script. This is the mother of all ant scripts, utilizing every possible ant task available. And let me clarify, I did not write it. I inherited it and have been instructed to "make it work".

But my point again is this, what good is this "hot jump", oops, I mean Ant script if it doesn't have anything to build? At the end of the day, will the users be impressed with the quintessential Ant script or would they actually like to have the features they asked for?

DHH Interview

I just found a SYS-CON.TV interview with David Heinemeier Hansson that had some real quotable moments:

In his description of Ruby On Rails:

...we like to consider ourselves the Apple of open source web application development...

...it's extremely conventional. Most of the ideas in Ruby on Rails are the same ideas people have been building web applications on for the last ten years. They just hurt a lot less...

On Convention over configuration:

...In Java we call it XML sittups. You can produce just as much XML configuration as application code and that's just insane in our mind...

Does it scale?

...unquestionably yes. simply because Ruby doesn't invent a new architecture. We use exactly the same architecture as people who have been building the biggest sites on the internet.... shared nothing...

On the value of Rails.

[People think] it's so valuable to have extreme performance when there are tones of other attributes that are more valuable: programmer productivity, how maintainable your application is over time, ...

Ruby jEdit Plugin

I've been a big fan of jEdit for quite a few years now and I just downloaded a new plugin that I just simply felt compelled to write about, the jEdit Ruby Plugin. It's not the simplest thing to install but the integrated code completion and documentation features (including Ruby on Rails) are quite good. For example, when I'm editing a model object I can type in "has", hit ctrl-space and get a popup with the options "has_one", "has_and_belongs_to_many", and "has_many" options, and if I'm displaying the Ruby Docs window (currently docked to the right side of my jEdit window), it will display the documentation for each option as I select each one. Very nice. Perhaps some competition for RadRails?

Agile or Traditional?

I was recently speaking with a friend I haven't seen in months and she mentioned that she was reading a book I had lent to her. The book is called "Balancing Agility and Discipline" by B. Boehm and R. Turner. The book's main theme is the attempt to balance agile software development methods with traditional methods. The idea is to try and assess your project by looking at several different characteristics and then choose the processes from each method that will allow you to best manage it.

So that idea got me thinking about my most recent project, a Java rewrite of an existing application, and how desperately we were trying to use a pure agile methodology when it didn't really fit.

Here's a quote from a presentation available on the "American Institute of Aeronautics and Astronautics" website:

Problems characterized by change, speed, and turbulence are best solved by agility.

• Accelerated time schedule combined with significant risk and uncertainty that generate constant change during the project.

Is your project more like drilling for oil or like managing a production line?

• Oil exploration projects need Agile processes.
• Production-line projects are often well-served by rigorous methodologies.

Our project was undoubtedly a production-line application with very low risk:

• Although I didn't really like the technology we used on this project, it was being used to some degree of success on two other projects. So while it was undoubtedly not very productive, it was usable.
• The database design and user interface design were to be lifted untouched from the existing application.
• We were mandated with reproducing an existing production application so the requirements were well known if not well documented.
• Business users were not co-located and not easily reached for consultation (during the entire project I never spoke to a single person who would actually use it).

So what drove us to try and use an agile methodology? Well.... it's trendy. Agile is "in" and waterfall is "out". IT people would like to live in a world of absolutes, where we can say with certainty that A is better than B. But the reality is that we live in a world of spectrums where things are classified in terms of a position on a scale between two extreme or opposite points. Just like we shouldn't try to develop a simple project using every conceivable J2EE technology and framework available, maybe we shouldn't try to apply every agile process available where it just doesn't fit. Sometimes it's okay to create a plan and to write down requirements. Sometimes it's the best thing to do.

Suffering

One of the Pragmatic Programmer tips is:

"select" Isn't Broken.

Which means that's it is rare to find a bug in the OS or the compiler, or even a third-party product or library. The bug is most likely in the application, i.e., my code.

The problem with the project that I'm working on is that there is so many layers of code and abstraction that I don't even work with the framework anymore but some class ten times removed.

For example I have a JSP calling a JSF configured backing bean, calling a Spring configured service, calling another Spring configured EJB in a different process, calling another Spring configured service calling another Spring configured DAO which is (eventually) a subclass of Spring's HibernateDaoSupport class which attempts to hide Hibernate's API altogether.

So while I try to take the tip seriously and not blame the framework, there is so much stuff happening before it even gets to Hibernate (and let's not forget all that XML goo that glues everything together) that I can't help but lay some blame at the feet of these frameworks that encourage so much obfuscation.

To get real geeky I'll quote C3PO from the original Star Wars movie:

We seem to be made to suffer. It's our lot in life.

Bruce Eckel on Ruby and Rails

I just listened to the latest podcast by the Java Posse; an interview with Bruce Eckel.  Since it's the "Java" Posse there's a fair amount of discussion about Java, but the second part of the interview dives into Bruce's thoughts on Python, Ruby and Rails.  Although he gets some things wrong, like attributing the creation of Ruby to David Heinemeir Hannson (DHH) instead of Yukihiro Matsumoto (Matz), he largely says a lot of good things about Ruby and Rails and admits where he lacks sufficient knowledge to comment.

The one part in particular I agreed with was his assesment of the Rails supporters as being "loud", as in "loud-mouthed".  He says it's a lot like the early Java days.  As a matter of fact, my first taste of Rails was a presentation by DHH.  His rhetoric in that speech was a bit much to bear, but if you can see your way past the hyperbole, and just see that the guy is truly passionate about creating "beautiful" code, you get a good idea of why Rails is the way it is.

Rails is Boring and Ruby is a Toy

Chad Fowler did a presentation last month that I happen to be listening to at the moment. He's talking about his "evolution" from a Java developer to a Ruby developer. He talks about the various XML files you have to edit when doing J2EE, the number of lines of code that's needed to do just the simplest things in Java, etc.

What I find so surprising is that some of my recent comments seem to be echoing his statements. Unlike him however I was already on the path to trying to simplify my Java development and Rails has simply dropped in a great big exclamation point. Web application development doesn't need to be the morass we suffer through in Java. Ruby can make it fun and Rails can make it so easy to do. Geez now I'm starting to sound like an evangelist too! Oh well I use a Mac too... maybe it's in my DNA...

Hibernate vs ActiveRecord

Over the last 2+ years I've grown quite accustomed to the facilities of Hibernate (one of the most popular Java object-relational persistence frameworks). And before that I used TopLink for a couple of years. So now when it comes to my work with Ruby on Rails I have some pretty high expectations.

The out-of-box experience with Rails' persistence framework is undoubtedly different but similar enough to Hibernate to make life fairly comfortable. But recently I started watching the SQL that gets generated from Rails and decided I'd like to optimize them a bit. But unfortunately that isn't as easy as everything else in Rails. I found this article at TheServerSide that sums things up pretty well. I think Rails comes out on the losing end. Now the question becomes, are the things you lose worth the things you gain?

What I don't have to know

I was thinking recently about how someone with no knowledge of Java or Rails web development would choose which of the two technology stacks to learn. Then I started comparing the lists of technologies on both sides.

1. For the View you have JSP (Java Server Pages) vs ERb (Embedded Ruby). (We'll assume that (X)HTML, CSS, & Javascript are a given knowledge requirement regardless of framework choice.)
2. For the Controller you have JSF (Java Server Faces) vs ActionPack
3. For the Model you have Hibernate vs ActiveRecord. (Since both of these technologies are Object Relational Frameworks we'll assume that relational database knowledge, including SQL is a given requirement regardless of framework.)
   

So on the surface things look pretty similar. But let's dig a bit deeper.

First let's start at the View: In Rails we take standard HTML and embed Ruby to dynamically generate the content. In JSF-flavoured JSPs on the other hand we substitute HTML tags with custom JSF tags. You must also know JSF EL (Java Server Faces Expression Language) because it is used heavily in the JSF tags.

Secondly, the Controller: In Rails, controllers are written in Ruby. In JSF the controllers are thankfully written in Java but in order to make them work you have to configure the dependency injection framework with XML. Your JSF XML file must conform to a particular DTD. Now if you also believe in the value of dependency injection frameworks you may want to use Spring. Spring isn't mandatory, but it's popular enough to be part of a typical Java stack of technologies. Unfortunately it requires it's own XML DTD.

Finally let's move onto the Model: In Rails we use Ruby to define our model. The naming convention is usually sufficient to allow ActiveRecord to determine what classes map to which tables. We typically use snippets of SQL to tell ActiveRecord how to fetch data from the database. In Java, Hibernate is the common persistence framework choice. In Hibernate another XML DTD tells the framework how to map the classes to the tables. But since people grew tired of maintaining so much XML they turned to XDoclet. And since that became so popular there are now Java 5 annotations that allow you to annotate your domain model wth mapping metadata. Currently you can use any of the three choices. Hibernate also uses its own query language, HQL, to instantiate objects from the database. This query language is based on SQL and has quite a few benefits but is yet another syntactically different language to learn.

So let's add it all up. To do Rails web development you need to learn:

1. Ruby
   
2. the various Rails APIs (e.g., ActiveRecord & ActionPack).
   

To do Java web development you need to learn:

1. Java
2. JSF tag libraries
3. JSF EL
4. JSF's XML DTD
5. Spring's XML DTD
6. Hibernate's HQL
7. Hibernate's mapping DTD or XDoclet or Hibernate Java 5 annotations (and the DTD for configuring the framework)
8. The DTD for web.xml.
9. The APIs for Spring and Hibernate and JSF
10. (And I didn't even mention EJBs)
    

People may look at this list and say "wait a sec, you're being unfair, XML is XML and you're counting four different types of XML in the Java stack". Well all I can say is that when I have to edit one of these things I have to take my eyes off the code that I'm currently working with and make a contextual switch to something that looks very different. That switch is enough for me to penalize the Java list with a bullet point for each one. But even if you collapse the four XML items into one, my point still stands: there seems to be an awful lot more things you have to be comfortable with when developing Java web applications. Perhaps that's why I'm beginning to favor Ruby on Rails so much more. It's the Simplicity.

Manifesto

Everyone has a manifesto nowadays. I just read an interesting entry at the "Creating Passionate Users" blog that pointed to the the 37signals manifesto. Both recommended reading.

Patterns

Christopher Alexander says:

Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution ...

So why is it so commonplace to use patterns where a problem doesn't exist?

Example 1. The Data Access Object was originally created to hide various data access APIs (LDAP, RDBMS, ODBMS, XML, EJB, etc.) behind an interface. But in reality most applications use a single data access API, and it's usually a relational database. So if you already use something like Hibernate as your data access framework, is it really necessary to wrap the data access in yet another class? Hibernate already abstracts you from the specifics of the target database (e.g., SQL Server, Oracle, PostgreSQL, etc.) Your queries are usually created in HQL and are written in the terms of your object model not the data model, so why add another layer that just delegates to Hibernate? Think about it, how many of your DAO methods look like "return session.getNamedQuery("someName").list();".

Example 2. The Data Transfer Object was originally conceived when Entity EJBs were in vogue. Because you wanted to use course grained data access and eliminate network traffic you'd package up your data and pass it over the wire in a DTO. But in a world where you're using something like Hibernate in the same VM why incur the overhead of DTOs and the marshalling they require. Hibernate can give you a domain object that you can disconnect from the Hibernate session and work with independently. Secondly DTOs are horribly brittle. If you change a domain object chances are pretty good you'll have to change a DTO.

I'm not saying the two patterns I used above are necessarily bad, it's just that they don't need to be used in every single case. People should look to patterns when they discover a problem with the way things are being coded. Remember the old agile mantra "Do the simplest thing that could possibly work". Don't try to build the most complex solution with every conceivable pattern. Instead think of building the simplest most elegant solution with the fewest lines of code. The people who have to maintain your code will thank you. Well okay, they might not thank you, but maybe they won't be cursing you every hour either ;-)

RadRails 0.5.3

In the rails community it would appear that the majority of developers use Macs and the text editor TextMate. (What is it about Rails that seems to attract so many Mac people? I'm not arguing, I'm a Mac guy too, but it's an interesting question to ponder) But coming from a Java background I've grown fairly accustomed to Eclipse and so the RadRails and RDT Eclipse plugin projects caught my attention. But they each have their own strengths that made me wish I could take the best of both worlds. Well version 0.5.3 of RadRails shows the benefits of those two teams collaborating. RadRails is now using RDT for editing Ruby. It also means you get all the other nice features of RDT like a visual Test::Unit runner, the RI view, etc. I'm also pretty excited to hear where the developers of RadRails want to take RadRails. In short:

The focus of RadRails is going to move towards achieving dynamic content assist.

Rails development gets better all the time and the future looks bright.

Snakes and Rubies

Over on the Django website you can find a video of presentations made in Chicago on December 3rd, 2005 by two leaders in web application development. Adrian Holovaty, one of the Django creators, and David Heinemeier Hansson, the creator of Ruby on Rails, present their respective frameworks and then take some questions from the audience. Naturally I was curious about the Ruby on Rails presentation. David quotes Richad Feynman:

You can recognize truth by its beauty and simplicity. When you get it right, it is obvious that it is right.

and then goes on to make the assertion:

Beauty leads to happiness, happiness leads to productivty, thus Beauty leads to productivity.

and that Ruby on Rails is part of his search for beauty. He may be on to something. It's also fun to hear him curse out Windows a few times, "this is ugliness" :-)

Rewrites

System rewrites must be extraordinarily popular because I've worked on my fair share of them throughout my career. And usually the story is the same...

Poorly-trained developers in an IT department or startup slowly build a system over the course of a few years and then eventually come to the realization that the system needs a rewrite. Here are some of the most common reasons:

1. The database design is bad (Usually it's atrocious. It's very rare to find good data models)
2. The code for the business logic or UI is a hopeless tangle of procedural code (You'd be lucky if thy UI and business logic are separate. I've even seen one implementation of a web application where data access, business logic, and even HTML markup was generated by Oracle stored procedures.)
3. Performance issues have cropped up (Most applications are bound by the performance of the data access code. Good data model design is crucial in this respect, but bad queries (easier to make with object relational tools btw) are the usual culrits and are a good place to start.)
4. The technology is obsolete. (Unfortunately this is a fact of life in IT and it is common for this old code to be the only "documentation". Keep the developers of the future in mind when writing your code and make it readable.)
5. It's just so fragile that implementing new functionality takes way too long. (If you see gigantic nested if-then statements, run away now before you run away screaming later.)
6. The new IT manager doesn't like his/her predecessor's legacy (It's sad but true. The "wiping out all vestiges of that 'bad' manager who was the harbinger of all things evil" is a bit of a spectator sport among developers.)
7. It's a combination of some of the above. (And if you've really arrived at the precipice and are gaping into the abyss it'll be all of the above.)

Eventually the decision to rewrite is approved and someone must make the decision about how to redevelop the system. People usually see these two paths:

1. They can redevelop the entire thing, affectionately dubbed the "big-bang approach"
2. They can incrementally replace the subsystems one at a time.

Usually the incremental approach is the most attractive because it's seen as having less risk. And of course managers who like their jobs like less risk. But this "lower-risk" decision usually comes with one very difficult problem: "How do you integrate the new code with the old and keep them both running at the same time?" Well the typical decision is to just leave the database alone and build the new application on top of it. While this seems to be an easy choice (look at all the stuff you can reuse and no data migration activities to endure), this is one of the worst things you can do.

No matter how good the database may appear at first glance and no matter how much faith you have in your object relational mapping (ORM) tool's ability to "abstract away the ugly parts", you are always going to compromise your object model. It is extraordinarily difficult to design the best domain model you can without repeating the design decisions made in the database. It's like trying to hum a tune while listening to another song on the radio, you inevitably end up singing the song on the radio.

For several years I believed that ORM really gave me an abstraction from the physical database, that the two were independent and it was meta data that glued them together. But after some real experience, I learned the hard way that was a fantasy. If you want to be productive and want to work with an object oriented domain model that you can easily refactor you really need to be able to change the data model just as easily. I now consider the domain model and the relational data model to be two perspectives on the same thing. If you don't believe me, I ask you to consider the following trends:

• In EJB3, the relational view of the domain model is becoming part of the domain model through Java 5 annotations. The main reason for this is practicality. It embraces the DRY principle (Don't Repeat Yourself): every piece of knowledge should have a single, authoratitive, and unambiguous representation in a system. It's an acknowledgment that the attribute oriented programming model advocated by XDoclet so many years ago is a good one. I suggest that you should even generate your database from this metadata, keep it DRY.
• In Ruby on Rails, much of the domain model is generated at runtime by querying the database meta data. The database can't express everything in a object model so the rest of the picture exists in the Ruby domain classes. This practical approach that intrinsically marries relational and object views is one of the many features of Ruby on Rails that make its supporters claim such high productivity benefits.
  

You may also want to consider what data warehousing people have been doing for years. They always treat the transactional databases as sources of data but they always stage that data somewhere else and "cleanse" it before making it part of their warehouse. So while they're dependent on that data model for ongoing feeds they have a real abstraction from it and none of their cubes or reports are impacted by a change in the data source.

Rewrites appear to be inevitable. Even if the reasons for doing it are dubious you'll probably find yourself in one of these projects sooner or later. But if you going to get involved, make sure it's a complete rewrite. Don't take shortcuts. The most successful rewrite project I worked on created a new domain model and new data model for the new application and used database technologies to keep the new and old data model synchronized. We got the productivity of not being bound by the old (and very scary database) and we kept everything else that was dependent on that old database running by keeping the old database alive. The most aggravating projects have been the ones that tie you down by saying "leave the database alone". Then there are the ones that say "leave the UI alone" too. Ugh! But that can be the subject of another article...

Checkboxes

Ever since I started developing Windows applications I've consistently seen other developers use two radio buttons for a boolean option with ridiculous labels like "YES"/"NO" or "TRUE"/"FALSE". And just today I'm redeveloping a page with 3 of these things. Why on earth don't people use a checkbox? It was made for this! On or Off! For crying out loud! Arghhh!

Déjà vu

On this date one year ago the Canadian junior hockey team beat the Russians in a gold medal match. This time they were one of the underdogs in the tournament but still came out on top. Congratulations again!

RoR & Foos update

Tonight I decided to stop tinkering with the object/data model and start building the simplest ladder web app that would work. In pretty short order I was able to display a sports ladder (e.g., foosball) with names and positions. But a display-only ladder application isn't teribly useful so I figured the "easiest thing that works" was to allow a user to swap the positions of two teams on the ladder by selecting a couple checkboxes and clicking on a button labelled "swap positions".

After a couple hours work I had something that was minimally useful! :-) Then I decided that I really wanted to be able to support multiple ladders at the same time for both singles and doubles play (after all my overly-analyzed object model already supported it). And fortunately as I refactored the code I learned a bit more about Ruby on Rails' layouts and partials features. But one of the better ideas I had was adding a signup method to my ladder class that looks like this:

def signup( players, screen_name = 'undefined' )
[...]
end

It can be used like this:

singles_ladder.signup([joe])
doubles_ladder.signup([bob, doug], 'Hosers')

Certainly a lot simpler than the code I had before that created a team, added the players to the team and then added the team to the ladder. For some reason it just felt more Ruby-like to do it this way.

2005 Redux

In 2005 I think I met my Pragmatic-Programmer-inspired goal to learn a variety of programming languages e.g.,

• Developed a small application in Mac OS X's Cocoa Environment (with the help of Aaron Hillegass's book)
  
• Attended a course about Microsoft's .NET Framework and C# (also messed around with Mono)
• Accepted a new contract and learned a few thing about JSF and Spring.
• Began developing a small web application with Ruby on Rails
• Took glancing looks at Tapestry, Smalltalk, Applescript, EJB3, etc.

One thread that seemed to permeate many of these technologies was object-relational-mapping. There's EJB3 and Hibernate for Java, ActiveRecord for Ruby, Core Data for Cocoa, and even LINQ for C#. While the implementations all vary, the basic principles apply to all. It seems like all that time I spent with TopLink in 2001 was well spent because ORM has become the norm.

It also looks like dynamic languages are becoming more popular. I'm glad I spent some time looking at Objective-C before getting on the Ruby bandwagon. A mature framework like Cocoa helped me appreciate the power of a dynamic language. Rails will undoubtedly continue to occupy me for at least a little while in 2006.

But not everything I did in 2005 was related to programming. I also spent some time with:

• A family trip to Maui
• Video Editing
• Looking into my family history
• Playing foosball
• Digital Photography

Travelling has always been something I've enjoyed doing since I met my wife. But when we had kids our amount of travel significantly declined. But now that they're getting old enough, it's nice to be getting back into it. In 2004 we made it to Disneyland, last year it was Maui. Who knows what 2006 will bring...

At the end of 2005 I finally burned a custom DVD. I made a slideshow out of the pictures I took at Christmas, added some videos I edited a few years ago and put them all together with some music and some fancy menus. With the exception of GarageBand and some help from Final Cut Express, I used all the other pieces of iLife for my grand DVD opus.

There's only one thing I wish I'd done more of in 2005, and that's art. I suppose the video/editing/photography thing could be considered somewhat artistic but I'm talking traditional oil-on-canvas kind of art. My son recently complimented me on a painting I'd made a few years ago. Maybe that'll inspire me to finally break out the paints... Here's looking forward to the new year. :-)

The Daily WTF

A co-worker just sent me this URL. I had a good chuckle looking at some of the code posted on this website so I figured I'd post it here for posterity. There's an RSS feed too so you can keep up to date with the latest WTFs.