Rails and Apple together in Leopard

David Heinemeier Hannson notes in his blog that Ruby on Rails will ship with the next major release of the Mac OS, Leopard (a.k.a., Mac OS 10.5). As a Mac user and Rails admirer I have to say that these two seem to go so well together... Not sure how to explain that... It's just that they both have a great aesthetic, and are two of the most elegant software products I've used in years.

Cool Reception

A friend of mine, who works at a Java shop, recently gave a Ruby on Rails presentation and was surprised and dissapointed by the people who didn't come to the presentation. In his blog he discussed some of his theories about why one person in particular didn't show up...

I have a simpler theory, one that was presented by Bruce Tate in the Ruby on Rails Podcast back in March of this year. He said:

You can either look at a programming language as a means or as an end. And if you're looking at a programming language as a means it's just a tool to get a job done and religion just falls out of it. But once a language is entrenched and that language becomes a meal-ticket then the language becomes the end. And when that happens you're gonna get passions stirred up whether you want to or not, because now when you threaten a language you're threatening somebody's ability to feed their family...

I wouldn't present it quite so dramatically as being unable to feed one's family, afterall Java isn't about to be replaced any time soon, but I think his position is probably correct. Java is a meal-ticket for a lot of people. Many have worked their way up to nice lofty "architect" or "senior lead" positions. Most of them through hard work. They know Java and the frameworks and tools that go with it (JSF, Hibernate, Struts, Spring, Eclipse, IntelliJ, Tomcat, JBoss, JUnit, etc). They've made a significant investment and may quite naturally feel threatened by something like Rails whether they realize it or not. It challenges their position and knowledge.

Andy Hunt, one of the authors of "The Pragmatic Programmer" said:

Languages and frameworks come and go, but your habits--your practices--stay with you for your whole career.

Unfortunately, most people don't understand that. They look only at what's directly in front of them and don't check the horizon to see what's coming. Today Java can be a morass of complexity and it's only by looking at things like Rails do we challenge our preconceptions and make our day-to-day Java development positions better.

Keep giving the presentations, continue to push the boundaries. People don't like to be pushed but constant gentle persuasion will eventually leave its mark.

SOA and APP

During a recent job interview they told me that one of their new technical directions is Service Oriented Architecture (SOA). So I'm thinking okay, Web Services probably figures into that somehow... But unfortunatley (or maybe fortunately) the last time I touched any of that stuff (UDDI, WSDL, SOAP) was at least a couple years ago. So I started Googling to get up to speed again...

All the regular Web Services stuff is still out there but I found a couple other articles/blog entries/presentations by Joe Gregorio that are much more profound:

• Implementing the Atom Publishing Protocol
• How to create a REST protocol
• The Atom Publishing Protocol: Publishing Web Content with XML and HTTP
  

In the meantime I also watched a presentation by David Heinemeir Hansson called "Discovering a world of resources" (slides, video) where he describes something obviously influenced by the Atom Publishing Protocol (APP).

My interpretation of all this is that we tend to naively use just the GET and POST methods of the HTTP protocol and tend to be fairly inconsistent and sloppy in how we choose our URIs. These articles suggest that we can more wisely and efficiently use all of HTTP's methods to provide basic CRUD functionality:

CREATE - POST
READ - GET
UPDATE - PUT
DELETE - DELETE

Rather than reinventing the wheel with SOAP and all the WS-* standards we can just keep it simple with HTTP. Sounds pretty smart to me.

MacCallisto

Even with a few weeks off from work I still like to keep up-to-date with what's going on in the Java world. And the biggest news so far has been the release of Eclipse 3.2 and the various other Eclipse based projects. So I downloaded 3.2 and then went on to install these other projects:

• Visual Editor (VE)
• Java EE and Web Tools (WTP)
• Test and Performance Tools (TPTP)
• Reporting Tools(BIRT)
• Data Tools (DTP)

I was happy to see that VE is now supported on Mac OS X. I was able to quickly create a very simple Swing app in just a couple minutes. I can't say too much about stability of the tool at the moment but it worked first time!

Then I moved onto the Data Tools. It was pretty easy to connect to PostgreSQL (my preferred local database) even though PostgreSQL wasn't one of the predefined driver types. I browsed my schema and then quickly learned that you need to create a SQL editor and then connect it to a database before you can issue ad-hoc queries. In the end it's nice to see these tools in Eclipse but the implementation is still pretty weak compared to something like Aqua Data Studio whose code completion features are more comprehensive.

Then because I haven't really messed around with Web Services before I figured I'd take the Eclipse web tools for a spin. Now I know why David Heinemeier Hansson calls the Web Services collection of standards, which usually sports the abbreviation WS-*, as WS-death-star! Yikes! But in the end I was able to use the wizards to create a bottom-up web service from a simple JavaBean method that returned a String and then deploy that web service in Tomcat 5.5 and then use the generated web service client, also deployed in Tomcat, to execute the method and return my String. It sure is a lot of plumbing to do something so simple but maybe for large heterogeneous enterprise IT shops the benefits for integration outweigh the overhead...

So I still have the reporting tools and the test and monitoring tools to look at but so far everything looks good in the latest Eclipse offering. But I must admit I was suffering some nasty Eclipse crashes at first with the rather nebulous "Out of Memory" exception. After a bit of monkeying around I discovered that the following changes/additions to my eclipse.ini file (within the Eclipse bundle on Mac OS X) made everything happy again:

-Xms256M
-Xmx768M
-XX:PermSize=128M

The last one in particular made the biggest difference. Let me know if you discover anything interesting...

Just Say No to Specialization

One of the topics of our standup meeting this morning was the communication problems with the database group. At the eleventh hour they wanted to change the names of tables, columns and constraints. While I suppose there's nothing too much wrong with caring about consistent naming conventions, I question a lot of the specialization that happens in larger IT shops nowadays.

I read this quote on Apple's Pro site:

Where these big shops have tiers and tiers of people focused on something specific and get bogged down by their infrastructures, we can win a huge campaign with just two artists and a Mac.

...and I can't help but see the parallels with IT development. The last few shops I've worked in (with one exception) have had Business Analysts, System Analysts, Technical Architects, Senior Java Developers, Junior Java Developers, Product Managers, Software Project Managers, HTML developers, Art directors, DBAs, QA managers, QA testers, Release Engineers, etc. etc.

As a result, even the simplest project ends up involving at least 10 to 15 people. If you then consider the communication overhead this introduces, there's not much wonder that productivity suffers. The funny thing is that all these shops actually think that they're agile.

Anyway, back to the database group's requests for change. WTF!? I've long since come to the view that the database model and the object model are just two perspectives on the same thing. But in a lot of circumstances I see the DB design and the object model design begin to drastically diverge as the product evolves. And naming conventions tend to be the worst offenders.

Usually a glossary of terms is developed (even informally) as the dev team and the business people talk about the business problem, and usually the object model's class, attribute and method names are changed to better reflect the real world problem space. But for one reason or another, the database is left to decay. And then when the all-knowing database group comes along to make the problem worse and suggests/enforces naming conventions, I just give my head a shake...

For me it's all this skill specialization that is destroying the holistic view of the software systems that we build. Smaller teams of people with a broader set of skills and responsibilities are what we need to get the job done.

That is just so awful...

I was watching a video about Django, a Python powered web framework, when the presenter, Jacob Kaplan-Moss, made the following comment (at around 29:22):

I've actually seen a system, I shit you not, that has every single URL on the site calls a stored procedure in an Oracle database that generates html in the stored procedure and returns it to the web for display. I'm not joking! That is just so awful I can't even start to begin...

That comment took me back a few years where I encountered just such a mess. I was brought into the organization to introduce Java and J2EE but the developers that lived there already had this Oracle solution in place. For reasons that I still don't understand to this day, they couldn't see that this was one incredibly awful idea. In any case, the next time you find yourself looking at some ugly bit of Java or Ruby or whatever, just remember it could be a lot worse!

Billion Dollar Boondoggle

The "Canadian Firearms Program" more widely known as the "Canadian Gun Registry" was a program instituted by the former Liberal government of Canada under Prime Minister Jean Chretian to license and register certain types of firearms across Canada. Bill C-68 was introduced in February 1995 and has been controversial ever since for a number of reasons but undoubtedly the issue that has caught everyone's attention has been the cost of the program, now over $1,000,000,000, and growing.

So leaving the politics behind, I was interested in the technical details behind this program from a software developer's perspective. What exactly did these guys do wrong that would result in such astronomical cost overruns? What technologies did they use and what could be learned from their mistakes?

Well it was surprisingly difficult to find information but finally I came upon a two-year-old article in eWeek titled "Canada Firearms: Armed Robbery". The system was originally comprised of an Oracle 7 database and a PowerBuilder application built by EDS. Given what I know about those technologies, I doubt that the problem was the technology. Instead, it appears that the biggest problem was incorrect initial assumptions and ballooning, ever-changing requirements.

But after eight years of mismanagement what was the solution? That's right! They decided to rebuild the application with a different set of technology and consulting resources (CGI)! Maybe the implementation really was poor, but I suspect some incompetent management types were deftly pointing fingers elsewhere and some unwitting bureaucrat believed them. Ahhh that old story...

DAOs and ORMs

The discussion that this article on the ServerSide generated reminds me of an article I wrote a little while ago. The original poster gets a bit confused about the problem because he assume the use of an ORM means that you need to use the "Open Session In View" pattern. But if you get by that and read some of the comments, you discover that many people (like me) don't believe that the DAO pattern is necessary when using an ORM solution.

A valid concern about ditching DAOs is the coupling that may happen between your domain objects and your persistence mechanism. But if you like ActiveRecord or Rich Domain Models or Annotations or XDoclet then you've already made the choice of productivity over ultimate framework plugability and should just get over it.

But if the coupling is still nagging you and you're concerned about sprinkling framework API calls throughout your code then maybe you want to consider looking at EJB3. Instead of using the vendor's API (e.g., Hibernate, Toplink, etc.) you can reference standard annotations and classes that are free of implementation specifics. At least that way you've provided some hope of switching out implementations.

In the end, question why you're using the DAO pattern and then ask yourself if you've already done something in your code that has violated that pattern. Think about your use of OpenSessionInView filters or the navigation of lazily instantiated collections in your view layer. Maybe DAO and ORM really don't mix...

XML Glue

When I discovered XDoclet I quickly stopped editing my hibernate mapping documents by hand. From the XDoclet website:

XDoclet lets you apply Continuous Integration in component-oriented development. Developers should concentrate their editing work on only one Java source file per component.

This approach has several benefits:

1. You don't have to worry about out dating deployment meta-data whenever you touch the code. The deployment meta-data is continuously integrated.
2. Working with only one file per component gives you a better overview of what you're doing. If your component consists of several files, it's easy to lose track. If you have ever written an Enterprise Java Bean, you know what we mean. A single EJB can typically consists of 7 or more files. With XDoclet you only maintain one of them, and the rest is generated.
3. You dramatically reduce development time, and can concentrate on business logic, while XDoclet generates 85% of the code for you.

But unfortunately I still find myself editing a ridiculous amount of Spring XML by hand. Spring sounds nice in theory but after using it for awhile I don't think there's anything wrong with the factory and singleton patterns. This unholy quest for ultimate flexbility by gluing components together with XML has to stop. Most of the time you simply don't need it.

On Frameworks and Top Management

I couldn't have said it better myself:

Frameworks

Top Management

MS Mac Lab

A friend of mine just sent me a link to a tour of Microsoft's Mac lab. Thanks Aaron... Thought I'd file it here under the category of "Holy Macs!".

Embracing Change

When I consider the "agile software development methodology", a key principle that comes to mind is "embrace change". The theory is that users don't really know what they want. They'll only be able to steer you to the goal once they see something working. So get ready for change. They'll change their minds a bunch of times as the software evolves. In the end they'll appreciate your flexibility to adapt the software to their changing vision and you'll be a happy developer because you've created something that will actually be used and appreciated by the end users.

But what happens when it's your team leader who keeps changing the technologies under your feet? Is it just another kind of change to be embraced? Let's see...

The elaboration phase of the unified process can be defined as: "This phase attempts to create an executable architecture that realizes architecturally significant use cases." In other words, we should select the appropriate technologies that will be needed to create a solution for the most difficult known use-cases and then actually prove that our chosen technologies will do the job by creating executable software. Then during the construction phase, we should be ready to tackle the problems (known and yet to be discovered) with the technologies in hand.

A second core principle of the "agile software development methodology" is "You aren't going to need it", which means you shouldn't overengineer something because you think you might need it in the future. Keep it simple. The idea is that you save time by not doing it now and you make your code better and more maintainable because it's simpler. If someone ever says to you "if we build it in now, it'll save us time later" and you already have too much other work to do right now, then tell them to forget it.

I'm currently in an environment where every "senior" resource is on a mission to turn the management spotlight on themselves and the way they try to achieve that goal is to way overengineer the solution with as many Java frameworks as possible. By doing so they can prove their technical merit with a complex solution. To give you an idea of what I mean consider this:

We have a framework for dependency injection, we have one for database persistence, we have one for role based security, and one for binding XML, we have one, no wait two (or maybe three) for doing UI. We're in the process of adding yet another Java web container (without retiring the previous two) and have added every conceivable open source code measurement and style checking software. We have two continuous integration servers and in addition to the JUnit testing framework, we're adding Ruby/Watir testing too...

Individually and maybe with some intelligent selection I don't have anything against most of these frameworks (except for Spring) but after six weeks of development with 9 programmers with only a login screen to show for it, I say enough is enough. Java web development may not be as productive as it is with Ruby on Rails, but it's not this tough!! Come on! Look at the requirements and choose proven frameworks that address problems you've identified and don't cost more to implement, learn and maintain than it would cost to create a custom solution. For example, if you have to spend more than a couple weeks implementing a role based security framework when nobody has actually said you need to worry about different roles, then you're probably doing something very wrong. Keep it elegant, efficient and simple.

Apple Listens to Customers

Well don't ever let it be said that Apple doesn't listen to its customers.  They released a new beta piece of software called "Boot Camp" that will allow people to dual boot Windows XP and Mac OS X on Intel based Macs.  I can't imagine that many self-respecting Mac owners would prefer to use Windows over Mac OS X, but there are undoubtedly occasions when it would be beneficial to do so.

Now Apple isn't being completely selfless here of course.  They make a lot of money selling hardware and if they can convince Windows users to buy a Mac instead of a Dell they'll be more than happy to accomodate them.  Maybe this is like a Trojan horse to get people to experience a good OS.

Next thing I'd like to see is the ability to run Windows in some kind of virtualization environment within Mac OS X...

Fast and Furious

I'm sure I have goofy speech patterns and favourite catch phrases of my own but I also have a talent for picking these colloquialisms out of someone else's speech as well. I have discovered over the years that the best time to capture these beauties within an IT environment is during the inevitable "status meeting". It's almost like a freestyle rap battle where two or more contenders try to outdo each other for status meeting supremacy. Fortunately for me, the agile process inspired daily standup meetings on my new project have become a veritable gold mine for these things. We have a few people on the project who are black-belt kung-fu masters of the colloquialism spinning these out with speed and grace... Here's a sample of the ones I've captured so far:

up to speed	high level design
make it real	snapshot
visionary thing	anecdotal evidence
identify the need	lay the groundwork
cycles available	disparate user experience
bottom feeder	look and feel
production capacity	nebulous
context	numerous permutations
drivers	shock effect
more or less	impact
grease the skids	fresh set of eyes
steer going forward	engage you
take forward	externalize
put the spin on it	socializing it
draft rollout plan	show stopper
iterative process	mitigate risk
right hand man	extra pair of hands
duck is cooked	messaging protocol
key artifacts	validation effect
verbage	sooner rather than later
put lipstick on it	technical components
bring to the table	consider the options
test bed	full meal deal
iterative cycle	lock it down in stone
instant gratification

Misplaced Priorities

I've worked on quite a few software projects in my career and I always wonder at the people who get hung up on their favorite features regardless of importance to the actual users of the system.

A few years back I worked with a developer who became analyst. At some point in one of his past projects he had created a feature for allowing a user to press some key combinations to "hot jump" from screen to screen. Although he never really elaborated I suspect this feature was more than just the typical shortcut keys you see in most desktop applications. But that's not the point. He became convinced this was THE feature to add to our app. Over the next few weeks he became fixated on "hot jumps". We kept hearing about "hot jumps" over and over. This was going to be the "most used feature" in the system.

But the other developers and I didn't agree with him... We felt "hot jumps" had absolutely no value unless the destinations of these hot jumps actually worked! And that was the problem and the point of this blog entry. We had tight deadlines, and a lot of actual business functionality to develop long before features like "hot jumps" should ever have entered anyone's mind. I'm not saying "hot jumps" would have been completely worthless, but at the time, it was undoubtedly a misplaced priority.

So it's with some surprise (and frustration) that I find myself in a similar situation today, only it's not "hot jumps", it's directory structures, eclipse project files, and (lord help me) Ant scripts. Under normal circumstances I would definitely say these things all have some importance, but what about actually developing the application? We're three weeks into development with five developers but under the direction of our "leader", I find myself spending my entire day renaming things and begging the "gods of Ant" to actually let me get this damn build.xml file right so it will do what I want. And gentle reader this is not just your regular run of the mill ant script. This is the mother of all ant scripts, utilizing every possible ant task available. And let me clarify, I did not write it. I inherited it and have been instructed to "make it work".

But my point again is this, what good is this "hot jump", oops, I mean Ant script if it doesn't have anything to build? At the end of the day, will the users be impressed with the quintessential Ant script or would they actually like to have the features they asked for?

DHH Interview

I just found a SYS-CON.TV interview with David Heinemeier Hansson that had some real quotable moments:

In his description of Ruby On Rails:

...we like to consider ourselves the Apple of open source web application development...

...it's extremely conventional. Most of the ideas in Ruby on Rails are the same ideas people have been building web applications on for the last ten years. They just hurt a lot less...

On Convention over configuration:

...In Java we call it XML sittups. You can produce just as much XML configuration as application code and that's just insane in our mind...

Does it scale?

...unquestionably yes. simply because Ruby doesn't invent a new architecture. We use exactly the same architecture as people who have been building the biggest sites on the internet.... shared nothing...

On the value of Rails.

[People think] it's so valuable to have extreme performance when there are tones of other attributes that are more valuable: programmer productivity, how maintainable your application is over time, ...

Ruby jEdit Plugin

I've been a big fan of jEdit for quite a few years now and I just downloaded a new plugin that I just simply felt compelled to write about, the jEdit Ruby Plugin. It's not the simplest thing to install but the integrated code completion and documentation features (including Ruby on Rails) are quite good. For example, when I'm editing a model object I can type in "has", hit ctrl-space and get a popup with the options "has_one", "has_and_belongs_to_many", and "has_many" options, and if I'm displaying the Ruby Docs window (currently docked to the right side of my jEdit window), it will display the documentation for each option as I select each one. Very nice. Perhaps some competition for RadRails?

Agile or Traditional?

I was recently speaking with a friend I haven't seen in months and she mentioned that she was reading a book I had lent to her. The book is called "Balancing Agility and Discipline" by B. Boehm and R. Turner. The book's main theme is the attempt to balance agile software development methods with traditional methods. The idea is to try and assess your project by looking at several different characteristics and then choose the processes from each method that will allow you to best manage it.

So that idea got me thinking about my most recent project, a Java rewrite of an existing application, and how desperately we were trying to use a pure agile methodology when it didn't really fit.

Here's a quote from a presentation available on the "American Institute of Aeronautics and Astronautics" website:

Problems characterized by change, speed, and turbulence are best solved by agility.

• Accelerated time schedule combined with significant risk and uncertainty that generate constant change during the project.

Is your project more like drilling for oil or like managing a production line?

• Oil exploration projects need Agile processes.
• Production-line projects are often well-served by rigorous methodologies.

Our project was undoubtedly a production-line application with very low risk:

• Although I didn't really like the technology we used on this project, it was being used to some degree of success on two other projects. So while it was undoubtedly not very productive, it was usable.
• The database design and user interface design were to be lifted untouched from the existing application.
• We were mandated with reproducing an existing production application so the requirements were well known if not well documented.
• Business users were not co-located and not easily reached for consultation (during the entire project I never spoke to a single person who would actually use it).

So what drove us to try and use an agile methodology? Well.... it's trendy. Agile is "in" and waterfall is "out". IT people would like to live in a world of absolutes, where we can say with certainty that A is better than B. But the reality is that we live in a world of spectrums where things are classified in terms of a position on a scale between two extreme or opposite points. Just like we shouldn't try to develop a simple project using every conceivable J2EE technology and framework available, maybe we shouldn't try to apply every agile process available where it just doesn't fit. Sometimes it's okay to create a plan and to write down requirements. Sometimes it's the best thing to do.

Suffering

One of the Pragmatic Programmer tips is:

"select" Isn't Broken.

Which means that's it is rare to find a bug in the OS or the compiler, or even a third-party product or library. The bug is most likely in the application, i.e., my code.

The problem with the project that I'm working on is that there is so many layers of code and abstraction that I don't even work with the framework anymore but some class ten times removed.

For example I have a JSP calling a JSF configured backing bean, calling a Spring configured service, calling another Spring configured EJB in a different process, calling another Spring configured service calling another Spring configured DAO which is (eventually) a subclass of Spring's HibernateDaoSupport class which attempts to hide Hibernate's API altogether.

So while I try to take the tip seriously and not blame the framework, there is so much stuff happening before it even gets to Hibernate (and let's not forget all that XML goo that glues everything together) that I can't help but lay some blame at the feet of these frameworks that encourage so much obfuscation.

To get real geeky I'll quote C3PO from the original Star Wars movie:

We seem to be made to suffer. It's our lot in life.

Bruce Eckel on Ruby and Rails

I just listened to the latest podcast by the Java Posse; an interview with Bruce Eckel.  Since it's the "Java" Posse there's a fair amount of discussion about Java, but the second part of the interview dives into Bruce's thoughts on Python, Ruby and Rails.  Although he gets some things wrong, like attributing the creation of Ruby to David Heinemeir Hannson (DHH) instead of Yukihiro Matsumoto (Matz), he largely says a lot of good things about Ruby and Rails and admits where he lacks sufficient knowledge to comment.

The one part in particular I agreed with was his assesment of the Rails supporters as being "loud", as in "loud-mouthed".  He says it's a lot like the early Java days.  As a matter of fact, my first taste of Rails was a presentation by DHH.  His rhetoric in that speech was a bit much to bear, but if you can see your way past the hyperbole, and just see that the guy is truly passionate about creating "beautiful" code, you get a good idea of why Rails is the way it is.