Rails 1.2 was recently released, and I've been updating and upgrading Pikaplanner to take advantage of some of the new features in 1.2.

The highlight of Rails 1.2 is built-in REST support, and I've been updating my controllers to provide what amounts to a standard, easy-to use, web-services API.

My first step in the process was to work through some of the examples in Rob Orsini's terrific developer volume Rails Cookbook. Diego Scataglini's examples in Rob's book provide a simple and quick intro to REST support in Rails 1.2.

REST is a great example of Clayton Christensen's concept of a disruptive technology. I've worked with SOAP and RPC in the past, and at first blush REST is less powerful than either. Still, with Rails, REST is easy and cheap to adopt at the service-provider end, and easy to experiment with at the receiver end.

For example, one of the first code areas I worked through was a simple product listing. The goal here is to provide both a web (HTML) and web services (XML) interface to our (sample) product information.

The REST support in Rails makes it easy, and all we need to do to turn our web pages into a web-services interface is tell (though the URL) our application that we want XML (instead of HTML) back.

So: to see the web page listing our products, we'd use "/products", as shown below:

So far, so good, so common—in Rails we've called up pages like this in the ":controller/:action/:id" form (that defaults to a GET on index.html here) zillions of times. What's new, with REST and Rails 1.2, is that all we have to do to have a web services interface is tell the application we want XML back, instead of HTML. So our URL is of the form ":controller/:action/:id.xml", as shown below:

and as you can see we get XML back - a web services interface practically for free (developer time-wise).

There is a lot of magic underlying this simple example, but a handful of key points stand out:

1) All we need for our "web services" interface is to tell our controller we want XML back.

2) We could have gotten JavaScript back (where that is appropriate) in the same manner

3) The key shift in development mindset is that the interface is not imperative (as SOAP and most web services are) but declarative—everything important in the system a) has its own URL for CRUD access, b) exists as a stateless resource, in which complex actions can be created by changes in state of one resource triggering actions to change the state of other, related resources, and 3) is capable of varying the format of responses, depending on what is requested.

That third point is a mouthful, and the core of stuff on which theses have been written.

Declarative programming is a significant mind-shift, one I'll be writing more about in the future.

One of the unsung facets of working reasonably up in the org structure of a large, successful tech company is that you get a unique view of how the tech media machine works.

Tech writing (like Tech Marketing) is hard, because 1) Any sufficiently advanced technology is indistinguishable from magic, 2) Only magicians really understand the magic, and 3) Magic skill != writing skill. So, back in those ol' days, I wasn't that surprised to find that media pieces often originated in the Marketing departments of big companies, but I am always surprised when media outlets appear to run the Marketing feed unedited.

As a sign that some of the old ways still prevail, today we have the following two, loosely-coupled headlines: Vista success hinges on developers and Developers take advantage of Vista. A nice two-headline tautology.

I'm sure both headlines are valid, but looking at things that shallowly completely misses what's going on in software right now. OEM Agreements guarantee Vista's success - things might be better if developers come aboard, but Vista should do just fine (on every new machine shipped) even if they don't.

This was probably true even back in 2001, when Microsoft CEO Steve Ballmer famously cried "Developers! Developers! Developers!", but it really doesn't bear up to scrutiny today. In 20-odd years of mass-market windowed software development, Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.") - has basically held, and now even the simplest "mainstream" apps offer a staggering amount of functionality.

In the desktop software world, probably 95% of all usage is performed using about a half-dozen applications (Word, Excel, PowerPoint, Outlook, Browser, Itunes, IM Client). Even if all the developers stay home, that still means that Vista will lose functionality only at the remaining 5% margin.

There are two things worth drawing from the articles: 1) We'll see better software on the Vista platform if developers flock to Vista, but Vista's success does not depend on that flocking, and 2) if you are writing software today you have to be mindful that most (95%, if you accept the guesstimated figure above) of the "horizontal software space" is already filled.

That's not to say that there's not room for great new software, only that great new software isn't going to take a piece of the current pie—the only way to create great new software is to make the pie bigger.

In a later post, I'll talk about models for doing just that.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. — Antoine de Saint-Exupery

I've always liked Clayton Christensen's book "The Innovator's Dilemma" as a metaphor to explain product innovation and succession. To vastly oversimplify the main idea, companies get so good at "hammering" that everything: screws, rivets, staples, welds, etc. is either treated as a "nail" or crushed in favor of "nails."

Software is an industry that produces natural monopolies, and so for software The Innovator's Dilemma applies, not just to individual companies, but to the entire industry.

This makes things really hard for software developers today. Microsoft, with Windows started out as a system for non-networked individual devices, and has tried to expand all the way up to clustered 'mainframe' solutions. Enterprise Java started out trying to boil the ocean, boiled it, only to find that ocean-boiling solutions aren't of much use when all you want is a cup of hot water for soup. In trying to solve the same universe of problems, both the Microsoft and Java camps have adopted the same set of ills. One size doesn't fit all.

In this case of two increasingly over-engineered systems, I prefer openness, not because of what you can add, but because of what you can take away. The key ratio for software managers to watch I call the "overhead ratio" which is (Time spent on the solution the customer will pay for) / (Total time spent). The ideal solution would have a quick installation, rapid configuration, and rapid user iterations to a state of solution-ness that is "just right." The ideal development effort will spend all its time on "value-add" components, and none on "configuration."

Both Microsoft and Java development have a pretty onerous overhead ratio - getting all the components right takes an increasingly long time. As I've previously noted, I like "virtual appliances", and the "Convention, not Configuration" mantra of Rails/Grails for just this reason.

But there's more to it that that, a notion that I'll posit as The Godfather's Law: "Technology components that reach a sufficient level of irritation are made to SIMPLY DISAPPEAR." Credit Mario Puzo with this: "As soon as the Corleone Family set up their usual business liaison with the local police force they were informed of all such complaints and all crimes by professional criminals. In less than a year Long Beach became the most crime-free town of its size in the United States. Professional stickup artists and strong-arms received one warning not to ply their trade in the town. They were allowed one offense. When they committed a second they simply disappeared."

Software licensing is a headache? BING - SaaS. MS Shop that doesn't run Linux? BANG - Virtual appliances. No (MS) or overcomplicated (EJB) ORM? - BOOM - Hibernate.

...and so on. As the best software people are "Lazy, Impatient, and Hubristic" (a description that also fit Sonny Corleone), no system of excessive overhead ratio will last long. One key to software success is to figure out when a paradigm is about to get "bumped off", and to move in on all the action when it happens.

There is a great opportunity for software development organizations who can surf the transition. The key is not in choosing the "next Java" or hot development environment, but in identifying all of the barriers to adoption and making them "simply disappear." Even if they might agree on nothing else, the two "C's" (Christensen and Corleone) would agree here.

Programming platforms are like holiday dinners - to succeed they have to offer something for everyone, and if they/you sample a bit of everything then they (like you) are likely to end up bloated.

Such is the case with Java, a pretty language with clean syntax that, in its full Enterprise-banquet form, has been compared with eating an elephant. Enterprise Java offers everything, and development teams have to make a lot of technical calls before the first line of code is ever written: EJB or JDO? bean-managed, container-managed, or framework-provided persistence? EJB 3.0 or JDO 2.0? Struts or JSF or Spring? The wrong call, of merely the passage of time can make for pretty hard-to-manage code.

Still, Java's been used in enterprise development for years now; it has a large library of code and many trained developers and all of this makes for a pretty appealing banquet-table. What is a development team to do?

A common answer lies in the realm of "opinionated code," in which the development team chooses a set of beliefs, and architecture follows. Ruby on Rails is a good example of opinionated code: everything, from default directory layouts to ORM framework (ActiveRecord) to dependency injection is baked in.

RoR is great, but it doesn't address the great banquet of Java development and platform that's evolved over the past nine years. Enter Ruby-on-Rails-for-the-Java-set: Groovy (the language) and Grails (the platform).

Groovy is a dynamic language based on Java and strongly influenced by Ruby, and Grails is a JEE-spiced framework modeled-on and very similar to Rails. Groovy reached it 1.0th birthday this week, and Grails is nearing it's 0.4 release, so the platform isn't particularly mature yet, but has solid financial backing and is already being explored in "mission-critical enterprise applications".

"So what?" you say—is this instant relief for enterprise development, or a "wafer-thin mint." My first experiences with GoG indicate the former. Setup is fast, and the initial development environment provides ORM (from Hibernate), inversion of control and MVC (from Spring), logging (log4j), and layouts (from SiteMesh). This is opinionated software, the decisions are made and many of the patterns that have made RoR popular (DRY, convention-over-configuration, etc.) are baked in to GoG as well.

Groovy and Grails may not be ready for full enterprise deployment, but the platform is easy to grasp for both Java and Ruby developers, provides much of the development speed and elegance of RoR, and embraces the libraries, seamless integration-to and feel-of JEE. It should be a great platform for prototypes and rapid JEE development, and 2007 should reveal whether it will mature to an enterprise-ready framework.

Tastes great, less filling...

Make it 12! Predictions and Snowfall

Might as well start with the snowfall first: another twelve inches on the ground today. We've fallen into a pattern - Thursday = storm starts, Friday = storm ends and some chunk of the day is spent digging out. Only a foot today, so maybe the weekly effect is damping out.

But on to my final prediction for 2007 - the Year of the Software Appliance.

I'm working on Pikaplanner, a lean manufacturing application that's designed to be run in a hosted environment. I have a great setup for it, with custom gems installed and a beta version of a hyper-fast Ruby virtual machine. It works great for me, but moving it to a hosting provider has proved problematic.

At this point in our malware-infested world, installing anything new anywhere is a risky proposition. Installing enterprise software can also be time-consuming: back in the day for a decent-size Oracle apps installation you would want to set aside a full week just to get the software installed.

All of these problems (custom environments, malware, and difficult installs) are remedied by the concept of a software appliance. Illustrative of the SA concept, Digium this week announced the release of it's Asterisk VOIP PBX as a software appliance. Asterisk is a terrific product—an open source (free as in beer) IP switching system. This monumental breakthrough in cheap communications systems has seen rapid growth, limited only by the difficulty of getting linux and telephony setup and configured by mere mortals.

Enter the software appliance - a bundle that includes the operating system, all the extra packages needed, and the VOIP switch software all pre-configured and delivered to be run virtual player environment. A tricky install becomes a simple, 30-minute exercise.

This is revolutionary—VMware (popular maker of virtualization products, recently acquired by EMC) products are generally already well-accepted in corporate IT departments, and encapsulated applications take a lot of the risk out of new software deloyments. SO ... if you want my Pikaplanner (complete with customized environment), all I need to do is package it up, lock, stock and barrel, and deliver it as a software appliance. The "tricky install" and "customer environment" problems are solved in one fell swoop.

A software appliance is completely contained within its virtual environment, so if you're worried about security, just throw the appliance away and start again!

That's really only the beginning. If you write linux-based applications and need to deliver them in a Windows-only environment, just package up an appliance and run it in a virtual space on that Windows machine.

I can barely scratch the surface here, but VMWare turbocharged the software-appliance idea with their VMWare Player, Xen is white-hot in pursuit of the same idea, and the force of it is so powerful that even Microsoft has to conform to it..

Billy on Open Source has some terrific writings on the software appliance idea. There's a lot more to discuss here, so for now I'll leave my final prediction that "2007 is the year of the software appliance."

Prediction: APIs rule, and REST will come to rule APIs in 2007.

Google rules the Web 2.0 world. They are the alpha and the omega, from Google Maps (which popularized Ajax and got the whole thing started back in 2005) to being the primary acquirer and exit strategy for many Web 2.0 companies. Google is also the whole alphabet in between, and their publishing of APIs for their applications re-ignited the "mash-up" concept from Web 1.0, and changed the face of web apps we see today.

Google is wise and omnipresent, but they aren't omniscient. Back in 2002, when the current round of API design decisions were made, Google had the choice of creating an API in well-known media-darling SOAP, or the little known academic paper-protocol REST. They chose SOAP. Not that SOAP was such a bad choice, but with the benefit of hindsight Google is now heading in another direction, and Yahoo has already reached the promised land.

SOAP (originally Simple Object Access Protocol) was a neat idea—to replace the bulk and complexity of integration schemes such as CORBA with a simple combination of XML and HTTP. Great idea, but to provide fully-functional enterprise integration SOAP had to expand, eventually absorbing much of the complexity of the protocols it meant to replace.

Enter REST. Representational State Transfer is the brainchild and 2000 PhD dissertation of Roy Fielding. Fielding observed that one of the great advantages of the HTTP specification (of which he was also a contributor) was that the client-server, stateless, cacheable, and layered design made access and architecture for the specification straightforward. REST extends these concepts to application-application communication. Very broadly, REST maps the basic CRUD operations (create, retrieve, update and delete) to familiar HTTP operations (POST, GET, PUT, and DELETE). As an additional conceptual benefit, these operations also map analogously to the database operations INSERT, SELECT, UPDATE, and DELETE.

Boiled down, the idea is to have applications interact through conceptually simple HTTP access for exchange of resources—remote resources, as opposed to remote procedure calls. API creation is then a breeze, because the access methods are already broadly familiar, and the receiving applications need only be ready to respond to requests based on the request information in the HTTP header - say for HTML (web pages), JavaScript (Ajax requests) or XML (application requests). Ruby on Rails has good REST support now (through the SimplyRestful plugin) and will have REST as a part of the core going forward.

In the Ruby/Rails world, URL's are beautiful and informational, and one of the downsides of REST are URLs that are more machine- than human-readable. Still, as applications are increasingly designed with integration and mashup in mind, you can expect REST to rise in prominence in 2007.