Pikasoft

This is not a Pipe

john.repko@pikasoft.com (John Repko) — Sat, 10 Feb 2007 02:51:00 GMT

Every once in a while, something happens that makes me completely reassess my take on "what's next" in the technology sector. The original Macintosh was one such epiphany, Apple's Hypercard was another (Hypercard is a much-unappreciated program - had Apple added network links to it we'd have had a writeable, scriptable WWW in 1989). Netscape's first browser was a third. On Thursday, the release of Yahoo! Pipes shook my world.

The Pipes slogan is "Rewire the web", and that's what it purports to do - as a visual editor for RSS feeds that dramatically lowers the time and expertise needed to create a "mashup". Pipes lets you create your own personal web channels, and Gina Trapani at Lifehacker has a great example of how to do this.

And so, if your interest is, say, the NFL draft, it's pretty easy to wire up all the draft-tracking sites so that if a potential 7th round pick out in East Treestump gets a hangnail, you'll know about it. But the implications go so much further...

In a REST-ian world, it's pretty easy, once you've modeled your world as a set of resources, to create RSS feeds that track the changes in those resources. With an application like Pipes, it's really easy to tie those feeds together into new applications. This is the future of "SOA" and "Web Services."

Forget SOAP, and the growing pile of WS-* standards as a means of wiring up disparate applications. REST-ify the applications, use RSS to publish the changes, and a tool like Pipes (or one of the imitators that should be quick to follow) to wire the whole thing up. SOAPians - put that in your pipe and smoke it!

Seriously, though, while the Pipes idea is great, there are still some issues to work through. The intuitive purity of the Pipes concept was proven when everyone from the merest script-kiddy to the deepest hacker jumped on Yahoo! and bought the Pipes site Down! all day on launch day. Yahoo! knows a bit about keeping large sites up under volume, so we are surely talking about a big human wave here.

But that's not all, either. XML is wordy, and XML transformation is slow and computationally intensive. I'm curious to see if the sheer processing load leads to more restricted (e.g. JSON) or more efficient/binary formats as a wire protocol for Pipe-y services.

I'm convinced these things will be worked out, and Tim O'Reilly's comments about Pipes as a "milestone in the history of the internet" will prove to be true.

As such, let this be my 14th prediction for 2007: February 7, the Pipes release date, will be looked back on as the day SOAP died. Not that it's bad, but the future has been shown to lie elsewhere.

RESTful Goodness - I declare!

john.repko@pikasoft.com (John Repko) — Wed, 24 Jan 2007 07:10:00 GMT

Rails 1.2 was recently released, and I've been updating and upgrading Pikaplanner to take advantage of some of the new features in 1.2.

The highlight of Rails 1.2 is built-in REST support, and I've been updating my controllers to provide what amounts to a standard, easy-to use, web-services API.

My first step in the process was to work through some of the examples in Rob Orsini's terrific developer volume Rails Cookbook. Diego Scataglini's examples in Rob's book provide a simple and quick intro to REST support in Rails 1.2.

REST is a great example of Clayton Christensen's concept of a disruptive technology. I've worked with SOAP and RPC in the past, and at first blush REST is less powerful than either. Still, with Rails, REST is easy and cheap to adopt at the service-provider end, and easy to experiment with at the receiver end.

For example, one of the first code areas I worked through was a simple product listing. The goal here is to provide both a web (HTML) and web services (XML) interface to our (sample) product information.

The REST support in Rails makes it easy, and all we need to do to turn our web pages into a web-services interface is tell (though the URL) our application that we want XML (instead of HTML) back.

So: to see the web page listing our products, we'd use "/products", as shown below:

So far, so good, so common—in Rails we've called up pages like this in the ":controller/:action/:id" form (that defaults to a GET on index.html here) zillions of times. What's new, with REST and Rails 1.2, is that all we have to do to have a web services interface is tell the application we want XML back, instead of HTML. So our URL is of the form ":controller/:action/:id.xml", as shown below:

and as you can see we get XML back - a web services interface practically for free (developer time-wise).

There is a lot of magic underlying this simple example, but a handful of key points stand out:

1) All we need for our "web services" interface is to tell our controller we want XML back.

2) We could have gotten JavaScript back (where that is appropriate) in the same manner

3) The key shift in development mindset is that the interface is not imperative (as SOAP and most web services are) but declarative—everything important in the system a) has its own URL for CRUD access, b) exists as a stateless resource, in which complex actions can be created by changes in state of one resource triggering actions to change the state of other, related resources, and 3) is capable of varying the format of responses, depending on what is requested.

That third point is a mouthful, and the core of stuff on which theses have been written.

Declarative programming is a significant mind-shift, one I'll be writing more about in the future.

Behind the Headlines

john.repko@pikasoft.com (John Repko) — Wed, 24 Jan 2007 06:23:00 GMT

One of the unsung facets of working reasonably up in the org structure of a large, successful tech company is that you get a unique view of how the tech media machine works.

Tech writing (like Tech Marketing) is hard, because 1) Any sufficiently advanced technology is indistinguishable from magic, 2) Only magicians really understand the magic, and 3) Magic skill != writing skill. So, back in those ol' days, I wasn't that surprised to find that media pieces often originated in the Marketing departments of big companies, but I am always surprised when media outlets appear to run the Marketing feed unedited.

As a sign that some of the old ways still prevail, today we have the following two, loosely-coupled headlines: Vista success hinges on developers and Developers take advantage of Vista. A nice two-headline tautology.

I'm sure both headlines are valid, but looking at things that shallowly completely misses what's going on in software right now. OEM Agreements guarantee Vista's success - things might be better if developers come aboard, but Vista should do just fine (on every new machine shipped) even if they don't.

This was probably true even back in 2001, when Microsoft CEO Steve Ballmer famously cried "Developers! Developers! Developers!", but it really doesn't bear up to scrutiny today. In 20-odd years of mass-market windowed software development, Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.") - has basically held, and now even the simplest "mainstream" apps offer a staggering amount of functionality.

In the desktop software world, probably 95% of all usage is performed using about a half-dozen applications (Word, Excel, PowerPoint, Outlook, Browser, Itunes, IM Client). Even if all the developers stay home, that still means that Vista will lose functionality only at the remaining 5% margin.

There are two things worth drawing from the articles: 1) We'll see better software on the Vista platform if developers flock to Vista, but Vista's success does not depend on that flocking, and 2) if you are writing software today you have to be mindful that most (95%, if you accept the guesstimated figure above) of the "horizontal software space" is already filled.

That's not to say that there's not room for great new software, only that great new software isn't going to take a piece of the current pie—the only way to create great new software is to make the pie bigger.

In a later post, I'll talk about models for doing just that.

Great Management Books: The Innovator's Dilemma, and The Godfather

john.repko@pikasoft.com (John Repko) — Thu, 18 Jan 2007 04:42:00 GMT

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. — Antoine de Saint-Exupery

I've always liked Clayton Christensen's book "The Innovator's Dilemma" as a metaphor to explain product innovation and succession. To vastly oversimplify the main idea, companies get so good at "hammering" that everything: screws, rivets, staples, welds, etc. is either treated as a "nail" or crushed in favor of "nails."

Software is an industry that produces natural monopolies, and so for software The Innovator's Dilemma applies, not just to individual companies, but to the entire industry.

This makes things really hard for software developers today. Microsoft, with Windows started out as a system for non-networked individual devices, and has tried to expand all the way up to clustered 'mainframe' solutions. Enterprise Java started out trying to boil the ocean, boiled it, only to find that ocean-boiling solutions aren't of much use when all you want is a cup of hot water for soup. In trying to solve the same universe of problems, both the Microsoft and Java camps have adopted the same set of ills. One size doesn't fit all.

In this case of two increasingly over-engineered systems, I prefer openness, not because of what you can add, but because of what you can take away. The key ratio for software managers to watch I call the "overhead ratio" which is (Time spent on the solution the customer will pay for) / (Total time spent). The ideal solution would have a quick installation, rapid configuration, and rapid user iterations to a state of solution-ness that is "just right." The ideal development effort will spend all its time on "value-add" components, and none on "configuration."

Both Microsoft and Java development have a pretty onerous overhead ratio - getting all the components right takes an increasingly long time. As I've previously noted, I like "virtual appliances", and the "Convention, not Configuration" mantra of Rails/Grails for just this reason.

But there's more to it that that, a notion that I'll posit as The Godfather's Law: "Technology components that reach a sufficient level of irritation are made to SIMPLY DISAPPEAR." Credit Mario Puzo with this: "As soon as the Corleone Family set up their usual business liaison with the local police force they were informed of all such complaints and all crimes by professional criminals. In less than a year Long Beach became the most crime-free town of its size in the United States. Professional stickup artists and strong-arms received one warning not to ply their trade in the town. They were allowed one offense. When they committed a second they simply disappeared."

Software licensing is a headache? BING - SaaS. MS Shop that doesn't run Linux? BANG - Virtual appliances. No (MS) or overcomplicated (EJB) ORM? - BOOM - Hibernate.

...and so on. As the best software people are "Lazy, Impatient, and Hubristic" (a description that also fit Sonny Corleone), no system of excessive overhead ratio will last long. One key to software success is to figure out when a paradigm is about to get "bumped off", and to move in on all the action when it happens.

There is a great opportunity for software development organizations who can surf the transition. The key is not in choosing the "next Java" or hot development environment, but in identifying all of the barriers to adoption and making them "simply disappear." Even if they might agree on nothing else, the two "C's" (Christensen and Corleone) would agree here.

Feelin' Groovy

john.repko@pikasoft.com (John Repko) — Mon, 08 Jan 2007 08:49:00 GMT

Programming platforms are like holiday dinners - to succeed they have to offer something for everyone, and if they/you sample a bit of everything then they (like you) are likely to end up bloated.

Such is the case with Java, a pretty language with clean syntax that, in its full Enterprise-banquet form, has been compared with eating an elephant. Enterprise Java offers everything, and development teams have to make a lot of technical calls before the first line of code is ever written: EJB or JDO? bean-managed, container-managed, or framework-provided persistence? EJB 3.0 or JDO 2.0? Struts or JSF or Spring? The wrong call, of merely the passage of time can make for pretty hard-to-manage code.

Still, Java's been used in enterprise development for years now; it has a large library of code and many trained developers and all of this makes for a pretty appealing banquet-table. What is a development team to do?

A common answer lies in the realm of "opinionated code," in which the development team chooses a set of beliefs, and architecture follows. Ruby on Rails is a good example of opinionated code: everything, from default directory layouts to ORM framework (ActiveRecord) to dependency injection is baked in.

RoR is great, but it doesn't address the great banquet of Java development and platform that's evolved over the past nine years. Enter Ruby-on-Rails-for-the-Java-set: Groovy (the language) and Grails (the platform).

Groovy is a dynamic language based on Java and strongly influenced by Ruby, and Grails is a JEE-spiced framework modeled-on and very similar to Rails. Groovy reached it 1.0th birthday this week, and Grails is nearing it's 0.4 release, so the platform isn't particularly mature yet, but has solid financial backing and is already being explored in "mission-critical enterprise applications".

"So what?" you say—is this instant relief for enterprise development, or a "wafer-thin mint." My first experiences with GoG indicate the former. Setup is fast, and the initial development environment provides ORM (from Hibernate), inversion of control and MVC (from Spring), logging (log4j), and layouts (from SiteMesh). This is opinionated software, the decisions are made and many of the patterns that have made RoR popular (DRY, convention-over-configuration, etc.) are baked in to GoG as well.

Groovy and Grails may not be ready for full enterprise deployment, but the platform is easy to grasp for both Java and Ruby developers, provides much of the development speed and elegance of RoR, and embraces the libraries, seamless integration-to and feel-of JEE. It should be a great platform for prototypes and rapid JEE development, and 2007 should reveal whether it will mature to an enterprise-ready framework.

Tastes great, less filling...

Prediction #12: Rise of the (virtual) machines

john.repko@pikasoft.com (John Repko) — Fri, 05 Jan 2007 11:32:00 GMT

Make it 12! Predictions and Snowfall

Might as well start with the snowfall first: another twelve inches on the ground today. We've fallen into a pattern - Thursday = storm starts, Friday = storm ends and some chunk of the day is spent digging out. Only a foot today, so maybe the weekly effect is damping out.

But on to my final prediction for 2007 - the Year of the Software Appliance.

I'm working on Pikaplanner, a lean manufacturing application that's designed to be run in a hosted environment. I have a great setup for it, with custom gems installed and a beta version of a hyper-fast Ruby virtual machine. It works great for me, but moving it to a hosting provider has proved problematic.

At this point in our malware-infested world, installing anything new anywhere is a risky proposition. Installing enterprise software can also be time-consuming: back in the day for a decent-size Oracle apps installation you would want to set aside a full week just to get the software installed.

All of these problems (custom environments, malware, and difficult installs) are remedied by the concept of a software appliance. Illustrative of the SA concept, Digium this week announced the release of it's Asterisk VOIP PBX as a software appliance. Asterisk is a terrific product—an open source (free as in beer) IP switching system. This monumental breakthrough in cheap communications systems has seen rapid growth, limited only by the difficulty of getting linux and telephony setup and configured by mere mortals.

Enter the software appliance - a bundle that includes the operating system, all the extra packages needed, and the VOIP switch software all pre-configured and delivered to be run virtual player environment. A tricky install becomes a simple, 30-minute exercise.

This is revolutionary—VMware (popular maker of virtualization products, recently acquired by EMC) products are generally already well-accepted in corporate IT departments, and encapsulated applications take a lot of the risk out of new software deloyments. SO ... if you want my Pikaplanner (complete with customized environment), all I need to do is package it up, lock, stock and barrel, and deliver it as a software appliance. The "tricky install" and "customer environment" problems are solved in one fell swoop.

A software appliance is completely contained within its virtual environment, so if you're worried about security, just throw the appliance away and start again!

That's really only the beginning. If you write linux-based applications and need to deliver them in a Windows-only environment, just package up an appliance and run it in a virtual space on that Windows machine.

I can barely scratch the surface here, but VMWare turbocharged the software-appliance idea with their VMWare Player, Xen is white-hot in pursuit of the same idea, and the force of it is so powerful that even Microsoft has to conform to it..

Billy on Open Source has some terrific writings on the software appliance idea. There's a lot more to discuss here, so for now I'll leave my final prediction that "2007 is the year of the software appliance."

Bonus Prediction: REST ye Merry!

john.repko@pikasoft.com (John Repko) — Tue, 02 Jan 2007 10:24:00 GMT

Prediction: APIs rule, and REST will come to rule APIs in 2007.

Google rules the Web 2.0 world. They are the alpha and the omega, from Google Maps (which popularized Ajax and got the whole thing started back in 2005) to being the primary acquirer and exit strategy for many Web 2.0 companies. Google is also the whole alphabet in between, and their publishing of APIs for their applications re-ignited the "mash-up" concept from Web 1.0, and changed the face of web apps we see today.

Google is wise and omnipresent, but they aren't omniscient. Back in 2002, when the current round of API design decisions were made, Google had the choice of creating an API in well-known media-darling SOAP, or the little known academic paper-protocol REST. They chose SOAP. Not that SOAP was such a bad choice, but with the benefit of hindsight Google is now heading in another direction, and Yahoo has already reached the promised land.

SOAP (originally Simple Object Access Protocol) was a neat idea—to replace the bulk and complexity of integration schemes such as CORBA with a simple combination of XML and HTTP. Great idea, but to provide fully-functional enterprise integration SOAP had to expand, eventually absorbing much of the complexity of the protocols it meant to replace.

Enter REST. Representational State Transfer is the brainchild and 2000 PhD dissertation of Roy Fielding. Fielding observed that one of the great advantages of the HTTP specification (of which he was also a contributor) was that the client-server, stateless, cacheable, and layered design made access and architecture for the specification straightforward. REST extends these concepts to application-application communication. Very broadly, REST maps the basic CRUD operations (create, retrieve, update and delete) to familiar HTTP operations (POST, GET, PUT, and DELETE). As an additional conceptual benefit, these operations also map analogously to the database operations INSERT, SELECT, UPDATE, and DELETE.

Boiled down, the idea is to have applications interact through conceptually simple HTTP access for exchange of resources—remote resources, as opposed to remote procedure calls. API creation is then a breeze, because the access methods are already broadly familiar, and the receiving applications need only be ready to respond to requests based on the request information in the HTTP header - say for HTML (web pages), JavaScript (Ajax requests) or XML (application requests). Ruby on Rails has good REST support now (through the SimplyRestful plugin) and will have REST as a part of the core going forward.

In the Ruby/Rails world, URL's are beautiful and informational, and one of the downsides of REST are URLs that are more machine- than human-readable. Still, as applications are increasingly designed with integration and mashup in mind, you can expect REST to rise in prominence in 2007.

Predictions for 2007

john.repko@pikasoft.com (John Repko) — Sun, 31 Dec 2006 12:17:00 GMT

Since a new year beckons, and this is a blog, it follows that predictions for the new year must follow. Hey, rules are rules. And so, with no further adieu, here are my predictions for 2007:

Easy Calls:

1) Microsoft Vista comes, new machines get it, nobody on XP upgrades. Microsoft will be content to turn over their installed base with new machines sold and old machines obsoleted. Otherwise, upgrades will be slower than currently forecast.

2) Apple delivers Leopard, new machines get it, but upgrades are slower than currently forecast. Tiger is sufficiently good, and external packages (e.g VirtueDesktops) simulate a lot of the new functionality. Apple has a big decision to make, between adding functionality for the IPod/digital home, and functionality for business and enterprise solutions.

3) Linux will continue plodding progress in '07, and boom in '08. Linux has a lot to recommend it - including good security, scalability, and cost. Its openness makes it impervious to horrible, enterprise-inappropriate additions (DRM), and has a huge leg up on 64-bit solutions. If Ubuntu can continue advancement of a plausible desktop system, Linux will start taking Mac users in '07 and MS users in '08.

4) Dynamic languages will keep making inroads on "traditional" (.Net and J2EE) development. Compared to compiled C++, Ruby is one slow language in execution, but there are *so* many available processing cycles that development savings dominate. It won't run Google or the would-be Googles, but it's fast enough for everybody else. The second wave of RoR apps will appear.

5) Ajax is here to stay, and full page refreshes will disappear in 2007. Good packages (such as Prototype), and the ease of integration in new environments (Ruby on Rails) make ajaxification easy for new applications. Existing apps will ajaxify or be replaced by ajax'd apps.

6) Google backlash begins. Past a certain size, it gets really hard to not "be evil." Google has passed that size, and the leading edge isn't willing to give them a pass anymore. Look for one new search engine to break on top of the pack.

Wild Swags:

7) Oracle buys SAP. Alexander the Great "wept, because he had no more worlds to conquer." Larry Ellison does not weep, and SAP is all that is left. The current permissive antitrust environment won't last forever, so Oracle goes Exxon-Mobil(e) in '07.

8) Indian IT giants start buying second- and third-tier US software providers. Outsourced development from the US to India is now commonplace; the next logical step is Indian vertical integration into the American market. Wilder swag: Wipro buys Lotus assets from IBM.

9) Google buys Ebay. See 7) above. "Froogle" has gone nowhere, Ebay got spat out of China. Google has innovative IT at scale, Ebay needs innovative IT at scale. Match made in heaven, and these kinds of deals won't find a favorable approval environment forever.

Final Swag:

10) Innovators rule. Vista is great looking, but is it a compelling upgrade if DRM restrictions give new users less control than they have now? Apple is also wading into DRM waters that make sense for IPods but not for enterprise-anything. Oracle has its hands full with Fusion, and if you like the JD Edwards, Peoplesoft or Siebel software you're running you might not want to Fuse in '07. As Tony Curtis once said: "In confusion there is profit." '07 looks like a great year for disruptive technologies and solutions.

More pictures of buildings and snow

john.repko@pikasoft.com (John Repko) — Fri, 29 Dec 2006 05:19:00 GMT

I spoke too soon about the "snows of December" being over. Eight days after the first blast, we have another 24", and an unshoveled depth of about four feet out back.

Again, good luck and best wishes to all holiday travelers, particularly the ones stuck in DIA or other airports around the country. Get home safe, and Happy New Year!

I'll be shoveling off now...

Three Feet Plus

john.repko@pikasoft.com (John Repko) — Thu, 21 Dec 2006 10:52:00 GMT

Well, the snows of December finally seem to be over. Five driveway shovelings and 39" later, the weather of our local world is finally at rest.

Good luck and best wishes to all holiday travelers, particularly the ones stuck in DIA or other airports around the country.

Get home safe, and Happy Holidays.