Doing big things with lightweight architecture

This post was also published in InfoWorld.

Quite a few folks are beginnning to realize that most big websites, including Yahoo!, Google and, run on lightweight architecture. To define lightweight architecture, it is helpful to define its opposite:

Heavyweight architecture means you are running complicated infrastructure software like J2EE with complicated API's on a small cluster of expensive SMP machines.

Lightweight architecture means you are running straightforward, open source software stacks with service oriented API's on large clusters of commodity machines.

There are four common ways of achieving a lightweight architecture. Three of them are open source solutions, and the fourth is Microsoft's attempt:

  1. LAMP - As many of you know LAMP is my favorite lightweight stack.
    LAMP runs a vast majority of the massively scalable websites out there, and is also the favorite deployment stack for most of the "Web 2.0" crowd, including Friendster, Facebook, MySpace, and Flickr. Both IBM and Oracle announced support for PHP in 2005, and there has been an upswing of enterprise adoption of the LAMP stack.

  2. Open Source Java - A lot of Java developers are moving away from full blown J2EE to lightweight Java, which includes Tomcat, Spring, and Hibernate, among other open source Java projects. There are some big websites running this architecture, including E*TRADE and EBay. Lightweight Java is built on open source projects, which are a very different set of Java APIs than J2EE and are not officially sanctioned by Sun or the JCP.

  3. Ruby on Rails - RoR has gotten quite a bit of traction with the intelligenzia but none yet from enterprise customers or massively scalable websites. They only just shipped version 1.0 of the Rails framework in December and some people are complaining because it is a code generator. However, it is a cool scripting lanuage and a nice framework and a nice addition to the lightweight trend.

  4. Skinny .Net - Companies like JetBlue and Charles Schwab are starting to make very selective use of .Net so that it is more lightweight. Windows was always meant to run on commodity machines, but it is challenging to get it to run on large clusters or grids of commodity machines. On the other hand, .Net offers the best tools in the business and great XML support.

Each of these four approaches differ with each other, but they are remarkably consistent when when contrasted with the old way to do things. Clearly lightweight architecture is building in momentum and helps you do some very big things that can't be accomplished with older technology!