Whatever happened to distributed operating systems?

  by Andy Ormsby | 27 Jun 2007 | Permalink | Featured, Hardware, IT Management, Virtualization

As we all know, the cost of servers has fallen dramatically over the last few years.

Simple economics tells us that when prices fall, demand rises. Sure enough, data centres around the world are now running out of space, power, cooling or all three. If you can cope with the cooling and power requirements, it really is amazing what you can get: with IBM blades, for example, you can get 84 respectably powerful dual processor servers in a single industry standard 42U rack.

Naturally plenty of people are delighted by this glut of computing power, not least software vendors that sell their products licensed by CPU. However, others are justifiably concerned by overheating data centres, the waste implicit in low utilization rates and the management headache of simply looking after all those servers.

Virtualization is seen by many as an attractive solution. Take the applications currently running one-per-physical-server and put run them on virtual servers consolidated onto a smaller number of physical hosts. Problem solved.

The trouble is that the solution is not quite so simple. It is certainly possible, even likely, that utilization rates will rise and that data centres will get chance to decommission some hardware, and that is clearly a good thing. But the cost of the (now virtual) server will fall further since it will no longer be tied to a lump of tin and the inevitable result will be the further ballooning of the number of servers (and operating system instances) that IT has to manage. Deploying a new application? Put it on a new virtual server. In fact, put it on several in case one fails (and hope they are on different hosts). IT organisations already hurting from having to manage too many servers will get even more to worry of them to deal with.

Tying applications to servers (even virtual ones) seems like an odd thing to want to do. The “server” abstraction is an accident of the history of operating system development. Why preserve it in a virtualized world? Why does an application need to be “installed” on a specific server at all?

In the context of a large data centre, it is surely far better that the application can run on any of the servers, as and when required. Instead, the thrust of much virtualization effort seems to be in bundling the application with its own operating system instance and then shipping that around as required. This seems like a lot of work to get an application running on a box. Am I the only person that thinks this might not be the best way forward?

Sadly it seems that we now need to either wait for Linux or Windows to catch up with the 1980s state of the art in distributed systems (think Locus or AFS). What went wrong? Products like DataSynapse’s FabricServer look like an interesting attempt to address the problem, at least for the Java world, but it feels to me that mainstream operating systems designers seem to have lost the plot somewhere along the way.