Service Oriented Computing and Tech Shamans

The Design and the Designer – Part 8

There is a lot of talk about Service Oriented Computing these days – mostly by people who don’t seem to understand what it really stands for.

Until now, we have very good models of infrastructure engineering and applications engineering, but we are yet to see a reasonable model for process engineering and structural engineering in the computing science. This could be because of the unfolding of services as a primary business model has just begun and we have not yet understood what service really is.

A very successful billionaire once told me that the Services Automation is several orders of magnitude more significant than the industrial revolution. The explanation he offered for this significance was very striking. It has the beauty of simplicity and the brilliance of Truth to it. According to him, the basis of industrial revolution is control of resources and centralization of knowledge and power. Who ever can control and marshal resources and centralize and guard knowledge becomes wealthy.

But, the services revolution is exactly opposite. Its basis is decentralization of information, power and knowledge. Its basic premise is that we should use resources as when required without really possessing them. Look at the business models of today, especially the Internet. Its basic principle is making information equally available to everyone, and make use of resources as when required without possessing any resource at all.

But, how is this possible? Why would the wealthy people - who became wealthy because of the exact opposite principle - allow such a thing to happen? Why would they want to let go of their control and let go of their wealth?

Well, the answer is that it was not made possible by the wealthy and rich business people. Most of the computing science and the consequent business models are the creations of enlightened hippies. They were the only people capable of such subtlety of thought and action. They even managed to get vast amounts of funding from the very same people that they resolved to dethrone them from their seats of power!!

Let's look at the evolution of computing science, its absorption by the 'industry' and the unseen hand of the hippy.

On one hand, there is IBM and all its 'industrial' representatives and on the other hand there were tech-shamans.

What did IBM and its representatives contribute to the computing science? They tried big, fat main frame machines, resource hogging, hierarchical operating systems like MVS, they tried unusable programming languages like COBOL and they tried to push for data modeling rooted in their three hundred years of arrogance. Now they are trying Java and J2EE. In the software engineering side, they perpetrated process centric methodologies. All these technologies, methodologies, disciplines have one thing in common - they all aim to make the process expensive, aimed to clear the "owner's" inventory and make their clients/customer pay a very heavy price. And, more importantly - they have some kind of 'lock-in' of the customer built into all of them. In the long run - all of them are very expensive - the price paid by people who adopted COBOL during the Y2K issue must be an eye opener. And, who spent the money to resolve the Y2K bug, and who earned it? The people who bought COBOL paid the price, and the people who invented COBOL made the money. The same thing is currently happening with Java and J2EE - all in the name of open computing. The Java technology is an invention of the large computer hardware companies - their aim is to push their hardware, and consequently - the Java Technology demands more and more of 'centralized' computing power. As a language, it is doesn't maintain a clean separation between the virtual machine, the environment and the programming constructs. It looks like SmallTalk, but its philosophy is that of ADA - a big, fat language with too many features and no simplicity. And, today these are the same people who are talking about service oriented architectures and design methodologies. How will it ever work? When they talk about service - they have only one thing in their mind, the customer serving them!! One critical look at the current SOA will reveal that it is a tiger disguised as a holy cow - the SOA wants more and more resources, introduces more and more technologies, centralizes all information and knowledge in the system and very cleanly hands over the customer to the corporations. It is nothing but the mainframe machines, COBOL and process centric methodologies put together in a new disguise.

On the other hand, what was the invention of the tech-shamans of computing science? They invented workstations, powerful user interfaces, networking, distributed computing, cryptography, artificial intelligence, gaming, multi-media, Internet, Unix Operating system, C-like programming languages, hypertext, grid computing and relational data bases, agile programming, extreme programming, design centric processes and problem solving and so on. The list goes on and on. More importantly, they invented open sources programming, making freely available their ideas and designs. Is there any real programmer on earth who learnt his programming from IBM and what it stands for? Is it even possible to learn programming in such closed circuit, big brother environments? All programmers learn programming from the open sources - by contributing to open sources, by using open sources, by reading and following the open sources programming models. One thing common in all their contributions - is a clean separation of concerns, extraordinary simplicity of design, an approach to very clearly communicate the problem-solving techniques, and all these above inventions do not make it heavy on the users - they take responsibility for their own mistakes, they are all designed to tolerate errors from the environment, all of them are extraordinarily reliable and consequently very cheap to own. Neither the inventors nor their inventions charge the customer for their own mistakes. The SOA is known to this community decades ago. In fact, all the systems and technologies built by were all inherently "service-oriented". They are very economical in their resource usage, and they all freely disseminate knowledge and information. I call these people the tech-shamans and hippies of computing science.

I need to explain what I mean by Shaman and Hippy.

In general, the public image of a hippy is a person who is a drug addict and a general misfit in the society, who cannot live in the usual demands of the societal pressures. But, in reality - the hippies are people who are "free" - they are free from the usual tendency to acquire material possessions, they are free from the usual conventions of linear thought processes, and they are free from fear, greed and the usual trappings. They only desire for ecstasy - a few degenerated hippies may get this ecstasy from drugs - but the majority of them get it from problem-solving, from the joy of work. They do not demand any other form of payment. For them, money, power, fame etc., are all byproducts. Their main interest is in the 'kick' they get in understanding, the suffering they go through in the effort required to assimilate and internalize information and in problem-solving - the application of knowledge. A recent psychological survey revealed that problem-solving – working on difficult problems – releases the same chemicals that are involved in drugs – it produces a ‘kick’ very similar to drugs and sex.

Shamans are evolved ‘hippies’ – in a way hippies of the highest order. He could sit with a plant for a month, talk to it to find out whether its herbs could be eaten. Finally he may eat the herb and die – that would be his contribution to humanity. On the other hand, there are priests – the organized institutional worker - working for control and centralization as opposed to the Shaman who acts as a bridge between two worlds.

In no other field of human Endeavour, the fight between the free spirited hippy and the controlling industrialist are more marked and visible than in the computing science. The history of computing science is littered with records of such fiercely fought battles - the failure and success of Multix and UNIX, the great debate on relational and hierarchical data models, between what is ugly and beautiful and so on. The hippies had Dijkstra, Knuth, Ken Thompson, Tony Hoare, Writh, and E.F. Codd as their keepers of faith and the industrialists had large bank accounts. The history of human beings recorded rather well that it is fearlessness and truth that win the war, not power, money and ignorance. The result is well known even before the war started.

But, this time the hippies learned their lessons from history. They were no longer interested in fighting useless battles. By definition, their knowledge and inventions are available to every one - including the control minded industrialist. The evolution of computing science really nothing but a statement of how the industry assimilated the contribution of the hippies and shamans, and is slowly and painfully learning to respect them and their contributions.

Some examples and illustrations are probably required at this stage.

First, let's take Object Oriented Programming. For any normal human being, objectivity is the hardest thing to understand. Many philosophers, over the last few millenniums devoted their entire lives to understand and explain to us objectivity. The aim of science is an objective understanding and explanation of the reality - did it succeed?

Some tech shamans like Marvin Minsky and others - unleashed in the early sixties - some thing called Frames as a knowledge representation scheme, and some other shamans developed SmallTalk programming language. Even the names give their Shamanistic origins. So, object oriented programming was born. This was a very original idea - for those who understand it, it appeared very simple, and for those who did not understand it - it was an alluring mystery, whose secrets must be cracked at any cost. This division continues even till today - very few programmers really understand object oriented programming - the rest of them use UML. The object oriented analysis, modeling and the modeling languages are nothing but the industry's attempt to assimilate a very hippy concept. After all the literature, tools, systems, training courses that the industry developed for Object Oriented Programming - no one can still explain 'how to' identify and design an 'Object'. This is still the best kept secret of programming - passed on from the Master to his most accomplished disciple.

Again in the Sixties, the patron saints of tech-shamans - Dijkstra and Tony Hoare - unleashed some thing called concurrent or non-linear programming. The concept simply is that it is more efficient use of resources if programming is relieved of its tiresome linearity. How many people can think non-linearly? The shaman hardware designers went ahead and fundamentally altered the way the machine is constructed - they made the machines interrupt driven, in other words, the behavior of the machine is non-deterministic and thus non-linear. Countless coding soldiers wasted probably billions of lines of code trying to understand how to configure such a non-deterministic machine. If you look carefully at the core of the unreliable software - you can trace it to the so called 'dynamic' aspects of the environment - aspects that are not known apriori. How can you design a program for an environment that you cannot predict it at the time of writing the program? For the shamans - it is a simple problem with an obvious solution, and for a control minded industrial freak - it is unthinkable.

One thing is very obvious. More than seventy percent of the programming projects undertaken by the industry are failures - they run into too many technical and budget problems, they run into performance and reliability problems. But, name one open sources project that has any of these problems?

The industry tried to tackle this problem using their age old technique and very old, useless thinking tools. They created software engineering methodologies and they write tons of documentation of their technical designs and programs, they spend millions of dollars in testing. But, for some mysterious reason - open sources systems do not have any such things. They don't come with tons of specification documents, high level designs, pseudo code etc.

Anywhere you care to look - this division is obvious. There are two streams of computing science and two streams of people. Let me call them programmers and coders, or if you don't mind - shamans and priests. Even computing literature can be divided into two classes - the shaman literature and the priest literature. This division is very marked especially in Software Engineering.

Software Engineering is perhaps the only discipline that aims to teach problem-solving. Its interest is not in providing some specific solutions to specific problems, but to teach problem-solving. But, the big, fat books on Software Engineering written by priests do not get the central idea. They only confuse it even further – they describe methodologies and processes, basically they preach what people should be doing, without even giving a hint of how to do something.

The best software engineering books were written by Shamans – people like Dijkstra, Knuth, Weinberg, Rob Pike, and Bertrand Mayer and so on. Knuth’s 70 page paper on Errors of Tex is perhaps the best description of the software engineering process ever published. These are works that are full of insights and techniques – not some religious preaching.

But, what is the secret of the shaman?

The secret is - as all best kept secrets in the history are - is very simple and very obvious. The tech-shamans are basically builders and engineers. The metaphorical links between Free Masons and Free Software Foundation is hard to miss. They understand the process of design, they understand engineering as a discipline, and they understand that the process of making something and its final object of creation are two completely different things altogether.