Service Oriented Computing and Tech Shamans

The Design and the Designer – Part 8

There is a lot of talk about Service Oriented Computing these days – mostly by people who don’t seem to understand what it really stands for.

Until now, we have very good models of infrastructure engineering and applications engineering, but we are yet to see a reasonable model for process engineering and structural engineering in the computing science. This could be because of the unfolding of services as a primary business model has just begun and we have not yet understood what service really is.

A very successful billionaire once told me that the Services Automation is several orders of magnitude more significant than the industrial revolution. The explanation he offered for this significance was very striking. It has the beauty of simplicity and the brilliance of Truth to it. According to him, the basis of industrial revolution is control of resources and centralization of knowledge and power. Who ever can control and marshal resources and centralize and guard knowledge becomes wealthy.

But, the services revolution is exactly opposite. Its basis is decentralization of information, power and knowledge. Its basic premise is that we should use resources as when required without really possessing them. Look at the business models of today, especially the Internet. Its basic principle is making information equally available to everyone, and make use of resources as when required without possessing any resource at all.

But, how is this possible? Why would the wealthy people - who became wealthy because of the exact opposite principle - allow such a thing to happen? Why would they want to let go of their control and let go of their wealth?

Well, the answer is that it was not made possible by the wealthy and rich business people. Most of the computing science and the consequent business models are the creations of enlightened hippies. They were the only people capable of such subtlety of thought and action. They even managed to get vast amounts of funding from the very same people that they resolved to dethrone them from their seats of power!!

Let's look at the evolution of computing science, its absorption by the 'industry' and the unseen hand of the hippy.

On one hand, there is IBM and all its 'industrial' representatives and on the other hand there were tech-shamans.

What did IBM and its representatives contribute to the computing science? They tried big, fat main frame machines, resource hogging, hierarchical operating systems like MVS, they tried unusable programming languages like COBOL and they tried to push for data modeling rooted in their three hundred years of arrogance. Now they are trying Java and J2EE. In the software engineering side, they perpetrated process centric methodologies. All these technologies, methodologies, disciplines have one thing in common - they all aim to make the process expensive, aimed to clear the "owner's" inventory and make their clients/customer pay a very heavy price. And, more importantly - they have some kind of 'lock-in' of the customer built into all of them. In the long run - all of them are very expensive - the price paid by people who adopted COBOL during the Y2K issue must be an eye opener. And, who spent the money to resolve the Y2K bug, and who earned it? The people who bought COBOL paid the price, and the people who invented COBOL made the money. The same thing is currently happening with Java and J2EE - all in the name of open computing. The Java technology is an invention of the large computer hardware companies - their aim is to push their hardware, and consequently - the Java Technology demands more and more of 'centralized' computing power. As a language, it is doesn't maintain a clean separation between the virtual machine, the environment and the programming constructs. It looks like SmallTalk, but its philosophy is that of ADA - a big, fat language with too many features and no simplicity. And, today these are the same people who are talking about service oriented architectures and design methodologies. How will it ever work? When they talk about service - they have only one thing in their mind, the customer serving them!! One critical look at the current SOA will reveal that it is a tiger disguised as a holy cow - the SOA wants more and more resources, introduces more and more technologies, centralizes all information and knowledge in the system and very cleanly hands over the customer to the corporations. It is nothing but the mainframe machines, COBOL and process centric methodologies put together in a new disguise.

On the other hand, what was the invention of the tech-shamans of computing science? They invented workstations, powerful user interfaces, networking, distributed computing, cryptography, artificial intelligence, gaming, multi-media, Internet, Unix Operating system, C-like programming languages, hypertext, grid computing and relational data bases, agile programming, extreme programming, design centric processes and problem solving and so on. The list goes on and on. More importantly, they invented open sources programming, making freely available their ideas and designs. Is there any real programmer on earth who learnt his programming from IBM and what it stands for? Is it even possible to learn programming in such closed circuit, big brother environments? All programmers learn programming from the open sources - by contributing to open sources, by using open sources, by reading and following the open sources programming models. One thing common in all their contributions - is a clean separation of concerns, extraordinary simplicity of design, an approach to very clearly communicate the problem-solving techniques, and all these above inventions do not make it heavy on the users - they take responsibility for their own mistakes, they are all designed to tolerate errors from the environment, all of them are extraordinarily reliable and consequently very cheap to own. Neither the inventors nor their inventions charge the customer for their own mistakes. The SOA is known to this community decades ago. In fact, all the systems and technologies built by were all inherently "service-oriented". They are very economical in their resource usage, and they all freely disseminate knowledge and information. I call these people the tech-shamans and hippies of computing science.

I need to explain what I mean by Shaman and Hippy.

In general, the public image of a hippy is a person who is a drug addict and a general misfit in the society, who cannot live in the usual demands of the societal pressures. But, in reality - the hippies are people who are "free" - they are free from the usual tendency to acquire material possessions, they are free from the usual conventions of linear thought processes, and they are free from fear, greed and the usual trappings. They only desire for ecstasy - a few degenerated hippies may get this ecstasy from drugs - but the majority of them get it from problem-solving, from the joy of work. They do not demand any other form of payment. For them, money, power, fame etc., are all byproducts. Their main interest is in the 'kick' they get in understanding, the suffering they go through in the effort required to assimilate and internalize information and in problem-solving - the application of knowledge. A recent psychological survey revealed that problem-solving – working on difficult problems – releases the same chemicals that are involved in drugs – it produces a ‘kick’ very similar to drugs and sex.

Shamans are evolved ‘hippies’ – in a way hippies of the highest order. He could sit with a plant for a month, talk to it to find out whether its herbs could be eaten. Finally he may eat the herb and die – that would be his contribution to humanity. On the other hand, there are priests – the organized institutional worker - working for control and centralization as opposed to the Shaman who acts as a bridge between two worlds.

In no other field of human Endeavour, the fight between the free spirited hippy and the controlling industrialist are more marked and visible than in the computing science. The history of computing science is littered with records of such fiercely fought battles - the failure and success of Multix and UNIX, the great debate on relational and hierarchical data models, between what is ugly and beautiful and so on. The hippies had Dijkstra, Knuth, Ken Thompson, Tony Hoare, Writh, and E.F. Codd as their keepers of faith and the industrialists had large bank accounts. The history of human beings recorded rather well that it is fearlessness and truth that win the war, not power, money and ignorance. The result is well known even before the war started.

But, this time the hippies learned their lessons from history. They were no longer interested in fighting useless battles. By definition, their knowledge and inventions are available to every one - including the control minded industrialist. The evolution of computing science really nothing but a statement of how the industry assimilated the contribution of the hippies and shamans, and is slowly and painfully learning to respect them and their contributions.

Some examples and illustrations are probably required at this stage.

First, let's take Object Oriented Programming. For any normal human being, objectivity is the hardest thing to understand. Many philosophers, over the last few millenniums devoted their entire lives to understand and explain to us objectivity. The aim of science is an objective understanding and explanation of the reality - did it succeed?

Some tech shamans like Marvin Minsky and others - unleashed in the early sixties - some thing called Frames as a knowledge representation scheme, and some other shamans developed SmallTalk programming language. Even the names give their Shamanistic origins. So, object oriented programming was born. This was a very original idea - for those who understand it, it appeared very simple, and for those who did not understand it - it was an alluring mystery, whose secrets must be cracked at any cost. This division continues even till today - very few programmers really understand object oriented programming - the rest of them use UML. The object oriented analysis, modeling and the modeling languages are nothing but the industry's attempt to assimilate a very hippy concept. After all the literature, tools, systems, training courses that the industry developed for Object Oriented Programming - no one can still explain 'how to' identify and design an 'Object'. This is still the best kept secret of programming - passed on from the Master to his most accomplished disciple.

Again in the Sixties, the patron saints of tech-shamans - Dijkstra and Tony Hoare - unleashed some thing called concurrent or non-linear programming. The concept simply is that it is more efficient use of resources if programming is relieved of its tiresome linearity. How many people can think non-linearly? The shaman hardware designers went ahead and fundamentally altered the way the machine is constructed - they made the machines interrupt driven, in other words, the behavior of the machine is non-deterministic and thus non-linear. Countless coding soldiers wasted probably billions of lines of code trying to understand how to configure such a non-deterministic machine. If you look carefully at the core of the unreliable software - you can trace it to the so called 'dynamic' aspects of the environment - aspects that are not known apriori. How can you design a program for an environment that you cannot predict it at the time of writing the program? For the shamans - it is a simple problem with an obvious solution, and for a control minded industrial freak - it is unthinkable.

One thing is very obvious. More than seventy percent of the programming projects undertaken by the industry are failures - they run into too many technical and budget problems, they run into performance and reliability problems. But, name one open sources project that has any of these problems?

The industry tried to tackle this problem using their age old technique and very old, useless thinking tools. They created software engineering methodologies and they write tons of documentation of their technical designs and programs, they spend millions of dollars in testing. But, for some mysterious reason - open sources systems do not have any such things. They don't come with tons of specification documents, high level designs, pseudo code etc.

Anywhere you care to look - this division is obvious. There are two streams of computing science and two streams of people. Let me call them programmers and coders, or if you don't mind - shamans and priests. Even computing literature can be divided into two classes - the shaman literature and the priest literature. This division is very marked especially in Software Engineering.

Software Engineering is perhaps the only discipline that aims to teach problem-solving. Its interest is not in providing some specific solutions to specific problems, but to teach problem-solving. But, the big, fat books on Software Engineering written by priests do not get the central idea. They only confuse it even further – they describe methodologies and processes, basically they preach what people should be doing, without even giving a hint of how to do something.

The best software engineering books were written by Shamans – people like Dijkstra, Knuth, Weinberg, Rob Pike, and Bertrand Mayer and so on. Knuth’s 70 page paper on Errors of Tex is perhaps the best description of the software engineering process ever published. These are works that are full of insights and techniques – not some religious preaching.

But, what is the secret of the shaman?

The secret is - as all best kept secrets in the history are - is very simple and very obvious. The tech-shamans are basically builders and engineers. The metaphorical links between Free Masons and Free Software Foundation is hard to miss. They understand the process of design, they understand engineering as a discipline, and they understand that the process of making something and its final object of creation are two completely different things altogether.

The Alchemy of Design and the Sound of Music

The Design and the Designer – Part 7

In the last post, I made some observations about the process of Architecture – very briefly I stated that Architecture is the process of bringing meaning to space, and expressing the meaning in terms of certain engineering transformations, which transform the problem into an engineering problem.

The process of design – transforming ‘meaning’ into a set of engineering problems is very analogous to music composition. The difference between music and sound is the same as the difference between a sweet home and a concrete building. Therefore, let’s try and understand how music is composed. Later, we will define what engineering really is, and then I shall describe the relationship between design and engineering.

It is very simple to explain the physics of Sound. Sound is produced when two objects make contact with each other. Sound consists of a series of waves that ‘move’ the air in a particular way. The two major characteristics of Sound are its amplitude and frequency plotted on a time scale. That’s pretty much it.

What is Music? What makes certain sounds musical and certain other types of sounds non-musical? Can we say that Music is the sound we like? What do we ‘mean’ by liking? We use many different ‘esoteric’ words associated with Music, but not with Sound – words like melody, harmony, rhythm, tempo, beauty and balance.

A music composer somehow ‘transforms’ these musical attributes into a set of sound waves. The music theory build up over thousands of years provides some guide lines of how this transformation could be achieved. The music theory – whether it is western music, or Indian music theory – is a body of knowledge that codifies how music can be produced using sound. This body of knowledge can be divided into two distinct categories – the first category enables a music composer to ‘recognize’ his/her creative impulse. This is the body of knowledge that deals with recognizing music. This body of knowledge does not talk about frequencies, amplitude or rhythm, but it deals with characterizing the emotional qualities of music, and tries to categorize them.

In Indian Music – the concept of Raga for example deals with systematizing the ‘melody’ into a set of information structures. A Raga is therefore nothing but a shared understanding of melody and how to express it objectively. For those who are not familiar with Indian Classical music, here is a very brief explanation of Raga.

A Raga defines a set of melodic patterns and a melodic structure. For example, if you take a Raga like Mohanam – it is used to express the feelings of love and romance. Love and Romance are very ‘semantic’ concepts. What the Raga manages to do is to transform these concepts into a melodic structure and a set of melodic patterns. So, a composer whose creative impulse connects him with love or romance, he can then use the codified body of knowledge of Raga Mohanam, and compose music in that Raga.

Melodic Structure: This deals with the ‘rasa’ or ‘experience’ of music. There are about nine primary ‘rasas’ and another nine sub categories. There are all kinds of permutations and combinations of these primary experiences are possible, but any composition’s melodic aspects can be explained using these ‘rasas’. Apart from the Rasas, there is a ‘presentation’ aspect of the experience. The presentation is further divided into three categories: Majestic, Mellifluous, Simple (the Sanskrit Terms are Ghana Ragas, Rakti Ragas and Desi Ragas).

Melodic Patterns: The human ear is apparently very logarithmic. All music uses a concept of Octave. An octave is a predefined set of ‘notes’ or ‘frequencies’. The most frequently used octave all over the world uses 12 notes. Basically, you start from a certain base frequency, and keep increasing the frequency for every subsequent note, and by the time the octave is completed, you would have doubled the original frequency. Basically a Raga is a particular subset from an octave (7, 6 or 5 notes). It defines how to move back and forth along the octave (called Ascent and Descent). It also defines certain relationships between the notes – these are called ‘sangati’ and ‘sancharam’. In other words, a Raga defines – purely theoretically – a certain ‘musical movement’ (chalan), and also gives a set of ‘musical phrases’ that could be used to create a certain type of experience. A ‘raga’ therefore defines both the melodic structure and the melodic pattern.

The beauty of a raga is that it is just a pattern – without any ‘fixed’ compositional aspect to it. The same pattern can be used in millions of different ways – giving rise to a wide variety of compositions. A raga is thus a design pattern or a ‘semantic abstraction’. In the same way that a bird does not exist any where in the physical world, but there are millions of manifestations of the bird, a raga has no physical existence – it is just a useful abstraction.

The second category of Music Composition is closer to the characteristics of sound – and this acts a bridge between the musical characteristics and sound characteristics. For example, almost all musical traditions have a concept of an octave and beat cycles. An octave is an information structure and a very useful abstraction. (Please refer to my earlier posts for a detailed discussion on abstraction, semantic types and concepts). This body of knowledge of basically deals with the structural aspects of music composition.

Music composition therefore is very similar to design and engineering. There is an understanding of meaning and transforming that understanding into a set of structural transformations. Perhaps this is the reason why architecture is followed by structural engineering, and there is a very strong connection between architecture and structural engineering.

In the next post, we will define structural engineering, and its relationship to design and architecture.

A Disturbing question

Recently a very close friend of mine asked me a very simple question. He heard from many radical fundamentalists that there is a sanction in Islam to kill infidels – meaning the Koran specifically grants the permission to do so. He wanted to know from people who read Koran whether such a statement does indeed exist. Basically his point is very simple – how can other religions coexist with Islam if such a provision does indeed exist in Koran? Of course, this friend of mine is a very fine scholar, so he believed that such claim by fundamentalists must be totally false because no true religion would ever make such a statement.

At a spiritual, metaphysical and philosophical level, such a statement can have many inner meanings, but if it were true – how can other religions co-exist with Islam at a theological level? If every non-Muslim is considered as a non-believer, and if Islam ‘mandates’ its followers that such non-believers should either be converted or killed, how can we even have a dialogue with them?

This is a widespread belief among many people and perhaps their source of discomfort with Islam. My friend told me that someone who read Koran thoroughly must come out strongly against such a widespread belief and make a public statement, providing the relevant references. How many people would have read Koran? If you are a Hindu, or a Christian or a Buddhist – you are not expected to read the holy book of Islam. May be half the Muslims do not know Arabic and so they cannot read Koran themselves.

What use is another newspaper article in any case?

Any sane person like my friend knows intuitively that such a belief has no basis and must be false. Islam apparently means peace and the Prophet called himself the messenger of peace. Therefore, any such belief must be self contradictory.

But, sanity today is an endangered species.

Our knowledge of history comes largely from various propaganda machines and is constructed by what we read about events in the newspapers. Newspapers report only events. Can they report progress, can they report ten thousand years of human evolution, can they even make an attempt to report what a religion and culture is all about?

But, it is a fact that we live in the age of short term memories and information overload. There is just too much information these days – most of it is pure junk – traveling at the speed of light. Short of asking everyone to read Koran – is there any other sane solution to such a wide spread belief?

There is overwhelming evidence in history that kings, leaders and clergy use their power with disastrous effects. Some societies – like some of the Polynesian islands like Easter Island and the Maya Kingdoms completely destroyed themselves largely because of the attitudes of their leaders. In all such cases, Religion was invoked by the leaders and rich people to justify their own ends, and to make people firmly hold on to certain belief systems.

Religion is a very deep rooted value system, and it can be and almost always is exploited by people in power to their advantage, to promote their own vanity or as an instrument to keep people ignorant.

However, religion and culture do not live in history books or buried in historical monuments. They exist for and through the common people. If we meet an average American, we can estimate the American value system, their religion and culture in five minutes of interaction – we may not be able to write detailed analytical essays, but we understand it intuitively. What a religion and culture stands for is represented by its living monuments – the common, average people. If you interact with an Indian – even casually – you get a ‘feel’ of Indianness, if you meet a Buddhist you get a ‘feel’ for Buddhism.

The average, common people – not people in influential positions – like scholars, clergy and leaders – but just average common people – internalize their culture so deeply and radiate their value systems all the time. They do it unknowingly, without any pretence. There is no drama, no cultivated, politically correct or incorrect responses. Their responses are natural.

If you meet a Muslim autoriksha driver in Hyderabad or a Muslim carpet merchant in Bangalore, a Muslim antique dealer in Delhi – what impression do you carry about them? I believe that impression is always correct. My impression is they are very tolerant, graceful, genuinely honest and nice people. To me, that is what Islam is.

Is there a better answer than that? Can any amount of scholastic reports answer such questions better than our own first hand experiences? Why don’t we trust what we observe, what we feel about our own fellow human being? Why do we place an overwhelming reliance on the media and other propaganda machinary?

There is so much of “manufacturing of our concent” that goes on in the world today by our governments, by our leaders and by the multi-billion dollar corporate houses. They have vested interests. Therefore, in today’s world – it is much better to trust our own first hand experience. If we don't then there is a danger that we lose our own innate ability to discern, and become unwitting instruments of manipulation by the powerful.

God is one when religion is dead.

Abstraction, Pattern Recognition and Problem Solving

The Design and the Designer – Part 6

A gap of more than two weeks interrupts the flow. But, the gap is probably required because from here – the approach is very different. Until now, I largely covered a lot of breadth, and was able to give examples in one paragraph – an approach that may not be sufficient from now onwards. It is like getting into the second chapter of a book – by nature, it gets a little deeper and a bit more boring.

The difference between Architecture and Engineering is like the difference between Music and Sound. Architecture is a word that is associated with many disciplines today – we talk of architects of nations, architects of enterprises and corporations, architects of cities, plantation forests, gardens, buildings, computers and information systems. Sometimes, this word is also used in relation to effects produced by an individual or a group of people – architects of success or failure, architects of war and of destiny. After the last Great War, this discipline seemed to have become main stream.

Even though, these days we use the term in its verb form, actually it exists only as a noun. The architect produces architecture – what he does is not architecture, but design. Therefore, the process used is the process of design, but the end product is “Architecture”. All architects are designers of a particular kind.

What does really an architect do?

Let’s consider the traditional architecture of buildings as an example. Even though the structural engineers deride architects as people who basically add unnecessary embellishments that do not have serve any functional purpose, in reality the job of an architect is one of transformation much like the work of an alchemist.

What the alchemist is to science, the architect is to engineering.

His work involves understanding the essence or meaning of space and transforming this understanding into an engineering problem.

Sounds abstruse even to me. We need some detailed explanation.

An architect basically brings meaning to space. An engineer does not make any distinction between kitchen and living room – both are basically enclosed spaces as far as the engineer is concerned. But, for people living in a house, kitchen and living room have very distinct functions, how they relate to these spaces is very different, the time they spend in these spaces is very different. It is the job of the architect to understand the function of a kitchen or a living room, and their relation to the people who use them and then somehow be able to express it in terms of certain ‘spatial’ characteristics – their geometry, placement, the equipment provided in that space. Basically, the architect defines what a kitchen is in certain engineering terms.

In short, the mysterious craft of architecture is very simple to express in words. The architect performs a series of semantic transformations; the last transformation achieves the feat of converting the problem into a structural engineering problem. If the alchemist transforms lead into gold, the architect transforms gold into lead – in a manner of speaking.

The reason why structural engineers do not respect architects is perhaps because of the fact that over the years, the community of architects was successful in creating a set of transformations that are universally accepted and understood, and the engineers themselves understand the difference between the kitchen and living room without the help of an architect. So, they do not see the necessity of an architect for the most common problems like the construction of a house, or a bridge any more.

In short, architects create new abstractions. An abstraction is an information structure – that codifies meaning objectively, an information structure that is universally understood. The power of abstraction is such that – if we have to state in one single line the difference between human being and an animal – we can say that the difference is the ability to come up with an abstraction.

What is an abstraction? A “tree” is an abstraction. ‘A Tree’ does not exist anywhere in the physical world – there are mango trees, there are lemon trees, there are all kinds of other trees, but there is no such entity called ‘tree’ – it is a pure abstraction that exists only in our consciousness. It helps us to group all mango trees, all lemon trees and all other trees in one sweep, and capture the ‘essential treeness’ that all of them share. Similarly, ‘A Man’ does not exist any where except in our consciousness – there is a John, there is a Joe, there is an Adam – but where is ‘Man’?

It took several thousands of years for us to come up with some very simple abstractions that we take it for granted today. One such example is the number system and the simple arithmetic of additions and subtractions. In ancient times, when people did not know the basic math, the methods they used for trading were very funny. If one goat equals two bags of rice (purely a concept of quantity decided by its value, there was really no such thing as ‘one’ goat and ‘two’ bags of rice) and if you want to want to exchange two goats, then what do you if you do not understand what is one and what is two? You give one goat, take two bags of rice, and then give one more goat again and take another two bags of rice. A very cumbersome process indeed – isn’t it?

But, what can people do if they do not know what is one and what is two? The numbers are very powerful abstractions.

An abstraction is valid only if it is completely objective. This condition means that an abstraction must mean exactly the same thing to everyone. The difference between an abstraction and a concept is in how strictly this condition is applied. A concept is not very strict in imposing the condition of objectivity. For example, “world” is a concept – but what I mean by world and what you mean by world can be somewhat different.

Abstractions, concepts or semantic types – are all information structures. It is a unit of information that retains the meaning of something, and is made accessible to everyone.

What is interesting is how we as humans acquire concepts and abstractions. We do it so naturally – we do not even realize that we are dealing with abstractions and concepts all the time. But, it is an amazingly complex process.

When as children we learn our alphabet – all we are shown is one particular shape of the alphabet. Your teachers writes ‘a’ in your notebook, and you practice recognizing ‘a’ and writing it yourself. That’s about it – after that you can recognize ‘a’ in what ever shape, size and form it appears – and you can recognize ‘a’ instantaneously. What you as a child accomplished is an amazing feat – by understanding one particular ‘a’ – you somehow extracted the ‘a-ness’ and then can apply that understanding of ‘a-ness’ like a flash. You can recognize ‘a’ in which ever font it is written, in millions of different handwriting samples. You do not need to be taught to recognize ‘a’ by going through several thousand samples, and various complex rules of recognition.

Such a simple act of extracting the meaning by understanding from one single illustration and then apply it on the fly to recognize millions of variations is an amazingly human ability – something very difficult for machines to do. Computers cannot even recognize their own hand writing!!

It is easy for a computer to produce millions of different ways of writing – you can type a word into a document, and then change the font any number of times. Computers are quite good at such manipulations. But, they are miserable when it comes to ‘recognition’ – at least until now. Suppose you take a printout of a document – and then use a scanner to scan the same document, connected to the same computer, and then try and reproduce the document in its original form ( a process called optical character recognition) – almost always you get a document with too many ‘errors’. The OCR programs are very complex where as the document creating programs (like Microsoft Word) are relatively simple. The OCR programs use several thousand ‘sample’ character sets, and ‘train’ the computer to recognize the ‘a-ness’. But, till now there is no foolproof method to accomplish this.

What makes us so good at pattern creation, extraction and recognition? Let’s leave that question to psychologists. However, this particular ability is at the heart of problem solving. In a most generalized sense, architecture, design, engineering, diagnosis are all a form of problem solving. Problem-Solving and Design involves an innate ability to recognize and categorize patterns, abstractions and concepts. Patterns, abstractions and concepts require associative thinking.

In the earlier posts, I mentioned about two kinds of thinking, two different processes of problem solving and so on. I tried to make a distinction between demonstrative reasoning, and heuristic reasoning. It is well recognized that the art of problem solving involves heuristic methods – for example, using analogy, generalization, reduction, specialization and so on. We also described in some detail about various forms of non-linear thinking – be it creativity, right brain thinking, lateral thinking, out of box thinking and so on.

Associative thinking and heuristic/plausible reasoning are the most generalized forms of non-linear thinking and problem-solving.

How does associative thinking work really and how can we use it systematically?

Here is a small exercise:

Let’s pick up some word randomly – let’s say it is Sun. Write down ten other words that come to your mind (without thinking) when you think of Sun – for example, star, heat, light, earth, round, fire, red, day, night, energy, Egypt, Japan and so on. Now, how is Sun connected with Egypt? The relationship between these two very different concepts is one association and it is association of some meaning. For example, Egyptians used to worship Sun God. Therefore, the relationship between these two concepts is “worshippers of”.

In the last post, I discussed about four structural relationships – hierarchy, network, hypertext and set. The relationship between Sun and Egypt is none of the four structural relationships. It can – at best – be represented in terms of a hypertext relationship, but it is in fact a “semantic relationship” – it is the inherent meaning that connects these two concepts.

Nasruddin – while solving a logic puzzle uses his associative network to come up with a completely out of the box solution. We need such an associative mental network to work with analogies, to work with induction, to generalize something, and from generalization to specialization and so on.

Another advantage with the associative thinking is that – you can recall your entire memory starting at any point. I suggest doing this as a fun exercise. Start with Sun – and write down all the words that come to your mind, and then for every word connected with Sun, write down all the words you can think of. For example, expand Egypt – something like – desert, pyramids, Africa, Nile, Pharaohs, Moses and so on.

Unfortunately, we are never taught how to make and keep an associative memory consciously. We are taught how to practice organizing our knowledge using the structural relationships, and we are taught how to use demonstrative (logical) reasoning, even though our brain uses an associative structure to organize its own memory and experiences. We are naturally gifted with the ability to organize and relate to our experience using associative networks. But, we are never taught how to practice it formally. As a consequence, our memory is a very randomly formed associative, semantic network that we cannot use efficiently.

The real trick of problem solving is nothing but consciously organizing our knowledge and experiences as an inner semantic network of concepts and relationships. In other words, if we can make and keep associative networks of our knowledge and experiences, then we have a better chance of becoming better problem solvers.

More on this in the next post.

An Apology

First – my apologies to all the regular visitors of my blog. Last two weeks have been very hectic. I made a promise to myself that I would upload at least two articles every week. Promises are like spoilt children – as soon as you make a promise - somehow, it brings along situations with it and is always bent upon proving that you are not worthy of it.

There are about seventy articles that I planed to write on various topics – I even made a brief abstract about each one of them – but the actual process of writing them down demands a kind of mental and physical discipline that is very hard to command all the time. The natural laziness is always waiting round the corner to catch up with its old friend, and its company is so enjoyable, one tends to forget everything else and spend all the time with that best of friends.

There is a wonderful Sufi story on this theme, published by Idries Shah in his Seeker After Truth. Here is the gist of the story:

A dervish visited a small town in India. He used to give a talk every night, and many people would go to listen to him. The discourse would begin some after dinner and usually lasted for about an hour and half.

An old man would attend the talk everyday along with his grandson. As soon as the dervish starts to speak, the old man would go to sleep. The dervish watched this for a couple of days, and then he called the grandson to him and promised to give him one rupee if he promises to wake up the old man every time he falls asleep.

The next two days, the boy would pinch his grand father whenever the old man dozed off. But, on the third day, the dervish noticed that the old man was snoring as usual.

The dervish called the boy aside and asked him what had happened – “I thought we had a deal. Why did you not keep your grandfather awake today?”

“That’s right. But when I told my grandfather about our deal, he offered me three rupees not to wake him up”.

The first rupee is our desire to do something productive. The other three are – our habits, our natural laziness and the unobserved opposition to truth.

******

Writing is very difficult. I know what to write, it is very simple to make up the outline of the article. The difficulty starts from the time I sit down to write the actual article. As I am writing the first line, the second line has to ‘spring up’ in the mind simultaneously. Otherwise, the flow is lost. I also have to think of the actual sentence at the same time, and the mind has to recall its typing experience. All this is just too much for the mind. It prefers to drift off, my old friend is back – and we go for a smoke break after every paragraph.

Sometimes the exact opposite happens. I can’t catch up with the speed of my thought – I can’t type as fast as I think. Too many thoughts flood my mind, and I just can’t type any more. So, I take a break and have a dialogue with myself. Before I realize, five hours are gone! By that time, there is too much content piled up in the head – I get enough content for another ten articles. I end up with a perpetual backlog.

I also have another difficulty. English is still a foreign language to me. The Indian languages and English are not natural friends. They are in fact quite the opposite. All Indian Languages are free order languages - it is so easy to construct phrases in Indian Languages. In fact, most words are not single words – most of them are phrases.

English does not have that kind of flexibility. So, when most Indians write English – they tend to make up wrong kind of phrases. For example, the meaning between “I am John’s friend” and “I am a friend of John” is quite significant depending on the context. For some strange reason, I always write the sentence in the reverse order, and then I have to do a lot of editing and rewriting.

Take the previous paragraph as an illustration. An Englishman would write almost every line in the reverse order: Most Indians tend to make up wrong kind of phrases when they write English. Depending on the context, the difference between “I am John’s friend” and “I am a friend of John” could be significant.

Can you see my difficulty? The previous paragraph was my first attempt. So, I have to apply the principle of inversion even to my English!! I always look for which sentences need to be reversed to make them right!!

I have a difficulty with the usage of tense as well. I still do not get the difference between the simple past and present perfect tense. The difference between I read the article and I have read the article eludes me all the time. For some strange reason, many Indians have a preference for “have, has, had” – they prefer “I have written a letter” to “I wrote a letter”. I am sure without the help of a grammar book, most of us can’t tell the difference between the two.

So, I end up doing a lot of editing on each article. I pray for the day when I can write effortlessly.

Forgive me for the delay. I will be back tomorrow again and continue the series on Semantics and other topics.