Freelancing in India – The oriental carpet shop

According to an ancient saying in India, the secret of long and fulfilling life lies in avoiding four types of people at all costs - bankers, doctors, lawyers and police. There is only way to avoid these four types – don’t earn too much, take care of your health, don’t get into disputes with anyone, and don’t commit any crime. This must be true in most countries, but here in India - this is the most practical thing to do.

In India, the laws are very favorable to the weaker party - it is impossible (at least legally) to fire an employee or avoid payment. But, the unfortunate part is that if you go to court - only your great grandson may hear the final ruling. This practically rules out any legal recourse for many people. Therefore, any legal documents that you sign are practically useless - at least as far as YOU are concerned. The advantage is you don't need to consult a lawyer. But, the disadvantage is you are on your own. You have to evolve your own way of getting your payments on time, getting your deliverables accepted and work out the complexities of the inner workings of your client organizations.

First, let me tell you a couple of stories from the field. The first story is about the need to read carefully all legal agreements and more importantly the price of arrogance and individualism that one pays for in this country. The second story is about the need to understand the subtlety of the eastern mind.

I know of a consultant who worked for a billionaire. There was a disagreement between them on some issues - the issue was not about money, but the working styles of the client and the consultant. This particular consultant was very arrogant - one day he had a major fight and walked out of the assignment. According to the agreement he signed - he could leave with one day notice, so he thought everything would be fine, and at best his client wouldn't release the pending payments.

Two days later, a battery of lawyers descended on his house - and practically sealed everything - his wristwatch, cell phone, microwave, TV, credit cards, computer - and practically every single tiny electronic item he owned was sealed. The reason: he signed an intellectual property document, and the document says that in the event of termination of the assignment, the client has a right to verify that no confidential data was stored in any electronic medium!! Technically even a wrist watch is an electronic medium.

This poor chap couldn't even make a phone call, and his client was out of contact for a week. Finally, he had to swallow his pride and beg for pardon.

The important lesson is that be careful of the documents you sign with your clients. Any condition in the agreement must be something that you yourself should be able to establish.

The second story is my own first hand experience. In India, your clients will never dispute anything - especially about payments. They fully agree that they have to pay and they will always tell you that they have every intention to clear all the pending payments as soon as they can. They will never get into an argument with you on payments. If you ask about payments - they will talk to you for two hours, tell you how much they value your work and relationship, take you out for lunch, make you very comfortable - but they always find some way of delaying the actual payment. In the west, people are in general have an 'on your face' attitude, but here it is very subtle. It is like going to an oriental carpet shop and negotiating for price - everything is discussed except the price.

Once, I completed my assignment, and sent all documents to my client, and sent them the invoice. There is a thirty day period for all payments - so, I waited for about three weeks. Nothing happened. I contacted the finance department - they told me that they did not receive any approval from the VP, Engineering - who is the person in charge for my consulting work. I met him again - he apologized profusely for the delay for about half an hour, and in the meanwhile asked me to 'help' them with a few small things and so on and so forth. I obliged - it took another one week. I sent the invoice again to the VP and the finance department. This time, the VP took a printout of the invoice - and he wrote on it the three magic words: "please pay" and signed. I handed over the invoice to the finance department. The accountant gave me a charming smile and said everything will be done in a week's time. Well, nothing happened for another two weeks.

Again, I met the VP. This time he cursed their finance department - he said they are always very slow, don't do their job on time etc., etc. He again took a printout and again wrote the three magic words and signed. I took the invoice to the finance department. This time the accountant asked me to wait - he wrote the check and gave it to me within half an hour.

I asked the accountant that why they did not clear my payments the first time the bill was passed by the VP. The accountant said "Sir, the first time he cleared your invoice, but wrote it in blue ink. That is an instruction to us not to pay. If he writes it in green ink, then it means send the check after two weeks, and if he writes it in red ink - it means that we should pass it immediately".

This is how subtle the process could be here. I use some simple rules to deal with the eastern complexity and it works.

First, I always do one week of gift work before I formally engage with any clients. I don't charge for the first one week, and I work without any formal contract. This allows me to understand several things - most importantly I get to understand the problem I am expected to solve, I get to assess whether I could do it or not, whether the team is capable enough to solve it themselves - how much guidance they need from me and so on. Most importantly, I can estimate the culture of the organization. Metaphorically speaking - I get to know in which color the invoice has to be signed!!

A consultant has to deal with three different people in the client organization. The person who employed you, generally it is a senior manager - most often, the head of development, delivery or technology. The second person is the one who actually needs you and works with you - generally it is the development team, represented by a project manager, and finally - some one from the finance department. These three people can form a nice little triangle and play a perpetual football with you.

What I do is to try and break the triangle. I insist that one person from the development team - either a project manager, team lead or a programmer be assigned as my primary client contact, who will have all the authority to decide on the deliverables, to sign off on my time sheets and who has to ensure that payments are cleared. This means that even though the agreement is signed between me and a senior manager of the company, the agreement explicitly nominates one person by name.

The advantage is that - programmers do not switch off their cell phones, they do not go on extended foreign tours, and they don't have secretaries. You can always call them up and go for a lunch. And, they understand which color of ink their boss has to sign. Basically, I make my internal client contact run around to clear all my payments. Once they accept this responsibility - they are in general very sincere and prompt. Contrary to all generally accepted beliefs, I found that the finance department tends to be the most efficient and prompt of all other divisions of the organizations. Once they get clear instructions, they act very promptly.

The second part of my consulting agreement is to clearly identify three types of deliverables. In India, the clients do not draw a clear boundary between the consultant and the organization. The consultant - especially if he is a freelancer - is always treated as one of their own. This has several advantages and disadvantages.

The disadvantages:

• The scope always expands.
• No clear idea of what the deliverables are - the distinction between the work and the deliverable are always blurred. You are asked to do work but get paid based on certain concrete deliverables that you have to produce.
• Confusion about who is owner for a certain area of work.

Normally, what I do is to identify three types of responsibilities and document them as part of my 'work contract':

• Primary responsibilities: I have the primary ownership for these deliverables, and I have control on the project plan.
• Collaborative responsibilities: My expertise is required for these deliverables, but someone else has the overall ownership and control on the pace of the activity.
• Participatory responsibilities: Areas where my inputs are sought - but, I have a right of refusal to participate.

Finally, it is important to get the payments on time. Suppose I expect the payment to be done on 30 of the month. If I send the 'deliverable' document on 25th, I can never get the payment on time. This is because, the deliverable is not completed when I send it, but when it is read and understood by the receiver. So, if I want my payments on 30th, I send my documents on 10th of the month - and give them two weeks time to accept the deliverable, and also I use the two weeks to ensure that they indeed make the effort to understand it clearly.

I wish all of you and your families a very happy and successful new year. May the Lord bless you to find your heart's desire.

Freelancing in India

Many young people from India are writing to me asking about freelancing. If you are also ‘bitten’ by the freelance bug – and you are thinking of writing to me, read this before you shoot that email to me.

I believe self employment will take off in a big way in India in the next two to five years – therefore, there will be lot of scope for people to be on their own. There is a need for specialized knowledge, speed and flexibility in our industry. Many people are now well connected, well travelled and they know their bearings well. All this means that the situation is ‘cooking’. It is the right time to jump in for the early mover advantage.

However, freelancing is a difficult thing to make it work – anywhere in the world, and especially in India. It is not going to be straightforward.

First, let me dispel a few myths – freelancing is not financially lucrative in the long run. People who work as freelancers do so because they love to be independent, have some time on their hand to pursue various other interests and generally be the trailblazers. It is not a road to make wealth. If you are a Peter Drucker – it may be different, but even for Drucker – he would have made lot more money if he started his own company or worked for some other big company.

There is never any ‘job’ security if you are on your own – it is always a struggle to find the next project, and do it well and have some regular income.

There are a few tax advantages – but the overall expenses far out weigh any tax advantages. For example, you have to invest in your own training, and in your own career. If you work for any large IT-Company, they take care of all your training needs. If you work for a large IT company in India, this amounts to about 50K per year (including the training expenses + the salary you get even though you are attending training). Similarly, there are many other expenses if you are on your own. These days many IT companies provide many perks – which are tax free incentives, you don’t get anything more if you are a freelancer.

Part time job, second income is completely different from being on your own. There is no comparison at all.

There is also a big difference between contracting and consulting. One is getting some work and executing it, and the other is lending your expertise to produce results. These two ways of engagement work very differently.

So, think carefully before making a move. You can always get the first couple of contracts – but can you get business for the rest of your life? I do not want to discourage anyone – but only trying to throw some light on a few dark areas that people generally don’t see.

Now, here are a few tips:

To be successful you have to:

• Establish your credentials – credentials are not your resume, but your expertise and demonstrating that you can deliver. What you can do is to first try and get some work from internet. There are now some websites like rentacoder.com, elance.com etc., - which help freelancers find work across the globe. You can register yourself on these websites. It works pretty much similar to e-bay. Some companies post their requirements – and you can bid for projects, execute them from home. I suggest that you find out more about sites like this, and see if there is some work there that fits your skills and what you are looking for.
• Transition from ‘a skilled person’ to ‘an expert’ – you have to develop certain techniques, a way of doing your work that establishes your expertise. You have to learn to write well, communicate well, publish some papers, have a website of your own and carve out for yourself a niche. There are tons of open sources projects these days – if you participate in these projects, you get to learn the world class standards that are used today, and also you get to meet other people who are freelancers and create a network of your own.
• Have a network of contacts, friends and others from the same area. If you are working in a company – even though you don’t realize, you are part of a community which ensures that you get to know a lot of things from the environment. If you are a freelancer, you have to cultivate different kinds of relationships. There are many ways of accomplishing this – membership in professional bodies, attending conferences, teaching in universities, making contact with a group of people who work in your area (see point no: 2).
• Understand how consulting works. There are some excellent reference books on this subject. Flawless Consulting by Peter Block, Consultant’s calling by Bellman, Soloing are some books that I recommend. Read them, understand them and then think about how to go about it.
• Read a lot. You have to read many different kinds of books – not just technical books. The general thumb rule is that you have to be able to read at least 500 pages of material every week and absorb it. If you claim to be a knowledge worker – you have to have knowledge. Right? Some books I suggest to begin with: Gerald Weinberg’s books on Software Engineering, Quality Software Management and his books on consulting. Also books on agile techniques. Books on systems thinking, history and some poetry. Definitely you have to know a great deal about accounting, managing your own finances and legal aspects of self employment.
• Invest in your own career. This means that you have to have a definite idea of how much free time you will have, and how you will use the free time. If you do billing work all through the year – you will become stagnated very fast, and you can’t get business the next year. Our field evolves very fast, and for a freelancer it is very important to be on top of the technology. Therefore, it is mandatory that you have a definite plan how to stay ahead – this will include, reading a lot, attending workshops and training programs, write and communicate, teach and so on. In financial terms, you have to invest at least 30% of your revenue in your own learning at least in the initial years.

That’s pretty much it. If you are committed to it – it is not hard. Believe in yourself, make a small begining, do it well and don’t give up. That’s all there is to it.

Secrets of Successful Freelancing

Many people ask me about freelancing – how it works, how do I make it work and so on. Most often I duck the question because it is a difficult question. This article is a short guide on how to survive as a freelancer.

It is not easy to be a freelancer. Most freelancers want work, not a job – which makes it tough for them in the long run. They want freedom, they value competency, they think they bring real expertise to the table and want to continue to enjoy the challenge of work.

I always suspect the people who tell me that they enjoy learning. Learning is a painful process, why would anyone want to learn in the first place? We learn best only if we need to. Children learn so fast because it is a question of survival. Psychologists define learning as a permanent change in behavior due to a prior experience. Genetically, we are designed to resist any change – whether it is for good or for worse. Apparently in the same way we have an internal thermostat which controls our temperature – we have another instrument whose function is always bring us back to the original state. This is the reason why changing anything in ourselves is such a massive effort.

Learning is a lot like being in love – and being love is a painful experience. The separation from one’s beloved is a suffering and the lover secretly craves for that rupture involved in that longing. This is a mysterious process – much like the drug addicts cravings for a drug.

In the initial phases of my career as a freelancer, I deluded myself that my clients hire me because of my expertise, knowledge or the skill I possess. I deluded myself that I am expensive because I am very valuable. I deluded myself that doing the best job is the most important thing in the world. I deluded myself that doing the best job on time will get me my payments on time and also bring me fame and respect, and even an extension of the contract.

Well, people value those traits. As freelancers we are respected for our expertise, our knowledge and our skill. Eventually, these qualities might even allows us differentiate ourselves in the market place and allow us to quote a premium for our services. We may be valued for these traits, but we are not hired for these reasons. And, we never retain our contracts for these reasons. We retain a contract only if – as people – we demonstrate very high levels of personal integrity. If we have no personal integrity – then we cannot sustain ourselves as freelancers even for a month.

Clients hire freelancers primarily for business, economic and operational reasons. We are too expensive to be on their employee roles. We are temperamental prima-donnas who are fiercely independent and therefore in the long run we are an expensive maintenance problem that nobody wants to have on their hands. We are addicted to problem solving, and therefore we cannot stay in one organization long enough.

These are the real reasons why we are ‘allowed’ to be freelancers.

It took me a long time to understand this. Most problems between consultants and their clients arise because of ‘expectation’ mismatch. We expect to be valued for our contributions and for our love of work. Organizations value and reward loyalty – not loyalty to work, but loyalty to themselves.

It was a difficult learning. But, after I realized it, it is not too hard to internalize it. The following eleven rules are a direct consequence of this realization. If you practice these rules – you may not become a world class freelancer, but you may be able to survive as one – which is no mean accomplishment in itself.

1. If you are a Socrates, be prepared for a painful exit. Remember - Socrates did not complain about painful exits.
2. Do not blame the stimulus for the response. You are the one who seeks problems – so, never blame the client for giving you the problems. How you experience the problem and your (in)ability to solve it belongs only to you.
3. The estimation of the self must be tempered by the knowledge of the self.
4. A King will never ask his general whether he is ready to defend the country. The fact that the general is still in his job means that he is ready to do so. Therefore, the King will only give marching orders.
5. As a consultant, the only right you have is to do your duty. Always remember that it is your right. Fight for it, protect it but never allow anyone/anything to deny this right to you. Most important – don’t deny it yourself.
6. A freelancer works for cash - not for kind. Therefore, we don’t get paid twice. You can either take a check or receive gratitude. Learn to prefer the check to gratitude.
7. Always do a thankless job. The only thank-you note you can expect is the payment made on time.
8. Be thankful if you get some work to do, and you get regular payments. These are the only two objective measures that you can use to assess yourself. The rest are delusions.
9. The Shareholders control the company, the founders operate the company, the employees are the company, and a consultant is associated with the company. All data modelers know that association is the weakest relationship – it has no integrity constraint imposed on it. It is the easiest and most convenient relationship to terminate. No reason or rationale is required to terminate it – because such a termination does not break down the underlying system.
10. The success of a consultant is directly proportional to how fast he/she makes himself/herself redundant to the client.
1. Solve that problem as quickly as you can and get out.
2. If there is no problem to be solved, get out.
3. If you cannot solve the problem – get out.
11. Luxuries that a freelancer cannot afford:
1. Anger – never walk out of the contract in anger. Don’t leave especially after you have had a fight with your client over some issue.
2. Resignation – an employee can resign. A freelancer cannot. There are only two ways to exit: either complete the assignment, or get fired. Sometimes getting fired is not as bad as we imagine it to be.
3. Procrastination – you sell your time, so it does not belong to you.

The Weinberg touch to Rule No: 11:

I had some difficulty and even used to get angry with the last rule. I felt that there was something wrong with it – it did not sound right. But, I kept it – because, it served me very well in my initial years as a freelancer. My discomfort also meant that there is something deeper lurking inside that rule. I was confident that I will eventually discover what it is.

Recently I posted these rules to Gerald Weinberg’s Shape Forum. Jerry disagreed with this rule and suggested a very subtle shift to this rule. The nudge he provided suddenly threw light in the dark corners of the rule and the full beauty of the rule suddenly became evident to me. So, I dedicated the Rule Eleven to Gerald Weinberg.

I call it the Weinberg Touch:

11a. never leave anything behind – especially your anger.

11b. An employee has an option to resign – a freelancer has an option to renegotiate.

11c. Your non billing time is more valuable than you’re billing time.

11d. Getting fired is never as bad we imagine it to be and it is more often than not a blessing.

Effective application of these rules however requires some tact. Here are two Nasruddin stories that capture the essence of these rules and how to practice them:

******

Whose Servant Am I?
Mulla Nasruddin had become a favorite at Court. He used his position to show up the methods of courtiers.

One day the King was exceptionally hungry. Some aubergines had been so deliciously cooked that he told the palace chief to serve them everyday.

‘Are they not the best vegetables in the world, Mulla?’ he asked Nasruddin.
‘The very best, Majesty’

Five days later, when the aubergines had been served for the tenth meal in succession, the King roared: ‘Take these things away. I HATE them!’

‘They are the worst vegetables in the world, Majesty’ agreed Nasruddin

‘But, Mulla, less than a week ago you said that they are were the very best’

‘I did. But, I am a servant of the King, not of the vegetable’

The Gold, the cloak and the horse

‘I cannot get a job’ said the Mulla, ‘because I am already in the service of the All-Highest’
‘In that case’, said his wife ‘ask for your wages, because every employer must pay’
Quite right, thought Nasruddin
‘I have not been paid simply because I have never asked’, he said aloud
‘Then you had better go and ask’

Nasruddin went into the garden, knelt and cried out: ‘O Allah, send me a hundred pieces of gold, for all my past services are worth at least that much in back pay’

His neighbor, a moneylender, thought he would play a joke on Nasruddin. Taking a bag of hundred gold pieces he threw it down from a window.

Nasruddin stood up with dignity and took the money to his wife. ‘I am one of the saints,’ he told her. ‘Here are my arrears’

She was very impressed.

Presently, made suspicious of by the succession of delivery men carrying food, clothing and furniture into Nasruddin’s house, the neighbor went to get his money back.

‘You heard me calling for it, and now you are pretending it is yours’ said Nasruddin. ‘You shall never have it’.

The neighbor said that he would take Nasruddin to the court of summary jurisdiction.

‘I cannot go like this’, said Nasruddin. ‘I have no suitable clothes, not have I a horse. If we appear together the judge will be prejudiced in your favor by my mean appearance.’

The neighbor took off his own cloak and gave it to Nasruddin, then he mounted him on his own horse, and they went before the judge.

The plaintiff was heard first.

‘What is your defense?’ the magistrate asked Nasruddin.

‘That my neighbor is insane’.

‘What evidence have you, Mulla?’

‘What better than from his own mouth? He thinks that everything belongs to him. If you ask him about my horse, or even my cloak, he will claim them, let alone my gold’

‘But they are mine!’ roared the neighbor.

Case dismissed.

Service Oriented Computing and Tech Shamans

The Design and the Designer – Part 8

There is a lot of talk about Service Oriented Computing these days – mostly by people who don’t seem to understand what it really stands for.

Until now, we have very good models of infrastructure engineering and applications engineering, but we are yet to see a reasonable model for process engineering and structural engineering in the computing science. This could be because of the unfolding of services as a primary business model has just begun and we have not yet understood what service really is.

A very successful billionaire once told me that the Services Automation is several orders of magnitude more significant than the industrial revolution. The explanation he offered for this significance was very striking. It has the beauty of simplicity and the brilliance of Truth to it. According to him, the basis of industrial revolution is control of resources and centralization of knowledge and power. Who ever can control and marshal resources and centralize and guard knowledge becomes wealthy.

But, the services revolution is exactly opposite. Its basis is decentralization of information, power and knowledge. Its basic premise is that we should use resources as when required without really possessing them. Look at the business models of today, especially the Internet. Its basic principle is making information equally available to everyone, and make use of resources as when required without possessing any resource at all.

But, how is this possible? Why would the wealthy people - who became wealthy because of the exact opposite principle - allow such a thing to happen? Why would they want to let go of their control and let go of their wealth?

Well, the answer is that it was not made possible by the wealthy and rich business people. Most of the computing science and the consequent business models are the creations of enlightened hippies. They were the only people capable of such subtlety of thought and action. They even managed to get vast amounts of funding from the very same people that they resolved to dethrone them from their seats of power!!

Let's look at the evolution of computing science, its absorption by the 'industry' and the unseen hand of the hippy.

On one hand, there is IBM and all its 'industrial' representatives and on the other hand there were tech-shamans.

What did IBM and its representatives contribute to the computing science? They tried big, fat main frame machines, resource hogging, hierarchical operating systems like MVS, they tried unusable programming languages like COBOL and they tried to push for data modeling rooted in their three hundred years of arrogance. Now they are trying Java and J2EE. In the software engineering side, they perpetrated process centric methodologies. All these technologies, methodologies, disciplines have one thing in common - they all aim to make the process expensive, aimed to clear the "owner's" inventory and make their clients/customer pay a very heavy price. And, more importantly - they have some kind of 'lock-in' of the customer built into all of them. In the long run - all of them are very expensive - the price paid by people who adopted COBOL during the Y2K issue must be an eye opener. And, who spent the money to resolve the Y2K bug, and who earned it? The people who bought COBOL paid the price, and the people who invented COBOL made the money. The same thing is currently happening with Java and J2EE - all in the name of open computing. The Java technology is an invention of the large computer hardware companies - their aim is to push their hardware, and consequently - the Java Technology demands more and more of 'centralized' computing power. As a language, it is doesn't maintain a clean separation between the virtual machine, the environment and the programming constructs. It looks like SmallTalk, but its philosophy is that of ADA - a big, fat language with too many features and no simplicity. And, today these are the same people who are talking about service oriented architectures and design methodologies. How will it ever work? When they talk about service - they have only one thing in their mind, the customer serving them!! One critical look at the current SOA will reveal that it is a tiger disguised as a holy cow - the SOA wants more and more resources, introduces more and more technologies, centralizes all information and knowledge in the system and very cleanly hands over the customer to the corporations. It is nothing but the mainframe machines, COBOL and process centric methodologies put together in a new disguise.

On the other hand, what was the invention of the tech-shamans of computing science? They invented workstations, powerful user interfaces, networking, distributed computing, cryptography, artificial intelligence, gaming, multi-media, Internet, Unix Operating system, C-like programming languages, hypertext, grid computing and relational data bases, agile programming, extreme programming, design centric processes and problem solving and so on. The list goes on and on. More importantly, they invented open sources programming, making freely available their ideas and designs. Is there any real programmer on earth who learnt his programming from IBM and what it stands for? Is it even possible to learn programming in such closed circuit, big brother environments? All programmers learn programming from the open sources - by contributing to open sources, by using open sources, by reading and following the open sources programming models. One thing common in all their contributions - is a clean separation of concerns, extraordinary simplicity of design, an approach to very clearly communicate the problem-solving techniques, and all these above inventions do not make it heavy on the users - they take responsibility for their own mistakes, they are all designed to tolerate errors from the environment, all of them are extraordinarily reliable and consequently very cheap to own. Neither the inventors nor their inventions charge the customer for their own mistakes. The SOA is known to this community decades ago. In fact, all the systems and technologies built by were all inherently "service-oriented". They are very economical in their resource usage, and they all freely disseminate knowledge and information. I call these people the tech-shamans and hippies of computing science.

I need to explain what I mean by Shaman and Hippy.

In general, the public image of a hippy is a person who is a drug addict and a general misfit in the society, who cannot live in the usual demands of the societal pressures. But, in reality - the hippies are people who are "free" - they are free from the usual tendency to acquire material possessions, they are free from the usual conventions of linear thought processes, and they are free from fear, greed and the usual trappings. They only desire for ecstasy - a few degenerated hippies may get this ecstasy from drugs - but the majority of them get it from problem-solving, from the joy of work. They do not demand any other form of payment. For them, money, power, fame etc., are all byproducts. Their main interest is in the 'kick' they get in understanding, the suffering they go through in the effort required to assimilate and internalize information and in problem-solving - the application of knowledge. A recent psychological survey revealed that problem-solving – working on difficult problems – releases the same chemicals that are involved in drugs – it produces a ‘kick’ very similar to drugs and sex.

Shamans are evolved ‘hippies’ – in a way hippies of the highest order. He could sit with a plant for a month, talk to it to find out whether its herbs could be eaten. Finally he may eat the herb and die – that would be his contribution to humanity. On the other hand, there are priests – the organized institutional worker - working for control and centralization as opposed to the Shaman who acts as a bridge between two worlds.

In no other field of human Endeavour, the fight between the free spirited hippy and the controlling industrialist are more marked and visible than in the computing science. The history of computing science is littered with records of such fiercely fought battles - the failure and success of Multix and UNIX, the great debate on relational and hierarchical data models, between what is ugly and beautiful and so on. The hippies had Dijkstra, Knuth, Ken Thompson, Tony Hoare, Writh, and E.F. Codd as their keepers of faith and the industrialists had large bank accounts. The history of human beings recorded rather well that it is fearlessness and truth that win the war, not power, money and ignorance. The result is well known even before the war started.

But, this time the hippies learned their lessons from history. They were no longer interested in fighting useless battles. By definition, their knowledge and inventions are available to every one - including the control minded industrialist. The evolution of computing science really nothing but a statement of how the industry assimilated the contribution of the hippies and shamans, and is slowly and painfully learning to respect them and their contributions.

Some examples and illustrations are probably required at this stage.

First, let's take Object Oriented Programming. For any normal human being, objectivity is the hardest thing to understand. Many philosophers, over the last few millenniums devoted their entire lives to understand and explain to us objectivity. The aim of science is an objective understanding and explanation of the reality - did it succeed?

Some tech shamans like Marvin Minsky and others - unleashed in the early sixties - some thing called Frames as a knowledge representation scheme, and some other shamans developed SmallTalk programming language. Even the names give their Shamanistic origins. So, object oriented programming was born. This was a very original idea - for those who understand it, it appeared very simple, and for those who did not understand it - it was an alluring mystery, whose secrets must be cracked at any cost. This division continues even till today - very few programmers really understand object oriented programming - the rest of them use UML. The object oriented analysis, modeling and the modeling languages are nothing but the industry's attempt to assimilate a very hippy concept. After all the literature, tools, systems, training courses that the industry developed for Object Oriented Programming - no one can still explain 'how to' identify and design an 'Object'. This is still the best kept secret of programming - passed on from the Master to his most accomplished disciple.

Again in the Sixties, the patron saints of tech-shamans - Dijkstra and Tony Hoare - unleashed some thing called concurrent or non-linear programming. The concept simply is that it is more efficient use of resources if programming is relieved of its tiresome linearity. How many people can think non-linearly? The shaman hardware designers went ahead and fundamentally altered the way the machine is constructed - they made the machines interrupt driven, in other words, the behavior of the machine is non-deterministic and thus non-linear. Countless coding soldiers wasted probably billions of lines of code trying to understand how to configure such a non-deterministic machine. If you look carefully at the core of the unreliable software - you can trace it to the so called 'dynamic' aspects of the environment - aspects that are not known apriori. How can you design a program for an environment that you cannot predict it at the time of writing the program? For the shamans - it is a simple problem with an obvious solution, and for a control minded industrial freak - it is unthinkable.

One thing is very obvious. More than seventy percent of the programming projects undertaken by the industry are failures - they run into too many technical and budget problems, they run into performance and reliability problems. But, name one open sources project that has any of these problems?

The industry tried to tackle this problem using their age old technique and very old, useless thinking tools. They created software engineering methodologies and they write tons of documentation of their technical designs and programs, they spend millions of dollars in testing. But, for some mysterious reason - open sources systems do not have any such things. They don't come with tons of specification documents, high level designs, pseudo code etc.

Anywhere you care to look - this division is obvious. There are two streams of computing science and two streams of people. Let me call them programmers and coders, or if you don't mind - shamans and priests. Even computing literature can be divided into two classes - the shaman literature and the priest literature. This division is very marked especially in Software Engineering.

Software Engineering is perhaps the only discipline that aims to teach problem-solving. Its interest is not in providing some specific solutions to specific problems, but to teach problem-solving. But, the big, fat books on Software Engineering written by priests do not get the central idea. They only confuse it even further – they describe methodologies and processes, basically they preach what people should be doing, without even giving a hint of how to do something.

The best software engineering books were written by Shamans – people like Dijkstra, Knuth, Weinberg, Rob Pike, and Bertrand Mayer and so on. Knuth’s 70 page paper on Errors of Tex is perhaps the best description of the software engineering process ever published. These are works that are full of insights and techniques – not some religious preaching.

But, what is the secret of the shaman?

The secret is - as all best kept secrets in the history are - is very simple and very obvious. The tech-shamans are basically builders and engineers. The metaphorical links between Free Masons and Free Software Foundation is hard to miss. They understand the process of design, they understand engineering as a discipline, and they understand that the process of making something and its final object of creation are two completely different things altogether.

The Alchemy of Design and the Sound of Music

The Design and the Designer – Part 7

In the last post, I made some observations about the process of Architecture – very briefly I stated that Architecture is the process of bringing meaning to space, and expressing the meaning in terms of certain engineering transformations, which transform the problem into an engineering problem.

The process of design – transforming ‘meaning’ into a set of engineering problems is very analogous to music composition. The difference between music and sound is the same as the difference between a sweet home and a concrete building. Therefore, let’s try and understand how music is composed. Later, we will define what engineering really is, and then I shall describe the relationship between design and engineering.

It is very simple to explain the physics of Sound. Sound is produced when two objects make contact with each other. Sound consists of a series of waves that ‘move’ the air in a particular way. The two major characteristics of Sound are its amplitude and frequency plotted on a time scale. That’s pretty much it.

What is Music? What makes certain sounds musical and certain other types of sounds non-musical? Can we say that Music is the sound we like? What do we ‘mean’ by liking? We use many different ‘esoteric’ words associated with Music, but not with Sound – words like melody, harmony, rhythm, tempo, beauty and balance.

A music composer somehow ‘transforms’ these musical attributes into a set of sound waves. The music theory build up over thousands of years provides some guide lines of how this transformation could be achieved. The music theory – whether it is western music, or Indian music theory – is a body of knowledge that codifies how music can be produced using sound. This body of knowledge can be divided into two distinct categories – the first category enables a music composer to ‘recognize’ his/her creative impulse. This is the body of knowledge that deals with recognizing music. This body of knowledge does not talk about frequencies, amplitude or rhythm, but it deals with characterizing the emotional qualities of music, and tries to categorize them.

In Indian Music – the concept of Raga for example deals with systematizing the ‘melody’ into a set of information structures. A Raga is therefore nothing but a shared understanding of melody and how to express it objectively. For those who are not familiar with Indian Classical music, here is a very brief explanation of Raga.

A Raga defines a set of melodic patterns and a melodic structure. For example, if you take a Raga like Mohanam – it is used to express the feelings of love and romance. Love and Romance are very ‘semantic’ concepts. What the Raga manages to do is to transform these concepts into a melodic structure and a set of melodic patterns. So, a composer whose creative impulse connects him with love or romance, he can then use the codified body of knowledge of Raga Mohanam, and compose music in that Raga.

Melodic Structure: This deals with the ‘rasa’ or ‘experience’ of music. There are about nine primary ‘rasas’ and another nine sub categories. There are all kinds of permutations and combinations of these primary experiences are possible, but any composition’s melodic aspects can be explained using these ‘rasas’. Apart from the Rasas, there is a ‘presentation’ aspect of the experience. The presentation is further divided into three categories: Majestic, Mellifluous, Simple (the Sanskrit Terms are Ghana Ragas, Rakti Ragas and Desi Ragas).

Melodic Patterns: The human ear is apparently very logarithmic. All music uses a concept of Octave. An octave is a predefined set of ‘notes’ or ‘frequencies’. The most frequently used octave all over the world uses 12 notes. Basically, you start from a certain base frequency, and keep increasing the frequency for every subsequent note, and by the time the octave is completed, you would have doubled the original frequency. Basically a Raga is a particular subset from an octave (7, 6 or 5 notes). It defines how to move back and forth along the octave (called Ascent and Descent). It also defines certain relationships between the notes – these are called ‘sangati’ and ‘sancharam’. In other words, a Raga defines – purely theoretically – a certain ‘musical movement’ (chalan), and also gives a set of ‘musical phrases’ that could be used to create a certain type of experience. A ‘raga’ therefore defines both the melodic structure and the melodic pattern.

The beauty of a raga is that it is just a pattern – without any ‘fixed’ compositional aspect to it. The same pattern can be used in millions of different ways – giving rise to a wide variety of compositions. A raga is thus a design pattern or a ‘semantic abstraction’. In the same way that a bird does not exist any where in the physical world, but there are millions of manifestations of the bird, a raga has no physical existence – it is just a useful abstraction.

The second category of Music Composition is closer to the characteristics of sound – and this acts a bridge between the musical characteristics and sound characteristics. For example, almost all musical traditions have a concept of an octave and beat cycles. An octave is an information structure and a very useful abstraction. (Please refer to my earlier posts for a detailed discussion on abstraction, semantic types and concepts). This body of knowledge of basically deals with the structural aspects of music composition.

Music composition therefore is very similar to design and engineering. There is an understanding of meaning and transforming that understanding into a set of structural transformations. Perhaps this is the reason why architecture is followed by structural engineering, and there is a very strong connection between architecture and structural engineering.

In the next post, we will define structural engineering, and its relationship to design and architecture.