The recipe for Coca-Cola is one of the most closely guarded secrets in the world. Yet a small Canadian software firm has sold 150,000 cans of a rival fizzy cola, which tastes very like Coke, and has made the recipe public. The firm behind the drink, Opencola, makes software, not drinks. It used the drink (and its open recipe) as a metaphor for the most important trend in software today.
Unlike most traditional software firms, Opencola produces open source technologies. Open source is a philosophy for software licensing designed to encourage the improvement and use of software by anyone who wants to join in. It ensures that the source code, the underlying instructions of the software, can be examined and modified freely.
The open source movement eschews proprietary controls and its software is usually produced not by firms, but by networks of volunteers who look after different pieces of an application. For this reason it has, until recently, been regarded as anti-corporate-associated with hackers' bedrooms and academia, an eccentric corner of the market.
Today, open source has grown up and has an uncontested momentum in several key areas of the software business. Linux, an operating system that competes in Microsoft's dominant market, has gained a beachhead in many companies. Oracle's dominance in databases is coming under threat from MySQL, whose software was downloaded over the internet around 10m times last year. And nearly 70 per cent of the computers that serve web pages run Apache, an open source application. Almost three quarters of large US companies intend to increase their use of open source technologies, according to Forrester Research.
In its February 2003 filing to the securities and exchange commission, which regulates stock markets in the US, Microsoft admitted that open source posed a significant challenge to its business model. And it is starting to fight back. Microsoft's original attack on Linux concerned security. But that has backfired. The growth in internet viruses that hop from Microsoft server to Microsoft server has demonstrated that a product with a single architect, like Windows, is more vulnerable than one, like Linux, with many. The open source philosophy means that although the code is open to malicious actors, it is also accessible to more friendly eyes, which can track down and resolve problems quickly.
Steve Ballmer, Microsoft's aggressive chief executive, once called open source a "cancer." But other blue chips have rallied to the Linux flag. Three years ago IBM announced it was investing $1bn in consulting and support services for Linux-based systems. The greatest concern large companies have about software is support: who will fix problems when they arise? IBM's decision to offer support contracts has calmed those fears, and partly for that reason key components of the New York stock exchange now run on Linux.
In a few years, Linux-based computers will handle many of our telephone calls. The new range of telecoms systems from Nokia, the mobile telephone company, will run Linux. And now governments are questioning the need to stretch treasuries to finance proprietary software when free versions are available. When the Munich city government hinted it would dump Microsoft in favour of Linux, Ballmer cut short a holiday to petition the mayor. The intervention was in vain. Last May the city announced that it would install open source software on its 14,000 machines. The Microsoft contract, for licences, training and support, was initially worth $36.7m according to USA Today. The winning bid from IBM and the German Linux group Suse was actually $39.5m, higher than Microsoft's. But Microsoft's strategy of introducing expensive upgrades every few years adds a financial risk for any purchaser. "On strategic issues, it was clearly open source," says Harry Maack, who advised the city.
In November, Sergio Amadeu, Brazil's top technology mandarin, declared his wish to "create a continent" of open source in the federal government. And this January, the Israeli government announced it would stop buying Microsoft software in favour of open source.
In Britain, the offices of government commerce and the e-envoy are now backing trials of Linux-based applications. Last October, Whitehall announced trials of open source applications on civil servants' computers. If successful, Microsoft software on 1.3m government desktops might be replaced with open source alternatives. The savings on licence fees could run into tens of millions.
When a company or government department wishes to use a piece of open source software, all it needs to do is find it on the web and download it for free. This has not changed over the past decade; what has changed is the back-up support and consultancy, which, as the example of Munich shows, is far from free. Big technology companies like IBM and Linux-support specialists like Red Hat provide company IT directors with the expertise to customise open source applications to their needs and the reassurance that when things break down there will be help.
Origins of open source
Open source can be traced back 20 years, to the time when only a few dozen machines in the US research community were connected to the internet. The scientists managing applications on the network were hackers-not in the sense of cyberspace criminals, but in the sense of programmers who solve software problems.
These people wrote the ancestors to software used today in the internet without much thought to ownership. For the network to work, the computers needed to talk to each other. And the easiest way to do that was to share. So Sendmail, written by Eric Allman in 1979, delivers the bulk of email sent on the net today. Bind, which powers the domain name system, allowing us to use addresses like www.google.com rather than the underlying machine address, was created by a team at Berkeley. (You can bypass Bind: reach Google, for instance, by typing in 216.239.59.99.)
By 1984, things had changed. Companies had begun to bury software under patents and copyrights, keeping the source code secret. Richard Stallman, an MIT computer scientist, believed this was damaging the industry. It stopped the free flow of ideas, because without the source code, programmers could only see what a program did-not how it did it. Stallman developed the idea of distributing free software with its source code and a licence that allowed you to modify the source code as long as the modifications were kept in the public domain. Software needed to be free: "Free as in free speech, not free beer," as Stallman likes to put it. The licence was known as the GNU General Public Licence (GPL).
The most famous software issued under the GPL is Linux, an operating system created in the early 1990s by a young Finn, Linus Torvalds. Linux is older than the term "open source," which was coined in 1998 by Christine Peterson, a nanotechnology expert, and was intended to scare business less than Stallman's techno-utopian "free software." Open source embraces a variety of licensing regimes but what they have in common is that the underlying source code is shared and visible to all.
The design and development process is radical too. Programmer Eric Raymond has characterised it as a "bazaar" of chattering opinions. Commercial software is normally designed like a cathedral, with a single architect directing the operation top-down. By contrast, open source is a bottom-up development, with people volunteering to do what they do best. This is not anarchy. Work on the software is usually guided by an ultimate arbiter, normally the person who started it all off. This arbiter resolves conflicts which contributors haven't managed to sort out among themselves.
In the case of Linux this is Torvalds, working in his spare time, and a group of lieutenants. In practice, though, most work gets done without Torvalds casting an eye on it. Linux users themselves provide quality control: downloading updates to the software and hunting for bugs. In Raymond's words, "with enough eyeballs, all bugs are shallow."
Commons-based peer production
Yochai Benkler, a law professor at Yale University, has called this "commons-based peer production." The commons refers to the sharing of the underlying code or the output that is open to all, akin to the public land that farmers once grazed their livestock upon. Peer production means that producers participate for their own varied reasons and in ad hoc ways, not necessarily via legal contract or management fiat. Benkler calls this a third mode of production for the market, distinct from the company and the "spot market" (or, in employment terms, the freelancer). Open source shows that it is possible for part of the economy to function without companies but with many self-employed individuals contracting with each other.
Benkler cites examples of areas in which commons-based peer production is superior. In one case, Nasa used volunteers to identify geological features on Mars. The project, called Clickworkers, allowed anyone to look at images of sections of the planet and identify features. Around 85,000 people took part, and nearly 2m images were checked. According to researchers, the result was "virtually indistinguishable from the inputs of a geologist with years of experience."
As a template, Clickworkers has its limitations. Each individual task was discrete and required little expertise other than common sense and the ability to control a mouse. It would be more challenging, say, to get volunteers to write an encyclopedia. But that, remarkably, is what Jimmy Wales, a Florida-based entrepreneur, has achieved. For the past three years he has been running Wikipedia, an internet-based, open source encyclopedia.
An encyclopedia normally takes years to create. World experts write the articles and an editorial committee reviews them. An encyclopedia written by the average internet user in under three years does not sound promising. But Wikipedia appears to work. There are almost 200,000 entries created by thousands of different people. The website now receives more visitors than Encylopaedia Britannica's online edition. And unlike the Britannica, parts of it are available in 55 languages.
Wikipedia is Wales's second attempt at creating a public domain reference work. For his first attempt, Nupedia, he insisted potential contributors provide credentials before they could participate. After two years he had 12 articles to show for his efforts. Contributors just couldn't be bothered with the rigmarole of identifying themselves. So Wikipedia has no controls. Anyone can visit the site and create or edit an entry. You might think this is a recipe for mayhem. In fact, it provides for an open peer review of every article. Monomaniacs obsessed with a single issue, like Holocaust denial, can cause problems. But ultimately Wikipedia has a gatekeeper in the form of Wales himself, and he can take steps to exclude troublemakers.
Education is another fruitful area for open source. In 2001 the Massachusetts Institute of Technology unveiled OpenCourseWare, which publishes the syllabuses, lecture notes, reading lists and even student solutions to 500 different courses. While MIT retains copyright, it allows anyone to access the courses. The institution is clearly confident that the value it adds lies not in the written part of the courses but in the teaching and the environment, which students can get only by attending and paying fees. By contrast, Oxford limits access even to its exam papers to university members.
Several organisations are now compiling class notes for all educational levels and releasing them under a form of public licence. The aim is to provide teachers with access to resources that don't tax school budgets. They include Commontext and the Open textbook project, both launching this year.
Academic journal publishing is another business targeted by open access. The market is a cartel, dominated by Reed Elsevier, Springer-Verlag and a few others. Many academic institutions complain of the high price of subscribing to these publishers' journals. The open access movement aims to increase distribution of research through journals unhampered by restrictive licensing regimes and high subscription costs-indeed, it has been doing so for several years, and some of them have become essential sources for scientists.
Open access journals usually charge authors for each published article rather than readers for subscriptions. Charges for BioMed Central, a biosciences open access publisher, are around $500 an article, which covers peer review, editing and distribution costs. The journals themselves are available free online. This is an appropriate model for scientists whose objectives in publishing are to establish reputations, secure funding sources and add to the body of knowledge. A subscription fee is a barrier to at least two of these objectives, while author charges reflect the benefits to researchers of publication.
In December, the British parliament's science and technology committee announced an inquiry into scientific publishing, and in particular the cost of journals and the extent to which open access journals could replace them. This follows a Royal Society report which called for reform of the intellectual property system to achieve greater openness for scientific literature.
Benefits, motives and public goods
Why do people join open source projects without financial reward? There are many reasons, including enjoyment and the acclaim of peers. The latter motive, especially in fields like software, can also overlap with opportunities to demonstrate talents to employers.
One of the best technology magazines on the web, Slashdot, has only a few members of staff who post short articles and allow readers to comment and elaborate: most of the site content comes from readers. In South Korea, OhMyNews, a leading online newspaper, follows a similar model. Its 50 editorial staff are supported by 27,000 volunteer "citizen editors," who produce nearly 200 stories a day, most of which are published. In a typical edition, these volunteers contribute four fifths of the stories. OhMyNews is no bit player-it outstripped its conservative rivals, Chosun Ilbo, JoongAng Ilbo and Dong-A Ilbo, in its coverage of Roh Moo-hyun's election campaign in late 2002. Its reward was the first interview with the new president. While OhMyNews does pay its contributors, it doesn't pay much. A typical story will earn you less than $1, a front-page story about $16. Volunteers are driven by a mix of non-financial motives.
Several of the above publishing examples are not strictly "open source" in the sense of projects based on the idea of continual improvement by self-selected individuals whose enhancements must be freely shared. They are more accurately described as "open access" projects, in many cases only made possible by the internet. But they overlap with open source practice in two ways. The first is that they provide a public good that can be exploited by all. The second is the idea that those contributing to the public good may want to do so for non-financial rewards.
Even in sectors where open source has a significant role, private companies will often be more appropriate for a task. Billing software, for example, is a staple for business but boring and unchallenging for hackers. Without the non-financial incentives, the open source sector may not be able to produce a solution as easily as the private sector. And in practice, open source communities are sometimes more like private companies than they seem. Studies of open source projects show that a few contributors do most of the important work. The rest test the software and spot bugs. Nevertheless, the sector is an important new means of creating public goods.
Another open source related public good is the human genome project. Britain's Wellcome Trust invested ?210m in the project so that data would be freely available in the public domain. The trust's Sanger Centre led the work, partly in response to the activities of Celera Genomics, an American company that charges for access to human genome data. The Sanger database is being used commercially by companies, such as GlaxoSmithKline and Oxford Glycosciences, to help them find better drugs.
Public goods - from roads to academic research - have long been provided by the state or by philanthropists. But even competitive private businesses can produce a variant of public goods when they collaborate. Between the 1860s and 1880s, competing American steelmakers clubbed together in a process of "collective invention" and shared their patents for making steel using the new Bessemer process. At the time, demand for steel was exploding, fuelled by the westward unrolling of the railroad. Once a process was proven to work in one company, it could be put to work in another.
Collective invention is also taking place today. In 1998, Nokia, Ericsson, Motorola and Psion set up Symbian, a joint venture to build software for the mobile telephone of the future. Although Motorola has now left the group, Symbian has added Panasonic, Samsung and Siemens to its shareholder list.
What does it mean for Microsoft?
Open source draws on some of these collaborative traditions but remains a highly distinctive phenomenon. It has come of age thanks to global electronic networks that make exchanging information cheaper than ever before, at a time when the market for information-based products, whether software, music or opinion, is larger than ever. As James Boyle, a law professor at Duke University, argues: "A large, leaky market may actually produce more revenue than a small, tightly controlled market."
But because the open source movement is so closely tied to broadening access to intellectual property, it has unavoidably collided with the interests of powerful companies. This has led to a perception of open source idealists ranged against hard-headed companies out to protect their profits.
This view is out of date. Open source has at least three benefits for conventional enterprises. First, it creates new wealth and new businesses, such as the Linux-based consultancy Red Hat, which now has an annual turnover of $100m. Second, it stops companies reinventing the wheel in areas of open source research, and thus releases time and money for more fruitful activity. Third, open source's decentralised methods may, as we have seen, produce results otherwise unobtainable. Member-based websites are yet another example. Friends Reunited has assembled a database of the school histories of 9m Britons. This is unavailable anywhere else and the cost of compiling it has been distributed among its members, who fill out their own details.
For Microsoft, the open source threat still looms large. The company is trying in a number of ways-from price-cutting to aggressive advertising-to draw attention to the potential failings of open source software. But it has also paid its rival the compliment of imitation. Through "shared source licensing," unveiled in 2001, it allows large clients, such as governments, to inspect the source code of its core operating systems. It is also courting small developers by allowing them to inspect the source code of Windows CE, the operating system for handheld computers. The source to Windows CE has been downloaded nearly 180,000 times by developers keen to understand how it works. It isn't full open source, but it is a respectful nod towards it.
The symbiosis is working in the other direction too. Organisations in the open source market are adapting more commercial strategies. MySQL, for example, has developed dual licensing. It offers its products under an open source licence for non-commercial use and insists that any changes to the code be passed into the public domain. But it also allows companies that want to tinker and keep those enhancements secret to purchase a commercial licence for anything from a few hundred to several thousand pounds.
Open source will continue to squeeze the income of companies like Microsoft and Oracle. Financial Insights estimates that US financial services firms invested nearly $400m in Linux-based systems last year and will more than quadruple their annual investment within four years. Customers will continue to defect, forcing Microsoft to cut prices. However, it is also likely that proprietary software will continue to play a dominant role in high-value applications. A private company may be better placed to perfect a specific expertise, say scaling a database to hold trillions of records, than a decentralised network of volunteers. It is no surprise that open source software is as good or better than commodity software-like web servers or operating systems-but nowhere near as good for highly specialised applications such as video editing or really big databases.
For many popular desktop applications, such as word processors, open source software often follows the innovations made in proprietary software. And so Microsoft Word remains a more polished product than its open source rivals. But it is by no means clear that all buyers, especially organisations, choose software solely on the basis of their technical merits. What an open source application may lack in refinement, it may make up for in its friendly licensing regime, its ability to run on older or cheaper computers or the fact that its users are not dependent on the strategic choices of a single company-Microsoft, Oracle or whoever.
And in many other areas, open source software may lead the way. Thus the first web servers were (and have remained) open source, with commercial rivals appearing years later.
In coming of age, the open source sector has shown that there is room for a wide variety of commercial strategies, on a spectrum from the highly secretive to the completely open. And it has shown, not for the first time in business history, that sharing knowledge and collective research can be of benefit even in a competitive market system. Co-operation can work in the very heart of 21st-century capitalism.
