Too much choice

Eight years ago I wrote an article in Prospect (January 1997) attacking the idea that ever-increasing consumer choice was a panacea for happiness. I hardly addressed the extension of the notion of choice to the public sector because it did not occur to me it might become a serious policy prescription.

What did I know? Today, less than a decade later, the extension of individual choice in the public services is one of the main policy initiatives of this third- term Labour government. In Labour's first term there were few harbingers of this. The government chose, for good reason, not to increase spending for two years. Since effective choice requires an excess of provision over demand—that is to say it requires increased spending—it was not high on the agenda. But the second term saw the descent of a Niagara of public money, into health and education in particular.

This did not lead to a complacent defence of the public services as they had been run. The lessons of America's "new public management" had been absorbed: there is state failure as well as market failure. What could readily be targeted was targeted. What could sensibly be privatised was privatised. Internal markets did what internal markets do best.



Most of this made sense. For every British Rail, where the complexities of the new systems went wrong, there were several British Telecoms, where competition and reform worked. In health the waiting list target was crude, but it was a price worth paying for tumbling queues. Without the internal market in health (which Labour triumphantly abolished then sheepishly restored), the extra resources would have seeped out into myriad inefficiencies.

If anyone doubts the value of the government's reform of the public services, they should visit Wales, where I live. The Welsh and their assembly have a traditional view of how services should be provided. As a result, the health service in Wales is now performing far less well than in England.

There is a place for competition and even choice in public services. For example, a primary care trust (PCT) responsible for commissioning care for its area can now seek quotes from a variety of hospitals for treatments or operations that do not exceed a nationally set maximum.

This is sensible competition to drive down costs and contain producer power. Without some such mechanism, inefficient hospitals that run up large deficits are rewarded and those that produce surpluses are punished. (One effect of the old system was to cause the extra NHS money channelled northwards—intended to combat social class inequalities—to flow back southwards.)

Note that these desirable reforms are not the result of individual choice. They are the result of choice exercised by a public body on behalf of its community. This kind of reform is now widely accepted. In England at least, we are all modernisers now—bar some self-interested trade union officials, conservative old Labourites and the far left.

Beyond such modernisers, however, is another group of "super-modernisers" who believe that the future of public services lies in the introduction, wherever possible, of individual consumer choice, analogous with the private sector. Its most distinguished theorist is the academic Julian Le Grand, now back at the LSE after a stint at No 10. This view dominates No 10 (though not No 11) Downing Street, where one of its main proponents is Philip Collins, Tony Blair's speechwriter. It is also prevalent among most of the cabinet and the ministerial teams in the relevant departments, but not yet among Labour MPs nor what used to be called the Labour movement.

As Blair told Labour's national policy forum (NPF) on 23rd July 2005, "In every other walk of life, the position of the consumer had been revolutionised. As their demands changed and became more individualised, so producers of goods and services were forced to customise, to adapt and be flexible… The danger in unreformed public services is clear."

This is a peculiar argument. For the superiority of the market in the private sector depends on two preconditions. The first is that the market is embedded in a wider social system that does not generate a set of inequalities which are morally or socially intolerable. We are happy for the rich to consume 100 times more caviar than the poor. We are not content for them to consume 100 times more education.

The second even more fundamental precondition is that consumers in the private sector face a set of prices and a budget constraint which between them determine what their income will buy. This doesn't apply to publicly funded services. There is a budget constraint, but it is set by the capacity to raise taxes, not by individuals. As services such as education and health are free, the price mechanism does not apply.

Thus the analogy with markets is flawed. But that is not the end of the story. For the super-modernisers ignore a further flaw. It is the existence in public services of market failure. For example:

Imperfect information The economics of choice depends on the assumption that the consumer is, or at low cost can become, reasonably well informed about the choice he or she makes.

Agency The person making the choice must be the person most affected by it. If I decide what to buy for someone without knowing what that person wants, there is no reason to suppose that welfare is optimised.

Externalities Externalities are common in the market sector. For example, if I drive to work rather than take the bus, I cause pollution and congestion. For this reason, society uses devices such as fuel duty and congestion charging, which are designed to equate the costs and benefits to me with the costs and benefits of my actions to society.

For most of the market sector these limitations do not intrude too much, and free choice will generally provide efficient outcomes. For public services, however, they pose some extraordinary difficulties—which is why we chose to provide some things publicly in the first place. (This incidentally leads me to predict with confidence what at least one part of Labour's individual choice lobby will propose when their remedy fails to bring about the changes they expect. They will want to have the goods provided in the public sector charged for just as they are in the private sector.)

Perfect information as to the best health treatment, for example, is hard to obtain and impossible for those without medical expertise to evaluate. There can therefore be no presumption that consumer choice optimises outcomes in health.

Agency too is problematic. Children do not choose schools; parents do. Parents have certain advantages in terms of choice for their children; they will know most about their characters. However, parents cannot be assumed to be unbiased in the choices they make. Their own future financial security will be greater if they choose education that means their children get well-paid jobs rather than education that makes their children happy. It follows that it is not necessarily the case that parents on their own are better at choosing the right education for their children than anyone else; professionals have a role.

Broadcasting provides a good illustration of the externality problem. If fewer and fewer people watch a common channel, such as that provided free-to-air by the BBC, then the common culture, an aspect of social capital, is eroded. So my decision to take Sky has costs that extend beyond me. My extra choice of programmes may mean that the BBC is unable to justify spending as much on a particular programme as it otherwise would. That imposes costs on others who no longer enjoy the choice and the benefits of watching that programme.

Such considerations show why there can be no automatic presumption that individual choice maximises either efficiency or welfare in public services. Moreover, when everyone in the public sector tries to maximise their personal welfare through the exercise of individual choice, it may have unpredictable consequences on institutions like schools and hospitals. The effect may be to give people less of what they want. Competition and certain kinds of choice can work but the decision has to be made on the facts of each case, weighing gains and losses.

Let us apply some of this thinking to actual examples in health and education. Health first. Three successive health secretaries have put individual choice at the centre of NHS reform. But it is not clear what that means.

Take a patient who has just been told he has cancer. Here are two kinds of choice he could be offered. The first is a choice of treatment. No longer would the doctor say: "It's a big operation and chemotherapy for you, Mr Smith." They would instead say that you can have chemotherapy, with a 10 per cent chance of extending your life by three years but at the cost of a wretched time in the meantime, or palliative treatment only, which will be less unpleasant but will probably end in an earlier death. Since only you can know what you think suits you, you are in some sense best placed to make that judgement. (Even here the case is not altogether clear-cut. In The Paradox of Choice, Barry Schwartz cites an American study which found that 65 per cent of people said they would want to choose their treatment if they got cancer, but only 12 per cent of actual cancer patients said they wanted the same freedom.) The safest conclusion is that choice of treatment is probably a good thing, but by no means the most important consideration in patients' minds.

The second kind of choice, however, is the one on which the government is now focusing: that which involves giving patients a choice of whom they want to provide their treatment. From the end of 2005, patients needing hospital treatment will be able to choose between four to five providers, selected by their PCT. From 2008 they will be able to choose any provider, if it can do the job within the set price.

Leave aside whether we want hospitals to go bust—is this choice one that patients will welcome? It is not easy for anyone to assess whether one hospital is better than another. Performance tables are of limited utility. Just as schools' performance depends on the background of the children who attend them, so health performance depends on the patients they treat. There are ways to compensate for variations in intake to produce more comparable figures, but none is wholly satisfactory.

Choice is not free to the chooser. It requires an investment of time and effort to gather information. In the case of the things we buy in the supermarket that choice may not be very onerous. Besides, capitalism has found ways (such as branding) of reducing those costs to manageable levels. But if what is at stake is literally a matter of life or death, taking a decision may be very onerous. Even if you think the result is likely to increase efficiency, account should be taken of the burden of choosing.

The individual choice lobby recognises the problem of lack of information. It talks about creating networks of personal advisers to help people to understand better the choices they face. But this is to devote resources to a problem that they have created. Under the existing system, after all, everyone has a personal adviser already: their doctor. And in practice it is likely to be your doctor who will choose the hospital you go to. You should have a right to be informed of his or her grounds for making that choice and, ultimately, to be able to ask him or her to try to get you in somewhere else. This is still choice—and a welcome change from the previous monolithic reality of much of the public sector—but it takes account of real individuals and their circumstances.

There is another problem with making individual choice too central to public service reform. If choice is to be real it has to be effective. It is no good being allowed to choose a school for your child if all the places at the school you would like have been taken. Indeed, if you are offered a choice, and then find you cannot exercise it, you will be even less satisfied than if you had not been offered the choice at all.

In a Guardian article in May, James Meek analysed secondary schools in Birmingham. There were, he said, 76 secondary schools, so it seemed that a wide choice was on offer. But the 76 include eight grammar schools, nine Catholic schools, one Anglican and one Muslim, for which entrants need to be qualified by exam success or faith. A further 38 only take pupils from nearby, and so on. In fact, only 15 Birmingham schools offer places to anyone at all. "If you're poor and secular," Meek concludes, "it doesn't matter much what you put on the preference form—you'd better hope there's a decent school on your doorstep."

If choice is to be worth anything, it has to make a difference to what you get. However, a corollary of this is that there must be over-provision. So if you are to have a real choice between being treated in your local hospital or at another farther afield, places have to be available in each. Since capacity is expensive, that will increase the cost of the service.

As Michael Bichard, the former permanent secretary in the education department, pointed out recently, choice and efficiency in public services can conflict. "If you build capacity you must to some extent be in conflict with achieving… efficiencies. Choice of provider requires some slack…." The question then becomes: are the extra efficiencies you get from competition and choice greater than the inefficiencies required by having plenty of slack in the system?

This is not, of course, an issue in the private sector. The model for the private sector works because the things chosen are priced, and because people buying them face individual budget constraints. In the public sector most things are not priced, and the budget constraint is collective.

John Appleby, Anthony Harrison and Nancy Devlin wrote a King's Fund study called "What is the real cost of more patient choice?" They concluded: "There is an irreconcilable conflict… between allowing individual patients unconstrained choice of treatments that are free at the point of consumption, and the allocation of resources in a cost-effective manner. Individuals may choose treatments that are most effective (and that best meet their preferences) but not the most cost-effective (or that reflect the preferences of society as a whole)—with corresponding opportunity costs in terms of health gain foregone by other patients. "

Will expanding individual choice produce more equality in the consumption of public services? Advocates of individual choice argue that it will, for two reasons. First, they want choice precisely because it panders to the middle class, who will otherwise turn increasingly to the private sector for their needs. "The well-off… have choice," Blair suggested in his NPF speech. "They can go private and pay." The public sector will be left as a rump service providing poor services for poor people. But this threat of middle class opt-out is exaggerated. Just 7 per cent of pupils are educated privately and just 11 per cent of the population have private health insurance (the numbers for both are higher in London, upon which most public policy is based). Those numbers have been broadly stagnant for some time, which is not surprising given the cost of a private education or private health insurance. Of course, they could increase if cheap and cheerful private schools emerge on a significant scale, but there is no evidence so far that they will.

The second reason that individual choice is good for equality, according to its supporters, is that the middle class do better out of the education and health service as it now stands because they are better at lobbying and navigating around the bureaucracy. With more choice that advantage would be reduced. In the jargon, "voice"-driven systems tend to favour the better off, while the chance of "exit," which in theory choice provides, helps the less well off.

But there are some good reasons to believe that in practice, more choice will have the opposite effect. If the best-informed parents choose school A over school B, then school A is likely to have a better intake than its rivals. As a result it will produce better results and so on in—for it—a virtuous circle. School B, which has to put up with a less good intake, will have less good results. Inequality has increased.

In health, survey evidence suggests that, faced with a choice of hospitals, middle-class patients decide to go to the hospital which maximises their recovery chance; and working-class patients the local hospital. Unless choice drags up the standards of the worst, there is a risk of increasing health inequality.

There is an argument that choice will improve standards for everyone. The Social Market Foundation (SMF) think tank, which I chair, has published a study, "Choice: the evidence" by Jonathan Williams and Ann Rossiter, which assessed the impact of greater choice on bad schools. Having surveyed the available evidence, it concluded that the competition created led to improved standards in underperforming schools and even to an improvement relative to better schools. But I am sceptical about this evidence. Most of it came from the US and the researchers may have found what they wanted to find.

There are ways in which to improve the performance of poor schools, and competition between schools may have a role to play. But competition need not involve parental choice. Local authorities could promote it, by rewarding schools that perform well relative to the quality of their intake, and punishing those that perform badly. This is the concept of "earned autonomy" and is broadly speaking what is currently happening in education.

The sensible view on choice is that it is sometimes appropriate and sometimes not; that it has costs as well as benefits.

There are some public services where it is perfectly sensible to extend choice. One example is direct payments to older people so they can buy their own home care, rather than receiving local authority services. I became converted to this policy when I was a member of the royal commission on long-term care for the elderly (1998-99). I found that lots of money was being spent on sending in home helps to provide personal care to old people, but that the old people themselves tended to want things that the system did not provide, such as someone to tidy their garden. In Germany, old people were offered a choice: x euros to buy what they wanted or double that amount in local authority-provided services. To the astonishment of the authorities, people opted to an extraordinary extent for the option "worth" half as much. In Britain, the new social care minister, Liam Byrne, is pushing forward the direct payments agenda with vigour, and I applaud it.

Another benefit in health is choice of GP. At the moment, it is hard for a patient to quit one GP and then get accepted on another's list. Patients however are reasonably well informed as to what constitutes a good GP and what a bad one. The ability to strike up good relationships with patients is probably as important as technical knowhow.

Where the defects of choice are low (there are low costs of information), where the costs are relatively low (not too much spare capacity is required), and where the benefits demonstrably outweigh those costs, then choice may have an important role to play.

There is no general rule. There is no substitute for case-by-case assessments, based on an analysis of the costs and benefits of extending choice.

This requires a slower, more measured style of policymaking that combines general caution with a high premium on experimentation and on assessment of evidence. That at one point was meant to be a strength of Blair's government. More recently it has tended to be eclipsed by the prime minister's impatience, especially in public service reform.

There are signs that Gordon Brown appreciates some of the weaknesses of the philosophy of individual choice. But when he is prime minister, he will not lead a full scale counter-revolution, and I would not want him to. Choice has a role to play. It should however be a smaller role, in practice and in rhetoric, than today.