On 1st March 2007, in a nondescript Birmingham committee room, a panel of consultant surgeons decided to break their employment contracts. In doing so, they guaranteed the front-page storm that government officials were hoping to avoid. Two weeks later, thousands of junior doctors noisily demonstrated in London demanding jobs and training. Patricia Hewitt, secretary of state for health, apologised to the doctors, was "supported" by her party—and then duly dumped that summer. So began one of the most profound and emblematic crises the NHS has suffered in its 60-year history.
What did the Birmingham surgeons do? They simply refused to continue interviewing applicants who had been shortlisted for junior doctor appointments in their area. The system for shortlisting and appointing was, in their view, fundamentally unfair. The winter of 2006-07 had seen the introduction of a new centralised, website-based approach to the appointment of junior doctors to posts and to their training. All over the country, there was growing panic and anger among doctors at the results—both the nature of the new system and the enormous gulf between the 32,000 doctors applying and the 15,000 or so training places available.
A year later, another round of appointments is in full swing, and all seems quiet. So was last year just another one-off Terminal 5-style fiasco, in which new systems crashed and inexperienced staff made mistakes before things settled down? Sadly not. What happened was the result of a gathering crisis in medical education, born of chaotic policies for recruitment and training and made worse by a naive faith in central planning and the vagaries of immigration policy. Put simply, far too many junior doctors—British and foreign—are entering the system for the training opportunities and senior jobs on offer, and this problem will remain with us for years to come.
The young doctors who marched through the streets last spring are among the best and the brightest of their generation. Medical school entrants are consistently the most academically talented group of undergraduates, and doctors are always at the top of "most trusted occupation" surveys. Their training is long and expensive, involving a five-year undergraduate degree, at a cost of £250,000; a short, highly supervised period just after graduation, after which they are "registered" and can, for example, write prescriptions; and then a period of at least six years apprenticeship-style training and study. These postgraduate "junior doctors" provide a great deal of the actual service patients receive. (Think Meredith in Grey's Anatomy.) Junior doctors are usually the first point of contact in an NHS hospital; using traditional British terminology, they would work their way up from junior to senior house officer (SHO), and then registrar, en route, with luck, to consultant. Similarly, GPs have a post-qualification period of combined training and practice. The same basic pattern is found throughout the world, whether in France or the Netherlands, with their state-regulated insurance-based provision; Canada, which shares our "free at the point of use" approach; or the US, which combines its fragmented and much-criticised funding system with a highly organised approach to medical training. No modern service can function without junior doctors.
Labour swept to power in 1997 in part because the British electorate were tired of underfunded public services. Few people today, including the Conservative front bench, doubt that, irrespective of the NHS's efficiency, we were spending less on healthcare than was reasonable, given an ageing population, expensive medical advances and growing demand. The country was near the bottom of OECD tables for share of GDP spent on health, and lowest in Europe for doctors per head.
New Labour was therefore primed to approve the recommendations of its own medical workforce standing advisory committee. In 1997, this recommended an increase in medical student numbers from 4,500 a year to 5,500. The report went to the secretary of state for health, Alan Milburn. Milburn agreed that more doctors were a good idea. But why stop at 1,000? He promptly doubled the number. Why? We have no idea. There is no public record I can find explaining the grounds for this decision, which was ensconced in the NHS plan of 2000. No one in the professional and regulatory bodies at the time was consulted officially; their successors shrug their shoulders and mutter about political gestures.
The increased numbers therefore rolled through: not 1,000 but 2,000 extra, and not just that year, but every year thereafter. Funding was made available for 2,145 new medical places in English universities, and for four new medical schools. The first new cohorts were recruited and their long undergraduate training began. The secretary of state changed, not just once but three times before winter 2006. And the department of health, it seems, simply forgot that a step change in junior numbers was on its way.
***
At just this moment, the government decided to change the system for training and hiring junior doctors. And to do so not just for new graduates, but also for the junior doctors a few years ahead of them. Until 2006, junior doctors applied for specific jobs in specific hospitals at whatever point in the year vacancies came up—just as teachers do in schools. They would build up their skills and get them recorded as they went along. They also took, and take, exams set by the medical Royal Colleges.
The big bang of 2006 standardised training. It also centralised appointments. Everyone had to apply for junior training posts at the same time using a national website. The number of posts was centrally determined, and highly limited. If you got a post, it was long term, offering guaranteed "run-through training" for years, and putting you on track for a consultant's job. But if you didn't, you would be a discard, stuck in low-level jobs with no training or prospects. So a large number of young doctors faced professional meltdown at the very beginning of their careers.
As the process unfolded, juniors were at first simply confused. Behind the scenes, however, the senior medical organisations were starting to panic. They knew the numbers did not add up. Alan Crockard, an internationally renowned neurosurgeon, was head of "Modernising Medical Careers," the department of health (DH) initiative to reform postgraduate training. By autumn 2006, he had become aware that there were simply not enough places. "I said, 'We've got problems.' The department was saying everything was fine, but none of them had first-hand experience of medical education and training. It wasn't just Patricia Hewitt who was out of her depth."
The Crockard team was advised to ensure 10,000 training posts were available. They went for, and organised, 15,000. More than twice that number applied. Soon, any doctor you talked to had horror stories to tell. Gold-medal winners were failing to get training jobs; at Imperial College, not one of the junior doctors in a top research group was offered a single job interview. Other juniors, some with small children, were being offered a single take-it-or-leave-it job many hours' commute away.
At the root of the chaos was a complete miscalculation of the numbers of training posts that would be required. But other decisions contributed. In classic British government style, the DH opted for yet another untried IT system, then imposed a big change in the nature of the project when it was well under way. Every junior doctor's application had to be filed online using this application system, which was supposed to make everything more objective and "fair." To this end, the DH also completely re-engineered selection. CVs, and information about applicants' academic records and clinical attainments, were systematically excluded from the application process. Instead, everything depended on how people replied to general questions. These were scored by NHS officials, most of them not medically qualified, and used to draw up shortlists for interview by medical panels—such as the Birmingham surgeons who blew the process apart.
I saw these forms because, like many families, ours has a bemused young doctor caught up in the process. I can only concur with the House of Commons health select committee, which this May concluded that "the shortlisting process… descended into little more than a creative writing exercise." Junior doctors, with almost no work experience, had to "describe a situation when applying your clinical judgement had a significant impact on patient health." As doctor and MP Richard Taylor complained: "I would have been terribly pushed to have answered that question after being qualified for five years or more."
As the political storm gathered, the NHS bureaucracy made desperate attempts to create new training jobs. The junior doctors' protests spawned Remedy, a new lobby group; James Johnson, head of the doctors' trade union, the British Medical Association (BMA), was forced out after backing the DH's actions. In March 2007, Alan Crockard resigned, followed by his deputy: he had, he said, been left with responsibility but no authority. Neither the chief medical officer, Liam Donaldson, nor any officials or ministers felt obliged, then or later, to follow.
Fury over the selection criteria was compounded by failures in the online application system itself. The health select committee was told that in the West Midlands, the body responsible for allocating applicants across a region of 5.3m people had over 1,300 applications missing from the system on the day before the closing date. There were two major security breaches. And the algorithm that was supposed to match jobs to preferences failed. Finally, during a judicial review instigated by Remedy, the DH admitted that the system was not merely broke but unfixable. The facts on the ground were, in any case, intransigent. After the first round of appointments, with 90 per cent of training places gone, the DH's own figures showed that 28 per cent of British medical school graduates, and a quarter of British medical school graduates who were British nationals—2,975 doctors in all—had failed to obtain a place.
***
So now what? Even if the number of undergraduates entering medical schools this autumn were halved, we could still, on any plausible set of assumptions about drop-out rates, part-time work and so on, expect consultant numbers to almost double by the year 2021, before levelling off. Maintain current intakes, as we show every sign of doing, and there will be—according to Roy Pounder of the Royal College of Physicians—80,000 consultant-level doctors in the 2020s, compared to 30,000 today. And this assumes a freeze on immigration.
One can, if one tries, be upbeat about this. The government has, after all, embraced the idea of a "consultant-delivered service," in which we are increasingly treated by experts, not juniors. But the NHS is not a giant graduate school, it is a health service (and a cash-limited one). Specialists acquire their skills in the workplace, not in classrooms. Richard Marks is a consultant anaesthetist organising specialist training in north London (and a leading member of Remedy). At the higher levels, he emphasises, training capacity is limited by real life in hospitals: "Take my field: you have to do paediatric anaesthesia to be a consultant." That means if no small children turn up in the operating theatre, you simply can't get the necessary skills.
Does the DH, or its ministers, have any clear idea how ever-increasing numbers of doctors are to be given specialist training? Not obviously. Indeed, the history of the 2006-07 junior doctors debacle suggests a complete failure to recognise, let alone provide for, the young doctors entering practice. This failure is itself possible because of the peculiar nature of our medical system, and the way it organises education. The NHS is unique among rich countries in the degree to which health provision is centrally managed. This includes deciding, from London, what drugs can be prescribed, what patients can or must be charged for, and, in England at least, lists of compulsory performance targets. Since the 1990s, the NHS has also exerted an unusual degree of central control over junior doctors' careers; something increased by the recent controversial reforms.
In most of the world, universities have direct responsibilities for both undergraduate and postgraduate medical training, in collaboration with hospitals. Take, for example, France, generally seen as the home of centralism. Vincent Darrouzet, the postgraduate medical dean of the University of Bordeaux, explained the system to me. All those who successfully graduate from undergraduate medical studies proceed, automatically, into higher studies as an intern. There are, at national level, somewhat more internships than there are interns emerging from the graduating undergraduate class; and there is competition for the more popular specialties and the more popular towns. Decisions are based on academic performance using a Ucas-type selection system; but once a young doctor reaches his or her destination, the whole process of postgraduate work and training is decentralised. As interns they work, receiving a state stipend, and also study, and their university arranges it all, in collaboration with its network of hospitals.
At first sight, our system seems similar. Universities offer undergraduate medical programmes lasting, typically, five years. Teaching hospitals are linked to the universities and involved in undergraduate training. But at postgraduate level, doctors' professional training becomes the responsibility not of the universities but of the "deaneries." These sound academic, but they are not. Today's postgraduate deaneries are part of the NHS proper, and since the mid-1990s, the people who work in them have been, in effect, civil servants.
This reflects the all-embracing nature of the NHS. It also means that junior doctors are classified legally as employees, not postgraduate interns. For the managers running the hospitals where juniors work, and learn, what matters is the things for which they might get promoted or demoted—namely the main performance targets. What matters about junior doctors is how far they help achieve these targets now, not in a few years time.
At present, deaneries are funded through strategic health authorities (SHA). Money supposed to fund training goes from the department to the SHAs to the deaneries and on to hospitals. Yet the ever-shifting kaleidoscope of NHS organisation means deaneries and SHAs are rarely coterminous. The DH cannot even tell you how many people deaneries employ.
Prior to the 2006-07 reform, hospitals hired their own senior house officers. They often did so on very short-term contracts, reflecting the fact that managers (and finance directors) were primarily concerned with service provision, not training. The process of planning the training was nonetheless carried out with great seriousness and precision, as it still is. The bureaucracy decided how many specialists of each type need to be trained in each area of the country, and these carefully estimated numbers were translated into notional training places.
On paper, the pre-reform system was neat and tidy: medical needs were planned, training places allocated to hospitals and doctors hired to fill them. In practice, in a fashion familiar to any student of central planning, there was an official system, and then there was the real system on the ground. Real doctors were appointed to real jobs in line with local demands rather than planned "needs." Money for training was incorporated into hospital budgets rather than tied to individual trainees and was often completely untraceable. The clinical tutor at a large London hospital trust explained how this worked: "Suppose it was cardiology. You might have five SHOs who were officially in training, because the deanery said you could have five training places, but actually you need three more than that to fill your rotas. So we would select eight people, and it would all be blurred. The deanery didn't know who was in its five approved places, and a lot of the time no one in the hospital did either." So eight people were hired, worked and got more or less systematic training, while on paper five training places were recorded.
These parallel universes became established for good reasons. In recent years hospitals have been in need of far more non-consultant doctors than the training plans allowed for. So they had to appoint doctors to posts with no training attached—known as "trust doctor" positions. Also many young doctors wanted posts which were not designated as training posts. They might want to work on a research project. They might want to work in a particular place because of family commitments. (With more and more female doctors, and more doctors with career spouses, part-time opportunities and geographical location are increasingly important issues.)
So pre-2006, the "unofficial" system was working, more or less. No one, least of all the junior doctors, thought it perfect. It allowed hospitals to provide staff for service delivery, and offered real flexibility to junior doctors with differing interests and needs. It gave more junior doctors access to training than the official numbers allowed. But appointments were often very short term, and training hit-and-miss.
Liam Donaldson, the chief medical officer, was a convinced reformer. His proposals involved creating explicit training curriculums for all the different specialties. This idea was something which everyone agreed was long overdue, and it is, to date, the one positive outcome of the whole reform saga. But he and the DH also decided to "rationalise" and centralise the whole junior medical labour market. Senior civil servants asserted that there were frequent examples of appointments made on the basis of personal links and old-boy networks, although no evidence on the frequency or severity of such cases has ever been provided. The remedy that they imposed was to deprive hospitals of any control over appointments, and ensure that all training posts were both clearly delineated and long-term. Donaldson's and the DH's insouciance in the period before the reforms went live suggests that they really did not understand the system they were proposing to take over. If they had understood the gap between the official training system and the reality of an expanding and complex junior workforce, could they really have been happy with the number of training places they propose?
The 2006-07 reforms meant that applicants could specify which region they wanted to work in, but not where in the region. Instead, all junior doctors in training would be selected by NHS deaneries, and directed to the workplace to which they had been assigned. This would be equivalent to, for example, assigning all young teachers once a year, through the department for children, schools and families; or making all appointments to police sergeant from the home office. It is full-blown central labour planning.
Another government policy was also sitting unnoticed in the wings, ready to increase the chaos. The earlier centralisation which had removed postgraduate training from the universities, and lodged it in the NHS deaneries meant that junior doctors were normal employees—not postgraduate interns as in most other countries—and subject to general workforce legislation. This meant applications could be made by all doctors with the legal right to come to Britain, as well as graduates of EU universities. Together with a more or less open door policy on medical immigration this meant that large numbers of overseas-trained doctors could compete with Britain's own burgeoning graduate numbers for the limited supply of training places.
***
As anyone who has been near a hospital or surgery knows, this country employs huge numbers of medical staff who trained overseas. Among doctors, this reflects a number of different factors and decisions. First, spending on and demand for healthcare has, up to now, increased at a faster rate than our domestic medical schools could keep up with. Second, Britain offered "permit-free" immigration to doctors. They could arrive, sit a General Medical Council examination, and, if they passed, be placed on the register and look for a job in the NHS. And come they did, in their thousands, with total entrants for the exam rising from around 4,000 a year in the mid-1990s to over 15,000 at the turn of the millennium. Many of these doctors already had considerable experience, and as such were highly welcome to hospitals and trusts whose rationale is service provision (and target-hitting). Third, in 2000, the NHS plan not only announced a big medical school expansion but also committed itself to an immediate increase of 8,000 doctors. There was nowhere they could come from other than abroad, which they duly did, recruited from a government recruitment drive, largely in the Indian subcontinent. Thus when the changes to training were introduced, it was not just recent graduates of British universities applying, but thousands of others as well.
NHS power politics figures here too. Some people believe that last year's events were inspired by a desire to reduce doctors' bargaining power. Such a conscious strategy seems unlikely, and the claim is certainly unproveable. But there are many people within the DH, and elsewhere in government, who have a deep-seated dislike of professions generally, and the medical profession in particular. A good number were perfectly happy with the prospect that British graduates would have neither postgraduate training posts nor, sometimes, any jobs at all, and that many of these would instead go to overseas-trained applicants. One senior member of the DH staff referred to young doctors as "sunk capital." They could therefore be written off if better-trained and cheaper doctors came along. Others simply felt it wrong that every trainee doctor in the world should not get a chance to compete.
As the crisis became politically embarrassing, and the numbers without jobs more evident, the DH started desperately to look for ways to give British graduates preference in the appointments process. But they could not overturn employment law; in April, the House of Lords confirmed that foreign doctors already in the country must have equal access to training posts. And a number of people have told me, off the record, that, until very recently, the treasury could be relied on to block any serious attempts to stem medical immigration. For many people, what was happening was simply an unfortunate side-effect of a sound policy, which was also a good way of cutting doctors down to size. It was only this year, with another application round under way, that the government finally moved to exclude doctors from the highly skilled migrants programme, which had offered them open-ended permit-free entry. Most taxpayers, if asked, would probably think it crazy to pour money into new medical schools, recruit young people for arduous training with the expectation of secure employment, and at the same time continue with an unrestricted immigration system that drains poorer countries of their doctors. That, nonetheless, has been our policy.
***
So where, now, is our system of medical education and training? In many respects, extraordinarily close to where it was in 2006, give or take a few well-thought out curriculums and a lot of personal misery. An official enquiry by John Tooke made excellent recommendations for simpler structures. But large numbers of junior doctors are on short-term contracts, just as they always were. Official training posts have been redefined, so they are shorter than those offered last year; and something called a fixed term specialty training appointment has slipped on to the scene—a sort-of-training post for those who haven't got a proper one. Junior doctors are still the mainstay of hospital service, they are still primarily employees rather than postgraduate trainees, and there are still ever-growing numbers chasing a limited number of specialist opportunities. Moreover, deaneries have retained their power to decide where, in their region, to offer a junior a job. There has been no retreat on this basic principle, reflecting the DH's tendency to centralise as its preferred, and default option.
At 60, the NHS is truly unique, not simply in its size and workforce planning ambitions, but in its basic role as a national, monopoly employer. Consultants may be able to operate a private practice, but for young doctors there is effectively no choice; they have to work for the NHS. In the past, it offered them the classic public-sector bargain: top-down planning, modest pay, standardised jobs with common national conditions, but also job security. This included the implicit promise to all medical students that, while they might not get the speciality they wanted, or their first choice of town, they would practise medicine in a secure environment. It has been on that basis that the NHS has recruited, year after year, many of the brightest young people.
That promise is now, effectively, broken. Last year's events made this clear. Some of what happened reflected health officials' inability to understand their own system, and faith in untried IT. But the underlying problem is that expanding medical numbers cannot be fitted into existing, centrally set job structures. In the past, the system has muddled through, lengthening the periods juniors spend waiting for promotion, and running real and official systems in parallel. As the junior doctor numbers grow, this will become less and less viable.
It seems obvious that doctors' jobs and career structures need to change. But the NHS is dismally ill-suited to achieving this. The desirability of creating more specialist, non-consultant posts has been recognised for years, and nothing has happened. This reflects the internal structure of the service, combining complexity, overload and rapid turnover among its political masters. It is also because, as a huge public sector employer, the NHS has inevitably encouraged the growth of powerful national unions, negotiating partners who can deliver their members. Most powerful of all is the BMA, whose own power is symbiotically tied to the continued existence of a single monolithic paymaster. Its preferred solutions are simple—more and more money, more and more highly paid consultants—and it has set its face against any sub-consultant grade.
Can the NHS nonetheless develop an effective way of training and deploying its growing medical workforce? It is hard to believe it can without a fundamental decentralisation which allows recruitment to respond to local circumstances. Without that, the underlying problems can only worsen. And if I were 17 or 18, I would think very hard before embarking on medicine in Britain as a career.
