I recently attended a policy roundtable on the UK energy market. One of the subjects up for discussion was "consumer disengagement," which, according to the Competition and Markets Authority, is "impeding the proper functioning of [that] market." Too many customers, particularly those in the poorest 10 per cent of households, are on "default" energy tariffs and are unaware of the range of deals available. "We've tried to poke customers into becoming more active market players," one of the participants said, "but it's not enough."
I took this to be a disobliging allusion to the idea that governments can "nudge" (rather than poke) people into changing their behaviour without mandating or instructing them to do so. This is a notion popularised by the American social scientists Cass Sunstein and Richard Thaler in their 2008 book "Nudge". Sunstein and Thaler argued that the insights of the behavioural sciences could be used both to "help people to achieve their goals" and to improve the "efficiency and effectiveness of government." They called their philosophy "libertarian paternalism"—"libertarian" because it doesn't appeal to the coercive power of the state; "paternalism" because it asserts that the state nonetheless has an interest in behavioural change of certain kinds.
The book caught the eye of Steve Hilton, then an adviser to David Cameron. After Cameron became Prime Minister in 2010, Hilton helped to set up the Behavioural Insights Team (BIT) inside 10 Downing Street. The team was soon dubbed the "Nudge Unit" by the press. It was led by David Halpern, who'd previously worked in Number 10 under Tony Blair. Halpern had a staff of seven and a small office in Admiralty Arch. Today, the BIT has a staff of more than 60 and has been spun off as a "social purpose company" jointly owned by its employees, the government and Nesta, the charity.
In his book "Inside the Nudge Unit," Halpern tells the story of the BIT and explains how "behavioural insights are being applied to the 'nuts and bolts' activities of governments" around the world. I met him at the BIT's Westminster headquarters recently and began by asking him about how the organisation has evolved in the five years since it was set up.
DH: Given that the point of the approach was to spread understanding of [behavioural] approaches, you wanted them to be taken up. Changing the status [of the BIT] took the brake off, because it meant that it was much more straightforward for other public bodies to commission us. It meant that we could expand what we did in a number of areas and become more expert.
One of the weaknesses of British government, despite the many strengths, is that it’s subject to an extremely high turnover of people. And it breeds generalists. But if there are certain areas where you want someone to become really quite expert and to know what they’re talking about, both substantively and methodologically, you’ve got to hold on to people. The changed status [of the BIT] has enabled us to do that very effectively. Very few people have left the team. And they’ve been able to become really quite expert.
We’re able to carry on giving the same service to the Cabinet Office, the Cabinet Secretary, the PM and ministers. But other public bodies and indeed governments are able to commission us. And we’re able to use that to cross-subsidise our domestic policy work.
JD: You seem to suggest in the book that the coalition government was more hospitable to this kind of work than a majority government would have been.
It was the right time in terms of the world: you had a government faced [with] austerity, and which was also sceptical about the role of regulation and having a bigger state. So that created a very conducive environment for asking, “Well, what else?” If you take away those two major tools of government, what do you do? You also had support inside the administration, particularly from [then Cabinet Secretary] Gus O’Donnell, who’s a sophisticated economist well aware of the limitations of some of the models inside government, not least since 2008 and the financial crisis. Was there something about coalition per se? It probably helped a bit insofar as coalition government means you really have to discuss, debate and negotiate things. It creates an appetite for certain kinds of evidence, in a way you don’t necessarily need in another [kind of] administration.
The political context in which BIT was established was, as you say, one of austerity and cost-cutting. Did you then, and do you now, see the primary function of the BIT being to make government more cost-effective?
That was absolutely a major driver. And it’s certainly central to the “What works” movement. It’s not just, does something work? But also at what marginal cost? Governments have to make choices.
But it is clearly broader than that. Austerity has inevitably created a focus on cost efficacy. As well it should do. But there is a danger that governments get so obsessed with cost effectiveness that they neglect all the other dimensions of efficacy. All these other small differences, which aren’t about spending money, are phenomenally important to what you do.
Let’s consider the intellectual framework for this whole approach. In the preface, when you’re discussing the "realistic" picture of human behaviour that one finds in the behavioural sciences, which places considerable emphasis on the limits of human beings' cognitive capacities, you say that you dislike the application of the word “irrational” to it. Why?
That people use mental shortcuts means that under certain circumstances they are prone to error. But the corollary of that is that humans do incredible things, day in, day out, about which they think nothing. We have evolved to do certain things astonishingly. We don’t respect enough some of our capabilities and we don't see how extraordinary they are. We are superbly evolved creatures capable of doing some extraordinary things, but it turns out that in the modern world there are some things we’re not good at and we’re prone to error in.
The other problem with the word “irrational” is that it implies that someone else knows what the right answer is. And that’s not quite as clear cut as it sometimes seems. Richard Thaler has a similar view: he says we’re humans rather than “econs”. So to make a comparison with some Dr Spock-like character and say we don’t measure up… well, we do really measure up in lots of other ways.
So much for the word “irrational”. Let’s take the word “nudge”. How does a nudge differ from, say, an incentive?
In Cass Sunstein's work, you get the argument that a nudge is not an incentive because it’s not an overt economic signal and nor is it a mandation or a very strong sanction. This [is] a relatively narrow definition. There’s now a whole genre of papers in the academic world which go through a series of steps in order to define a nudge ever more narrowly and then say, “of course, therefore, nudges are much too narrow an instrument to be used in government.” You can understand why I find that frustrating! But despite the fact that book uses the word “nudge” in the title, actually the agenda is introducing a realistic model of human behaviour into what we do, including incentive-design.
You refer in the book to a course you used to teach at Cambridge in which you explored not just the way behavioural psychology might suggest alternative policy solutions, but also the way it might suggest alternative policy “aims,” as well.
If behavioural effects are playing in a particular area, it might be that we are mis-specifying where we want to go. If you read the Treasury Green Book, for example, it has a whole set of assumptions, quite deep assumptions, behind it around how you choose what is a better policy which start to look quite strange when viewed from a behavioural or psychological perspective.
So a lot of the work of the BIT has involved getting underneath those assumptions then?
Not directly. Most of the time you don’t have to do that. We’ve got a democratically elected government and it says, “You know what? We really need people to pay their taxes. It’s not fair if some people aren’t paying.” So you don’t have the answer deep philosophical questions to try and answer [the question], “Is there a way we can tweak and adjust the tax system to make sure that everyone pays their taxes on time?” You can answer many of these sub-questions without getting into the more profound ones.
In the book, you emphasise the role played early on in the life of the BIT by Steve Hilton. You have a nice description of him as a kind of Burkean conservative, who had, as you put it, “a deep-seated scepticism about government.” I’m interested in the extent to which that scepticism shaped the original design of the Behavioural Insights Team.
It clearly did. The early set-up was pretty overtly linked to the issue of de-regulation. In fact, some of the early sketches for the design of the Behavioural Insights Team were rooted in [the idea of] a de-regulatory unit. In the end, we slightly separated it from the “Red Tape Challenge,” a big de-regulatory exercise. We didn’t completely compound that with the Behavioural Insight Team, though a lot of our work related to it.
So a lot of the focus in the early days was on de-regulation, particularly lifting the regulatory burden on business. But at the same time you were doing a lot of work devoted to making life easier for consumers—giving them access to their data, enabling price-comparison and so on. Were those two strands of work ever in tension?
I think that, at a deep level, they are not in tension. But it’s a nuanced question that requires a nuanced answer. From a classical point of view, the de-regulation of a market—that is to say, the introduction of a number of market players—does your regulatory work for you. Competitive pressures will mean that a poor provider will be squeezed out of the market place and a better provider will grow accordingly. But as illustrated by the energy markets, it turns out that that’s not necessarily true—particularly given that consumers don’t have perfect information and are unable to model every possibility. What happens is that you end up with considerable “consumer detriment” that emerges in an apparently competitive market.
Now how does this bear on regulation? If you can make a market work better, it does a lot of the work that you’d want a regulator to do. A topical example of this, which the Competition and Markets Authority (CMA) is wrestling with, is the role of price comparison sites, or “choice engines” as they’re sometimes called. Why is this relevant? Imagine that there are myriad financial products on the market. Do I have to regulate all those products and players? If I’ve got price comparison sites which are doing this very effectively, they are in effect regulating the market for me, helping consumers to identify good and bad products.
QR codes on energy bills, which have recently come in, are a beautiful example of that. You could reduce the number of tariffs—that’d be a very hard regulatory [intervention]. But, on the other hand, if we can say to consumers, put your phone against that code and in two seconds it’ll give you the best tariff, it is doing the work for you. It doesn’t matter if there are 500 or 5,000 tariffs, because if that comparison is working you’ll find the better options. What you’re doing there is creating a self-regulating dynamic because the market is truly and deeply working, as opposed to working in a narrow, classical sense.
So is the underlying assumption there that when they are working well, self-regulating markets are always better than government regulation?
The general presumption is that if markets are functioning well, they do have a natural self-regulating bias. They are serving the consumer in a deep sense. And the consumer is effectively the one who’s driving the regulation of the market. If you think, and you know, “If this thing doesn’t work, I can return it in 14 days,” you are literally regulating the market. And you get very angry when someone breaks the rule—just as when someone drives on the wrong side of the road, you shout at them.
You’re interested in social norms and rules rather than explicit, coercive legislation, therefore?
We’re interested in a deep account of what makes societies tick. It’s almost like the invisible software of life. Take the long-term trend on crime. Manuel Eisner, whose work has been popularised by Steven Pinker, looked at 500 years of murder rates and found really dramatic falls occurring over time. A simple way of reading [his results] is to see [in them] the rise of social norms for acceptable behaviour. Instead of having an argument or a disagreement and then taking out your knife and stabbing someone, you learn different ways of resolving it—particularly, you see the emergence of what you might call politeness.
The emergence of social norms has done most of the heavy lifting in making societies function. Take the smoking ban in Britain. I think there has only been one prosecution in relation to the smoking ban. That is just amazing. People have clearly decided that that is an appropriate way to go on in the world. Our behaviour is riven by this software in our heads that guides us and tells what’s the appropriate thing to do or not.
Could we talk about the specific policy outcomes that have flowed from the work done by the BIT over the past five years? What would you say has been the most successful nudge?
I think most people would say auto-enrolment in pensions, because of its sheer scale. Its efficacy is off the scale. For me, the employment work [we’ve done] is [also] very interesting and important. The idea of encouraging people [looking for work] to plan ahead—When, where, how will you do this?—was partly born of work in other fields, including getting people to immunise. We didn’t know whether it would work, so it was really striking to see the impact it had. [It showed] that seemingly small details are really consequential. The way it was done before was backward-looking. It was essentially distrustful, it was saying, “We don’t trust you, you have to prove you’re looking for work,” as opposed to tweaking it, as we did, and saying, “What are you going to do next week?” That shifts people’s orientations and gets them to look in a different way. These are a class of interventions that have been systematically neglected in terms of their importance. But, boy, are they impactful.
You worked in Number Ten for several years during the Blair government. I’m interested in the relationship between Blair’s public service reform agenda and what you’ve done with the BIT, particularly in the area of customer and user choice. Do you think if the behavioural approach had been more in the mainstream back then, that Blair's reform agenda might have been more successful?
There are great continuities between the Blair administration and the Cameron administration in the ways they thought about reform. But you’re right, the New Labour reforms were quite deeply rooted in a classical model of choice and market operation. And this stuff [the behavioural approach] hadn’t really hit yet. It makes the world more complicated and it makes the way you think about reform more complicated.
We were prone, back then, to a naïve notion of markets. Our accounts [of markets] just weren’t good enough, as in the electricity market. [We said,] “let’s de-regulate the market, it will create choice and so on.” Our economics textbooks said this would increase utility. When things don’t work, it’s often because they were built on a classical model. In some areas, markets address and overcome our biases and function well. On the other hand, there are lots of areas where they don’t work. Some markets, under certain circumstances, seem to solve problems very well, and others don’t. This is true in relation to private sector markets, so you need to understand that as you move to bring market forces into public services. In other words, markets are sometimes incredibly efficient and sometimes they don’t work very well. If you believe in markets—whether you’re in the Blair administration circa 2005 or the Cameron administration in 2015—you need a nuanced account of that.
We’ve talked about successes. What about the failures?
That's a really important question. One of the points of the experimental approach is that you will have failures. You have to expect that. Arguably the biggest legacy of the BIT has been the incorporation of experimental methods into government, which brings in its wake a kind of humility, because you’re saying, essentially, “We don’t know the answer.” And that’s very, very hard for a political leader or a senior civil servant to say. To build a system which has that in it from top to bottom is an incredibly radical thing to do.
David Halpern's "Inside the Nudge Unit: How Small Changes Can Make a Big Difference" is published by WH Allen (£20)
I took this to be a disobliging allusion to the idea that governments can "nudge" (rather than poke) people into changing their behaviour without mandating or instructing them to do so. This is a notion popularised by the American social scientists Cass Sunstein and Richard Thaler in their 2008 book "Nudge". Sunstein and Thaler argued that the insights of the behavioural sciences could be used both to "help people to achieve their goals" and to improve the "efficiency and effectiveness of government." They called their philosophy "libertarian paternalism"—"libertarian" because it doesn't appeal to the coercive power of the state; "paternalism" because it asserts that the state nonetheless has an interest in behavioural change of certain kinds.
The book caught the eye of Steve Hilton, then an adviser to David Cameron. After Cameron became Prime Minister in 2010, Hilton helped to set up the Behavioural Insights Team (BIT) inside 10 Downing Street. The team was soon dubbed the "Nudge Unit" by the press. It was led by David Halpern, who'd previously worked in Number 10 under Tony Blair. Halpern had a staff of seven and a small office in Admiralty Arch. Today, the BIT has a staff of more than 60 and has been spun off as a "social purpose company" jointly owned by its employees, the government and Nesta, the charity.
In his book "Inside the Nudge Unit," Halpern tells the story of the BIT and explains how "behavioural insights are being applied to the 'nuts and bolts' activities of governments" around the world. I met him at the BIT's Westminster headquarters recently and began by asking him about how the organisation has evolved in the five years since it was set up.
DH: Given that the point of the approach was to spread understanding of [behavioural] approaches, you wanted them to be taken up. Changing the status [of the BIT] took the brake off, because it meant that it was much more straightforward for other public bodies to commission us. It meant that we could expand what we did in a number of areas and become more expert.
One of the weaknesses of British government, despite the many strengths, is that it’s subject to an extremely high turnover of people. And it breeds generalists. But if there are certain areas where you want someone to become really quite expert and to know what they’re talking about, both substantively and methodologically, you’ve got to hold on to people. The changed status [of the BIT] has enabled us to do that very effectively. Very few people have left the team. And they’ve been able to become really quite expert.
We’re able to carry on giving the same service to the Cabinet Office, the Cabinet Secretary, the PM and ministers. But other public bodies and indeed governments are able to commission us. And we’re able to use that to cross-subsidise our domestic policy work.
JD: You seem to suggest in the book that the coalition government was more hospitable to this kind of work than a majority government would have been.
It was the right time in terms of the world: you had a government faced [with] austerity, and which was also sceptical about the role of regulation and having a bigger state. So that created a very conducive environment for asking, “Well, what else?” If you take away those two major tools of government, what do you do? You also had support inside the administration, particularly from [then Cabinet Secretary] Gus O’Donnell, who’s a sophisticated economist well aware of the limitations of some of the models inside government, not least since 2008 and the financial crisis. Was there something about coalition per se? It probably helped a bit insofar as coalition government means you really have to discuss, debate and negotiate things. It creates an appetite for certain kinds of evidence, in a way you don’t necessarily need in another [kind of] administration.
The political context in which BIT was established was, as you say, one of austerity and cost-cutting. Did you then, and do you now, see the primary function of the BIT being to make government more cost-effective?
That was absolutely a major driver. And it’s certainly central to the “What works” movement. It’s not just, does something work? But also at what marginal cost? Governments have to make choices.
But it is clearly broader than that. Austerity has inevitably created a focus on cost efficacy. As well it should do. But there is a danger that governments get so obsessed with cost effectiveness that they neglect all the other dimensions of efficacy. All these other small differences, which aren’t about spending money, are phenomenally important to what you do.
Let’s consider the intellectual framework for this whole approach. In the preface, when you’re discussing the "realistic" picture of human behaviour that one finds in the behavioural sciences, which places considerable emphasis on the limits of human beings' cognitive capacities, you say that you dislike the application of the word “irrational” to it. Why?
That people use mental shortcuts means that under certain circumstances they are prone to error. But the corollary of that is that humans do incredible things, day in, day out, about which they think nothing. We have evolved to do certain things astonishingly. We don’t respect enough some of our capabilities and we don't see how extraordinary they are. We are superbly evolved creatures capable of doing some extraordinary things, but it turns out that in the modern world there are some things we’re not good at and we’re prone to error in.
The other problem with the word “irrational” is that it implies that someone else knows what the right answer is. And that’s not quite as clear cut as it sometimes seems. Richard Thaler has a similar view: he says we’re humans rather than “econs”. So to make a comparison with some Dr Spock-like character and say we don’t measure up… well, we do really measure up in lots of other ways.
So much for the word “irrational”. Let’s take the word “nudge”. How does a nudge differ from, say, an incentive?
In Cass Sunstein's work, you get the argument that a nudge is not an incentive because it’s not an overt economic signal and nor is it a mandation or a very strong sanction. This [is] a relatively narrow definition. There’s now a whole genre of papers in the academic world which go through a series of steps in order to define a nudge ever more narrowly and then say, “of course, therefore, nudges are much too narrow an instrument to be used in government.” You can understand why I find that frustrating! But despite the fact that book uses the word “nudge” in the title, actually the agenda is introducing a realistic model of human behaviour into what we do, including incentive-design.
You refer in the book to a course you used to teach at Cambridge in which you explored not just the way behavioural psychology might suggest alternative policy solutions, but also the way it might suggest alternative policy “aims,” as well.
If behavioural effects are playing in a particular area, it might be that we are mis-specifying where we want to go. If you read the Treasury Green Book, for example, it has a whole set of assumptions, quite deep assumptions, behind it around how you choose what is a better policy which start to look quite strange when viewed from a behavioural or psychological perspective.
So a lot of the work of the BIT has involved getting underneath those assumptions then?
Not directly. Most of the time you don’t have to do that. We’ve got a democratically elected government and it says, “You know what? We really need people to pay their taxes. It’s not fair if some people aren’t paying.” So you don’t have the answer deep philosophical questions to try and answer [the question], “Is there a way we can tweak and adjust the tax system to make sure that everyone pays their taxes on time?” You can answer many of these sub-questions without getting into the more profound ones.
In the book, you emphasise the role played early on in the life of the BIT by Steve Hilton. You have a nice description of him as a kind of Burkean conservative, who had, as you put it, “a deep-seated scepticism about government.” I’m interested in the extent to which that scepticism shaped the original design of the Behavioural Insights Team.
It clearly did. The early set-up was pretty overtly linked to the issue of de-regulation. In fact, some of the early sketches for the design of the Behavioural Insights Team were rooted in [the idea of] a de-regulatory unit. In the end, we slightly separated it from the “Red Tape Challenge,” a big de-regulatory exercise. We didn’t completely compound that with the Behavioural Insight Team, though a lot of our work related to it.
So a lot of the focus in the early days was on de-regulation, particularly lifting the regulatory burden on business. But at the same time you were doing a lot of work devoted to making life easier for consumers—giving them access to their data, enabling price-comparison and so on. Were those two strands of work ever in tension?
I think that, at a deep level, they are not in tension. But it’s a nuanced question that requires a nuanced answer. From a classical point of view, the de-regulation of a market—that is to say, the introduction of a number of market players—does your regulatory work for you. Competitive pressures will mean that a poor provider will be squeezed out of the market place and a better provider will grow accordingly. But as illustrated by the energy markets, it turns out that that’s not necessarily true—particularly given that consumers don’t have perfect information and are unable to model every possibility. What happens is that you end up with considerable “consumer detriment” that emerges in an apparently competitive market.
Now how does this bear on regulation? If you can make a market work better, it does a lot of the work that you’d want a regulator to do. A topical example of this, which the Competition and Markets Authority (CMA) is wrestling with, is the role of price comparison sites, or “choice engines” as they’re sometimes called. Why is this relevant? Imagine that there are myriad financial products on the market. Do I have to regulate all those products and players? If I’ve got price comparison sites which are doing this very effectively, they are in effect regulating the market for me, helping consumers to identify good and bad products.
QR codes on energy bills, which have recently come in, are a beautiful example of that. You could reduce the number of tariffs—that’d be a very hard regulatory [intervention]. But, on the other hand, if we can say to consumers, put your phone against that code and in two seconds it’ll give you the best tariff, it is doing the work for you. It doesn’t matter if there are 500 or 5,000 tariffs, because if that comparison is working you’ll find the better options. What you’re doing there is creating a self-regulating dynamic because the market is truly and deeply working, as opposed to working in a narrow, classical sense.
So is the underlying assumption there that when they are working well, self-regulating markets are always better than government regulation?
The general presumption is that if markets are functioning well, they do have a natural self-regulating bias. They are serving the consumer in a deep sense. And the consumer is effectively the one who’s driving the regulation of the market. If you think, and you know, “If this thing doesn’t work, I can return it in 14 days,” you are literally regulating the market. And you get very angry when someone breaks the rule—just as when someone drives on the wrong side of the road, you shout at them.
You’re interested in social norms and rules rather than explicit, coercive legislation, therefore?
We’re interested in a deep account of what makes societies tick. It’s almost like the invisible software of life. Take the long-term trend on crime. Manuel Eisner, whose work has been popularised by Steven Pinker, looked at 500 years of murder rates and found really dramatic falls occurring over time. A simple way of reading [his results] is to see [in them] the rise of social norms for acceptable behaviour. Instead of having an argument or a disagreement and then taking out your knife and stabbing someone, you learn different ways of resolving it—particularly, you see the emergence of what you might call politeness.
The emergence of social norms has done most of the heavy lifting in making societies function. Take the smoking ban in Britain. I think there has only been one prosecution in relation to the smoking ban. That is just amazing. People have clearly decided that that is an appropriate way to go on in the world. Our behaviour is riven by this software in our heads that guides us and tells what’s the appropriate thing to do or not.
Could we talk about the specific policy outcomes that have flowed from the work done by the BIT over the past five years? What would you say has been the most successful nudge?
I think most people would say auto-enrolment in pensions, because of its sheer scale. Its efficacy is off the scale. For me, the employment work [we’ve done] is [also] very interesting and important. The idea of encouraging people [looking for work] to plan ahead—When, where, how will you do this?—was partly born of work in other fields, including getting people to immunise. We didn’t know whether it would work, so it was really striking to see the impact it had. [It showed] that seemingly small details are really consequential. The way it was done before was backward-looking. It was essentially distrustful, it was saying, “We don’t trust you, you have to prove you’re looking for work,” as opposed to tweaking it, as we did, and saying, “What are you going to do next week?” That shifts people’s orientations and gets them to look in a different way. These are a class of interventions that have been systematically neglected in terms of their importance. But, boy, are they impactful.
You worked in Number Ten for several years during the Blair government. I’m interested in the relationship between Blair’s public service reform agenda and what you’ve done with the BIT, particularly in the area of customer and user choice. Do you think if the behavioural approach had been more in the mainstream back then, that Blair's reform agenda might have been more successful?
There are great continuities between the Blair administration and the Cameron administration in the ways they thought about reform. But you’re right, the New Labour reforms were quite deeply rooted in a classical model of choice and market operation. And this stuff [the behavioural approach] hadn’t really hit yet. It makes the world more complicated and it makes the way you think about reform more complicated.
We were prone, back then, to a naïve notion of markets. Our accounts [of markets] just weren’t good enough, as in the electricity market. [We said,] “let’s de-regulate the market, it will create choice and so on.” Our economics textbooks said this would increase utility. When things don’t work, it’s often because they were built on a classical model. In some areas, markets address and overcome our biases and function well. On the other hand, there are lots of areas where they don’t work. Some markets, under certain circumstances, seem to solve problems very well, and others don’t. This is true in relation to private sector markets, so you need to understand that as you move to bring market forces into public services. In other words, markets are sometimes incredibly efficient and sometimes they don’t work very well. If you believe in markets—whether you’re in the Blair administration circa 2005 or the Cameron administration in 2015—you need a nuanced account of that.
We’ve talked about successes. What about the failures?
That's a really important question. One of the points of the experimental approach is that you will have failures. You have to expect that. Arguably the biggest legacy of the BIT has been the incorporation of experimental methods into government, which brings in its wake a kind of humility, because you’re saying, essentially, “We don’t know the answer.” And that’s very, very hard for a political leader or a senior civil servant to say. To build a system which has that in it from top to bottom is an incredibly radical thing to do.
David Halpern's "Inside the Nudge Unit: How Small Changes Can Make a Big Difference" is published by WH Allen (£20)
