In future, pricing will be personalised to our shopping habits. Photo: PA

The future of shopping: pricing gets personal

Big Data means retailers can figure out exactly what you’re willing to pay—and they’re starting to charge it
December 8, 2017

Imagine strolling with a neighbour to the corner shop, only to be charged 50p more than her for buying the same box of cornflakes. You’d better get used to that feeling of unfairness because “price discrimination” could be a big part of the future of retail. In fact, it’s already here. On some websites, online prices now bounce up and down in ways that are specific to you—depending on what you’ve bought before, what you paid for it, and even what you’ve previously browsed and not bought. Big data, the cascade of digital traces we all leave in the networked age, allows firms to gather information on customers, and they can use that information to figure out, with unprecedented accuracy, how much we’re each willing to pay—and, therefore, the maximum they can charge.

Put like this, price discrimination sounds unappealing, and there’s no doubt it offends the expectations we’ve developed in the mass consumer markets of the last 70 years. We’ve grown used to prices not varying for the simple reason that companies haven’t known the maximum their customers were willing to pay—their so-called “reserve price.”

On top of this, even if companies had hazarded a guess at which customers they could charge more, they would have found it impractical to do so. Imagine changing your price tags, or reprinting your menus, for each new customer that walked in the door. Even if you could, the complexity involved wouldn’t justify the cost. Businesses have therefore, in general, had to play it straight, charging all their customers the same competitive price.

This isn’t to say prices have been static or simple. We’ve always known that prices can be pushed up—by surging demand or scarce supply. But, at least when they rose, they rose for all. There was an intuitive fairness to the system. With personalised pricing, by contrast, the individual takes his or her chances—depending on the unknowable things a supplier knows about them. Put like that, this emerging future doesn’t feel so right.

And yet price discrimination is neither entirely new, nor all bad news. Think of the young person’s railcard: it gives one group lower fares on the assumption that they can typically afford to pay less than the middle aged. Some chip shops offer “pensioner specials” on the same basis; and coffee shops in Belgravia charge more than cafés in Bootle—not just because the rent is higher, but also because they have a hunch that the locals will be willing to pay more. Prices also vary on an individual basis in sectors like second-hand cars, where haggling survives. And, in the right circumstances, economic theory suggests that price discrimination of this kind can make markets more efficient. By better matching demand to supply, so that more value-adding trades take place, it can boost output and—in aggregate—consumer outcomes.

So how big a change are we in for? And how worried should we be? Well, tech-savvy companies can already use data analytics to estimate the reserve price of individual shoppers. Furthermore, they can act on that information, using cookies left on a laptop or phone, to present different prices when different people visit their online stores. And, because data sources like a customer’s browsing history, are far better predictors of price sensitivity than yesterday’s simple demographic data, the discrimination itself gets far more precise, and so far more profitable.

Prices can also be varied to reflect the cost of serving individual customers. Big data makes it easier for firms to identify high-cost customers, such as people who are less likely to pay back their debts, and charge them more. You may not even know why you’re paying more than someone else. And that raises the prospect of prices becoming linked to proxies of personal characteristics like age, gender, or even ethnicity. Thus, discrimination re-enters business by the backdoor, with only the algorithm to blame.

*** To grasp the full implications of big data, however, you have to look at price discrimination alongside a second trend: the incredible power of behavioural insights. Companies can now nudge consumers, in ever more subversive ways, into parting with their money. Retailers are, of course, old masters at these knowing winks: think of the sweets by the till to encourage impulse purchases. What’s new is the unprecedented sophistication that big data allows. Retailers can now use us as guinea pigs in live behavioural experiments, making small variations to their offer, and seeing how our choices change.

Combine this nudging with personalised pricing and things can get pernicious. One example is the growing penalties for consumer inertia. In utilities and banking, these two tricks are combined in a “bait and squeeze” attack on the customer. Firms compete fiercely with an acquisition price, tempting you to sign a contract; this contract is then designed to maximise the chance you’ll auto-renew by mistake; and then the company hikes your price because you’re disengaged, charging you more for the same thing.

In Britain’s broadband market, for example, prices jump 43 per cent on average at the end of a fixed-term deal. With mobile phones, the same trick is played by continuing to charge customers for a handset even after it’s paid off. But it’s in energy that “bait and squeeze” creates the most resentment. It’s a service you can’t avoid buying, and so we rightly feel outrage when disengaged vulnerable people end up paying the most.

Put to one side the specifics, and there’s something more fundamental at play here: namely, regret. We all know the feeling: a free trial you keep forgetting to cancel, or exorbitant insurance you don’t even remember renewing. These experiences nag at your mind. More than this, they subvert the very idea of consumer choice—and, thereby, the market itself.

If a market functions well, with little regret, then the fact that you bought something in itself reveals it was worth buying. That premise has long been relied on to show that markets lead, a priori, to better outcomes. When behavioural tricks come into play, however, that argument goes weak at the knees. Does the fact that you’re paying for a monthly subscription, after a free trial ended, mean you want the subscription? What if you haven’t switched because the process was made deliberately burdensome? Are you choosing these outcomes? If not, then it doesn’t matter if the market is ostensibly competitive; the trade is making you worse, not better, off. In this way, nudges and personalised pricing can gnaw at the very basis of the traditional case for the free market.

This all helps to explain why, as the dynamics of consumer markets become increasingly unfamiliar, this is feeding a diffuse sense of unease about how capitalism serves ordinary people. More specifically, these trends also pose urgent challenges to regulators and economists alike.

So how should public policy respond? As ever, it is as well not to get carried away. Yes, personalised pricing and nudges are happening in some sectors, such as online retail and supermarkets, but they’re years away in others. And let’s not lose sight of the huge opportunities for consumers that come with the information age: the time you save when you’re offered things you want to buy, rather than things you don’t; the power that shoppers wrestle back by sharing notes on rogue retailers; the lower average prices these practices could allow.

It’s also important not to see this as an anti-business story. These trends don’t benefit all producers any more than they disadvantage all consumers. Entrepreneurs suffer from over-powerful companies just as much as shoppers; think of the start-ups trying to dislodge Amazon from its perch. Think, too, of the small suppliers—say, dairy farmers—engaged in lopsided negotiations with their giant, data-rich customers, the big supermarkets.

Nonetheless, a few things still seem clear. Big data will only become more ubiquitous; computational power and speed will keep soaring; retail will continue to move online and out of physical stores. All of this means that tomorrow’s consumer economy will look a lot like today’s tech sector, and regulatory policy simply isn’t ready for that. It needs, as Martin Wolf has argued, to get an “intellectual grip” on what is happening.

*** Governments will need to scrutinise prices for particular groups of consumers more closely—and, if need be, step in to protect them. That means the state being more willing to shape the market, on the basis of how consumers actually behave, to increase competitive pressure on prices. Data on tariffs and pricing must be made transparent, for example. This adds up to a novel approach—both more interventionist and more pro-competition.

There is a moral dimension to thrash out too—what are the ethics of price discrimination? What is OK, and what is not OK? And this will play into politics. Parties should be jostling for this new terrain, explaining how they will stop the more pernicious effects of these trends without blunting the irreplaceable value of the market. In Britain, as things stand, neither the Conservatives nor Labour are there yet. The Consumer Green Paper scheduled for the new year is good a place to start.

And behind this lies a bigger question: whose job is it to solve these challenges? Can a new relationship between the market and the state be settled technically, out of the hands of politicians, by the experts at our big utility regulators? That’s a nice idea, but when technology and public opinion move so fast, there’s every danger regulators will be caught short. In the end, whoever rises to this challenge, it seems likely that only a bold response will endure.