Image: RIBA Collections / GRANGER - Historical Picture Archive, Florilegius, Ratchapoom Anupongpan, Alamy / kosmofish, Shutterstock / Museo Nacional Thyssen-Bornemisza, Madrid / Composite by David McAllister, Prospect

Our true slaving history

A Scottish castle reveals the cruelty and riches of the slave trade, telling a story usually hidden by the narratives of British heritage
November 23, 2024

Every summer, we rent a cottage from a local landowner in the Glens of Angus, a peaceful, largely agricultural district in the foothills of the Grampians. The royal estate at Balmoral, 30 miles away on the other side of the mountains, is probably better known to tourists. We like this obscure part of the world because my wife grew up on a farm here, and so she knows people.

Foraging my way through the beech woods in the usual autumn drizzle, on a hunt for chanterelles, I gaze up the high, bare slopes of the hill of Cat Law and see a romantic vision on the skyline: the derelict battlements of Balintore Castle, a fairytale scene with conical turrets of antique fishtail slate. It is easy to imagine Rapunzel letting down her long hair from its Disneyesque towers. 

Its owner, David Johnston, a former video games designer, bought the castle from Angus council for £80,000 and has selflessly dedicated more than a decade of his life to reviving it. The previous owners had abandoned it to dry rot and rats. An over-optimistic Taiwanese developer had proposed to turn it into a casino—but promptly disappeared. Balintore was listed as a Category A architectural treasure (as per the Scottish scheme for listed buildings). Historic Environment Scotland, a government agency, saw it as one of the country’s most outstanding “buildings at risk”. Now paying visitors can pass the night there, experiencing the ruins of a grander age.

Johnston has indeed done his bit to rescue this “shooting lodge”, which was knocked up in 1860 in a mishmash of styles by a fashionable Victorian architect, William Burn, who made a London living catering to the fantasies of the rich. Tartanry was in vogue thanks to Prince Albert’s Balmoral Castle, built not long before that in 1856. 

But if only that was the end of this building’s history. As I discovered, Balintore has a less than savoury past. Scratch the surface and the shameful truth of Britain’s wealth, generated by the transatlantic slave trade, emerges. As Johnston wrote in 2015, on his blog chronicling the castle’s restoration, he was “quite badly shaken” when he found out the truth.

Balintore, with its faux-medieval style, was built by David Lyon, a past relative of King Charles through his Scottish grandmother, Elizabeth Bowes-Lyon (the late Queen Mother). According to the Gazetteer for Scotland, an encyclopaedia of Scottish geography and history published since 1885, Lyon had “inherited a fortune made by his family through investments in the East India Company”. This discreet mention is misleading: Lyon’s fortune came from Jamaica, not the East Indies—and the source of his riches lay in the darkest underbelly of British history.

David and his brother James are recorded as having sailed to the Caribbean from Scotland in the 1780s to buy slaves and produce sugar. On 18th July 1794, James is named as leading the hunt for a runaway slave called Austin— “rather elderly, has a thick beard, is marked on the right shoulder ‘DK’… and has lost part of one of his ears from the bite of a Negro woman”. There was a £5 reward for Austin’s recapture.

Jamaica was the richest, and one of the first, British slave islands, captured from the Spanish in Cromwell’s time. The Lyons acquired at least two slave estates there, Barnstaple and Holland, in the parish of Trelawny. The environment was challenging for the white slaveowners from across the ocean. On 4th June 1807, James died in Rio Bueno, Falmouth, aged just 47.

Where did the two slaveowning brothers come from? Peerage records assert that the Lyons were born near the burgh of Perth, sons of John Lyon of Castle Lyon. That hulking building by the banks of the Tay was said to be the seat of landowners descended from de Leonne, a Norman warlord who found his way to Scotland. In fact, the Lyon clan’s main estate was always some miles north at Glamis Castle, famous from Shakespeare, and more recently occupied by King Charles’s late grandmother. This divergent branch of the Lyons seems to have been less rich. (Castle Lyon, long out of the family hands, has been renamed Castle Huntly; it was converted into an open prison in 1949).

David took over the Jamaican plantations after James’s death. The year he died, Britain finally outlawed the grim Atlantic shipping trade that supplied the Lyons’ labour force. Between 1640 and 1807, Britain traded more slaves than any other country: more than three million enslaved people are estimated to have been transported from Africa by the British in that time. It had been a huge industry. Impecunious Scottish gentry and landless younger brothers were to be found all over the Caribbean, brutalising their slaves. Indeed, in 1786, before thinking better of it, Scotland’s national poet, Robert Burns, had himself notoriously booked passage to Jamaica, to be a “poor Negro driver”.

The 1807 reform made little real difference to the Lyons’ wealth. Their Caribbean slaves were still being bred like livestock or bought and sold on the black market. In Jamaica, slavery was to continue for the next 27 years. David soon moved to London, where he invested in the new West India Docks on the Isle of Dogs. His City firm, David Lyon & Co, made immense sums out of brokering the continuing slave-sugar trade, owning ships and financing fellow slave plantation owners. When he died in 1827, at his fine London townhouse in Portland Place, David Lyon was worth £600,000, the equivalent of about £80m today.

His son, David Lyon junior, inherited the family slavery business, a fortune estimated to have been worth £171,000 in all—about £20m today—at the age of 33. A couple of years earlier, he (or his father) splashed out 700 guineas (about £80,000 in today’s money) to have his portrait painted by Thomas Lawrence, the London society artist of the day. In a black coat “trimmed with fur and lined with silk… his head set brooding against clouds”, Lyon junior was pictured in the “clothing, pose and hair… of the nineteenth-century dandy”, as the catalogue of the Thyssen Museum in Madrid, where the painting is now displayed, puts it.

With his father’s fortune at his disposal, Lyon dabbled in politics. He became MP for the “rotten borough” of Bere Alston in Devon, via the local Tory squire Lord Beverley, who controlled a handful of purported village voters. Lyon never spoke in parliament and his constituency was almost immediately abolished in the Reform Act of 1832. He belonged to the West India Committee of London slaveowners, many of them absentees like himself. It was a lobby group against reform.

But further reform turned out to mean another payday for Lyon junior. In 1833, the Westminster parliament abolished slavery altogether in British colonies by offering to pay “compensation”—not to the victims themselves but to plantation owners. The compensation files in the UK national archives, now mined by academic researchers, provide an incontrovertible record of former slaveowners’ crimes. In Jamaica, the two sugar plantations originally acquired by Lyon junior’s uncle James now owned respectively 150 and 146 imprisoned slaves. A further 174 slaveworkers were imprisoned in a third plantation owned by Lyon junior. In return for releasing these 470 victims, Lyon was paid cash by the British taxpayer totalling £9,492, 12 shillings and eight pence, about £1.5m in today’s money.

Lyon claimed rights over a further eight plantations that his family had financed. Prolonged litigation to obtain the proceeds from the release of more than 1,000 slaves ensued. The Spring Garden sugar and rum estate had 346 slaves to be freed, the Prospect plantation another 320 and the Retreat estate another 311. Paradise Pen had 101 prisoners. Blue Castle had 222. Mount Edgecombe, 263. Mint Estate, 330. Moreland, 285. There was probably another £5 to £6m (in modern values) in prospect for David Lyon junior.

Flush with extra cash and the promise of more, Lyon set about constructing a gentleman’s estate on the English south coast at Goring. He planted an avenue of oaks, redesigned the local church and hired a fashionable sculptor, Francis Chantrey, to make a marble memorial to his mother. In 1840, he hired Charles Barry, the architect who rebuilt the Palace of Westminster in the mid-19th century, to design a brand-new mansion called Goring Hall. All this extravagance came from slavery money, whether inherited or paid in “compensation” by the state. Lyon is listed, according to the definitive history of the East India Company in this period, as a holder of East India stock, but not as a director or active participant. His wealth mostly came from the slave industry rather than colonial enterprise. 

In 1848, at the age of 54, Lyon junior acquired a wife. The outcome was unhappy. Blanche Bury was much younger, a 29-year-old languishing on the shelf, according to the attitudes of the time, and described by those who met her as “fragile”. His substantial wealth enabled Lyon to pay off the debts of Bury’s mother, Lady Charlotte, who wrote novels. Blanche very quickly went to pieces after a stillbirth. She was described as having “an unfortunate taste for stimulants” and was put away “under medical care” at Torbay. There were no heirs. Instead, at 66, David Lyon junior embarked on his last no-expense-spared project: the castle of Balintore. His shooting parties there seem to have been only occasional, however. When he died seven years after the lodge’s completion, it was in the south of France.

What became of Lyon’s wealth, made of other people’s blood and tears? He left about £26m in today’s money. Valuable items included Balintore Castle, Goring Hall and even that dramatic portrait of himself as a young dandy, painted by Lawrence. His brother, William, now inherited most of this fortune, adding to his own existing portion of the slavery wealth that he had also inherited from their father.

William soon sold off Balintore as a sporting lodge to Thomas Chirnside, a rich Australian bachelor sheep-farmer who returned to his native Scotland occasionally to kill grouse. Ten years later, Chirnside shot himself in a fit of depression, and his brother Andrew inherited all of Chirnside’s property. Balintore, attached to its surrounding estate, eventually ended up in the hands of local Scottish landowners, the Lyells. The castle then began its history of decline, finally being abandoned in 1963.

William Lyon, meanwhile, settled in at his deceased brother’s Park Lane mansion in London and at Goring Hall. In 1886, William bought another house, in Berkeley Square, costing £5,600 (about £1m today) and expended much effort on unsuccessful attempts to get elected to parliament. Goring Hall burned down in 1888. William rebuilt it. He was worth £162,000 in his 1892 will.

The Lyon heirs leased out Goring Hall in 1906. The last Lyon heiress was William’s grand-daughter, Joy. She inherited in 1914, at only seven years old. In 1934, at the age of 27, she sold off all the lands of her inheritance to developers. Goring Hall eventually became a boys’ boarding school and is now a private hospital. In this process of dwindling, the only major possession that Joy seemed to have kept was the Lawrence portrait, itself worth a tidy sum. 

When she died at 48—divorced and in “sad circumstances” in a psychiatric ward—Joy left the Lawrence painting to her friend Betty Carnegy-Arbuthnott, an Olympic fencer who, many years later, auctioned it off at Christie’s. We don’t know the price it fetched, but works by Lawrence can be worth up to £500,000. The portrait turned up, incongruously, in the drawing room of Daylesford, an English country mansion occupied for a while by a billionaire member of the German Thyssen industrial family. Spain bought the whole Thyssen collection after the owner’s death—his widow was Spanish—which is why David Lyon junior’s society portrait today sits in a museum in Madrid. The museum’s account of its provenance does not mention the word “slavery” once.

The saga of Balintore is just one demonstration of how the “long-lost story”, as Tom Devine, a professor at the Edinburgh university, calls it, rises everywhere out of the earth—even in such a little corner of Scotland. A few miles north of Balintore, for example, is Fasque. It was the historic home of the Gladstones, who gave us one of Victorian Britain’s most iconic prime ministers. William Gladstone’s inherited wealth came from his father John’s slave ownership. Charlie Gladstone, their descendant, still has an estate near there and is forthright about his feeling that the truth should come out, and that reparations and apologies need to be made to victims of the slave trade by his own family.

“John Gladstone committed crimes against humanity,” Charlie says. “The best that we can do is try to make the world a better place and one of the first things is to make that apology for him.”

The lost history of slavery rises everywhere out of the earth— even in this little corner of Scotland

To the south of Balintore Castle lies the mansion of Kilkerran in Ayrshire. Justin Welby, the former archbishop of Canterbury, was mortified recently to discover that he is descended from the Fergussons, Scottish slaveowners who lived there. He said he was “deeply sorry” for Anglican links to slavery. In 2023 the Church of England pledged a £100m reparations fund, though an independent report deemed this amount “insufficient”. 

Writer Alex Renton, another descendant of the Fergussons, donates the royalties of his self-exposing book Blood Legacy, also for reparation. A striking number of major British families have formed a group, the Heirs of Slavery, including names such as the Trevelyans and the Lascelles, to come clean in the same way.

At this winter’s Commonwealth conference in Samoa, a groundswell from the Caribbean eventually forced slavery reparations on to the agenda. It led to a rather strangled reference by King Charles in his presiding speech as to how “the most painful aspects of our past continue to resonate”. He would not now object, it is reported, to dropping the word “Empire” from the archaic British honours system. 

This is a bit less than an actual apology. The Starmer government says it won’t talk about reparations, and there are still some other reactionary holdouts. Richard Drax, a former MP with family wealth derived from slavery, and who himself still owns large chunks of Dorset, so far appears to have refused to make amends. In a forthcoming book Drax of Drax Hall: Landed Gentry, Sugar and Slavery (Pluto 2025), Paul Lashmar records that James Drax went to Barbados in 1627, where he pioneered what was to become the British sugar industry by developing an industrial process using slaves.

“I am keenly aware of the slave trade in the West Indies, and the role my very distant ancestor played in it is deeply, deeply regrettable,” Richard Drax is reported as saying, “but no one can be held responsible today for what happened many hundreds of years ago. This is a part of the nation’s history, from which we must all learn”. His old colleague Jacob Rees-Mogg posted on X in October: “They ought to pay us for ending slavery… we were motivated by Christian charity.” 

But both these Tory types were roundly ejected by the UK’s modern voters at July’s general election. I am faintly optimistic that something important may at last be shifting in the right direction. Perhaps then we will understand more of Britain’s true heritage, rather than the fake kind we sell to tourists.