Models are not reality. Nor are they data

Matt Ridley provides some perspective on models, be they COVID or climate change or anything else.

MR is on target.

------------------------------------------

Again and again, worst-case scenarios are presented with absurd precision, and the problem goes further than Britain's slow reopening

My article for The Telegraph:

Britain leads the pack on vaccination, but lags far behind America, Germany and France on liberation. A big reason is that our Government remains in thrall to a profession that has performed uniquely badly during the pandemic: modellers. The Government’s reliance on Sage experts’ computer modelling to predict what would happen with or without various interventions has proved about as useful as the ancient Roman habit of consulting trained experts in “haruspicy” – interpreting the entrails of chickens.

As Sarah Knapton has revealed in these pages, the brutal postponement of Freedom Day coincided with the release of a bunch of alarmist models predicting a huge new wave of deaths. The most pessimistic, inevitably from Imperial College, forecast 203,824 deaths over the next year. It did so by assuming just a 77-87 per cent reduction in hospitalisations following two vaccinations, despite the fact that real world data shows two vaccinations to be between 92 per cent (AstraZeneca) and 96 per cent (Pfizer) effective in preventing hospitalisation. That would cut the Imperial forecast of deaths by a gob-smacking 90 per cent to 26,854.

This keeps happening. In April the modellers assumed a 30 per cent effectiveness for the vaccine at preventing the spread of the virus. This was described as “a pessimistic view – but it is plausible, it’s not extreme”, by Professor Graham Medley, chairman of the SPI-M sub-group of Sage. It turns out it was far from plausible. At the end of March the BBC’s favourite modeller, Imperial College’s Neil Ferguson, was forecasting that by June 21, even with “optimistic” assumptions, less than half of Britain would be protected against severe disease by vaccination. The true figure is over 80 per cent of those aged 18 and over that have been vaccinated at least once.

This is the same Professor Ferguson who told us in the 1990s that millions might die of mad-cow disease. The correct number, as it turned out, was 178.

The experts would reply that ours is an uncertain world, but we knew that already. If you don’t know, say so. That new variants came along at the end of 2020 and ignited a terrible second wave may seem to have vindicated pessimists, but their models had no assumptions about variants in them. Being right for the wrong reasons was the excuse of haruspicy, too.

Again and again, worst-case scenarios are presented with absurd precision, sometimes deliberately to frighten us into compliance. The notorious press conference last October that told us 4,000 people a day might die was based on a model that was already well out of date.

Pessimism bias in modelling has two roots. The first is that worst-case scenarios are more likely to catch the attention of ministers and broadcasters: academics are as competitive as anybody in seeking such attention. The second is that modellers have little to lose by being pessimistic, but being too optimistic risks can ruin their reputations. Ask Michael Fish, the weather forecaster who in 1987 reassured viewers that hurricanes hardly ever happen.

As Steve Baker MP has been arguing for months, the modellers must face formal challenge. It is not just in the case of Covid that haruspicy is determining policy. There is a growing tendency to speak about the outcomes of models in language that implies they generate evidence, rather than forecasts. This is especially a problem in the field of climate science. As the novelist Michael Crichton put it in 2003: “No longer are models judged by how well they reproduce data from the real world: increasingly, models provide the data. As if they were themselves a reality.”

Examine the forecasts underpinning government agencies’ plans for climate change and you will find they often rely on a notorious model called RCP8.5, which was always intended as extreme and unrealistic. Among a stack of bonkers assumptions, it projects that the world will get half its energy from coal in 2100, burning 10 times as much as today, even using it to make fuel for aircraft and vehicles. In this and every other respect, RCP8.5 is already badly wrong, but it has infected policy-makers like a virus, a fact you generally have to dig out of the footnotes of government documents.

In 2020 even the BBC ran an article about how RCP8.5 had been misused. Yet a year later in March 2021, the Met Office published a study claiming that climate change would make dairy cattle and potatoes wilt in the heat in 30 years. Sure enough, it was based on RCP8.5, which the Met Office described as “credible” in its press release. They just cannot help themselves.

Nearly two decades ago, Professor Philip Thomas of Bristol University got the death toll from mad-cow disease right – “a few hundred”, he said – and was pilloried for his optimism.

He told an inquiry that “the Government’s continued inability to give proper consideration to the spectrum of scientific opinion… must be a cause for major concern. It is clear that those tasked with devising policy – ministers and civil servants – need to adopt a more critical attitude to the scientific advice they are offered, even when that advice comes from one of their advisory bodies.” That warning was ignored.