Intellectual honesty in the time of COVID-19

In my previous blog post, where I argued that we didn’t really know what was going on, I distinguished between two hypotheses:

The conclusion of this discussion is that, unfortunately, it’ not clear that even the data on the number of deaths are reliable and, in particular, that we can use them to make inferences about the dangerousness of the coronavirus from comparisons between countries. As I see it, there are basically two possibilities:

1. The virus is as dangerous as we feared and in particular sends far more people to intensive care than seasonal influenza, but due to a combination of luck in the way the epidemic initially spread, a very aggressive eradication policy launched early enough and/or a climate unfavorable to the spread of the virus, a number of countries have managed to stop the epidemic before it kills many people, while in other countries where these conditions were not present many people have already died and many more will die in the coming months, although sometimes you can’t see that yet because of the criteria used to count the number of deaths ascribed to the coronavirus.
2. For some reason, something went wrong in Italy and China at the beginning of the epidemic, which explains the large number of deaths there, but in reality the virus is less dangerous than we feared and the number of deaths will eventually subside, which is already the case in China and is perhaps beginning to be the case in Italy. The situation will improve in the countries that at the moment appear to be on course to follow Italy’s trajectory and the number of deaths will not suddenly explode in those where it’s still low. Similarly, in the countries that now seem to have managed to stop the epidemic or even to prevent it from ever starting, there will be no resurgence or rapid spread of the epidemic in the coming months.

If the first hypothesis is correct, then I’m not saying that the Imperial College simulations are reliable because they’re based on a lot of assumptions in addition to those about age-specific infection fatality rates, hospitalization rates and the proportion of hospitalized cases that require intensive care admission, but at least those assumptions are not crazy and it probably means that we’re at the beginning of an unprecedented health disaster. On the other hand, if the second hypothesis is correct, then those assumptions are far too pessimistic and I think that, regardless of the validity of the other assumptions in the model, the model’s predictions massively exaggerate the danger.

I’ve been asked recently where I now stood on those 2 hypotheses, so I wanted to give a quick update here, but I don’t have time to do a detailed analysis, so I will be quick. Beside, the truth is that I’m pretty much right where I was when I wrote that, so it doesn’t seem worth the effort to explain in painstaking detail why I’m just as confused as I was 10 days ago.

As you may remember, in my last post, I was very puzzled by the huge variability between countries in the number of deaths. Well, to be honest, I still am:As you can see, if you look at the number of deaths per 100,000, the situation looks totally different in different countries. By the way, I’m not going to show that, but the same thing is true within-country as well.

To be sure, if you look at how the number of deaths has changed over time, many of the countries that seem to be experiencing something very different suddenly look like they’re following roughly the same trajectory, only with a lag:A lot of people look at this kind of chart and they think it shows there aren’t really any substantial differences between countries other than the fact that some are further along the epidemic curve than others, but to be honest, I’m not so convinced by this argument.

First, note that not all countries where the number of deaths per 100,000 is still low have a trajectory that mirrors that of the countries where it’s high. In particular, it’s not the case of Japan, where there are still very few deaths because of COVID-19, at least according to the official figures. Yet it’s significantly more connected to China than Europe and the first case of COVID-19 was diagnosed in Japan before anywhere in Europe or the US. It also had a death before any country in Europe or the US. Thus, the virus has clearly been circulating over there for a long time and, even if you think that the reproduction rate is lower over there because of cultural differences or whatnot, there should be a lot of people who have been infected by now. Moreover, given the demography of the country (Japan has the world’s oldest population), many of them should be old. So where are the bodies? It doesn’t make sense.

I know a lot of people think that it’s just that the government is not testing, and it’s true that it seems to be testing very little and also has very few cases, but if the situation were really as bad as it currently is in Italy, France or Spain, we’d surely know it by now. Sure, Japan has more ICU beds per capita than any of those countries, but I don’t think people who reply to this argument by pointing this out realize how quickly they are filling up in France at the moment:If something like that were happening in Japan, I don’t believe for a second people wouldn’t notice, it’s just not credible. And there are other countries which are even more connected to China I didn’t show on the chart above, such as Vietnam, where apparently nobody has even died yet. Sure, Vietnam isn’t exactly the most transparent country in the world, but still I doubt they could hide something like what is happening in Italy.

Things do look a bit worse for Germany, which seems to be on a similar trajectory as the most badly hit European countries with a lag, and where the case fatality rate is slowly increasing:But even if they seem to be testing a lot, we don’t know what proportion of the population has already been infected, so even their relatively low case fatality rate could still overestimate the infection fatality rate by a lot. Moreover, the virus has also been circulating for a while in Germany and they’ve only gone on a very partial lockdown a few days ago, so a lot of people should already have been infected, yet only a few hundred people have died so far. Sure, it looks like it’s increasing, but the mere fact that it’s not already much higher if the infection fatality rate is as high as most people assume in their models is already very surprising. Of course, many people have all sorts of explanations for that, but they all seem ad hoc and speculative to me. They may be right, but I don’t think we’re justified in believing any of them at the moment.

I still regard as plausible the theory that the infection fatality rate is much lower than what most people assume, but our health care systems (some of them anyway) are fragile enough that, if just a few things go wrong (I have lots of theories about what those could be, but they’re just as speculative as the theories people who think the infection fatality rate is definitely above 1% use to explain away anomalies like Japan), things can spiral out of control and something just a little bit worse than what we’re used to can turn into a disaster. To be clear, at this point, I have no doubt this virus is intrinsically more dangerous than the flu. (As I briefly argue below, we already know for a fact that, at least in some places, it has put the hospital system under far more stress than any recent epidemic of flu. Perhaps the infection fatality rate is actually not that much higher, but nothing is going to change that, it’s already beyond dispute in my opinion.) But I’m not sure how much more dangerous than the flu it is exactly and, while an infection of fatality rate of 1% wouldn’t surprise me, it also wouldn’t surprise me if the actual figure were 0.3% and, if I had to bet, I would probably bet on the latter, although I wouldn’t bet much. It could be that even something like that, which is only ~3 times as bad as the flu in terms of infection fatality rate and such that a significantly higher proportion of cases require hospitalization, is enough to make our health care systems fail if a few things go wrong.

Of course, I’m not claiming that what I have said in this post establishes that, but that’s a theory I find plausible right now and I just hope that I have said enough for you to understand where I’m coming from and to convince you that it’s at least not crazy. There are many other things I could say to support this theory and there are many other things people who think it’s not plausible could say in response. (For instance, see this post by Tyler Cowen on Marginal Revolution, which focuses on the anomalous case of Iceland and have a look at the comments to see how people try to explain that away.) Given the data we currently have, which contain a lot of extremely weird things that no theory of what is going on can explain without making a lot of largely ad hoc hypotheses, I don’t think a consensus would or indeed should emerge from such a discussion. Unfortunately, as I argued 10 days ago in my previous blog post, I still think the data are way too all over the place to reach any other conclusion that we don’t really know what is going on because everything is weird. In that respect, it’s telling that, among the experts surveyed on March 16-17, only 3 out of 18 were able to predict the number of cases the CDC would have reported by March 29 and that’s only because the 3 experts in question had given a huge range of estimates, which betrayed their uncertainty. I know a lot of people don’t want to hear this, but the truth is that nobody has any clue what’s going to happen.

At this point, compared to most people I know, the fact that I’m taking very seriously the possibility that COVID-19 is not that much worse than the flu probably makes me kind of optimistic, although to be clear I’m still extremely worried. In fact, I’m still worried enough that, as I recently argued in the National Review, I think the right course of action right now is to shut everything down for at least 2 weeks. Hopefully, by then, someone will have done a serological study based on a large enough random sample and we’ll have the results. If a much larger proportion of the population turns out to have already been infected than we currently think, then it will mean the virus is not as dangerous as we think and we don’t have to stay much longer in lockdown, although in some places — such as France — we’ll probably have to wait for ICUs to empty before it’s safe to do so. In that case, it may be that the lockdown was unnecessary in some or even most places, but if the danger is past and it’s safe to return to a normal life, a few weeks of lockdown won’t be the end of the world. On the other hand, if the virus really turns out to be as dangerous as we fear and we didn’t go on lockdown, the consequences would be far worse.

Some people claim that, if we had followed this logic in the past, then we’d have shut everything down several times for nothing in the past 30 years. (Here is a thread on Twitter where someone makes this case very well in response to my argument. Although I ultimately disagree, I’m sympathetic to a lot of what he says, so I encourage you to read it, if only to challenge your priors.) This is supposed to be a reductio of my argument, but although I think it’s a real worry and I don’t want to suggest it’s a ridiculous objection, I don’t find it convincing for at least 2 reasons. First, I simply don’t think it’s true that, if we had followed this logic in the past, we’d had shut everything down for weeks several times in the past 30 years . I don’t think that, in the past 30 years, it has ever happened before that China, where those viruses usually come from (because for various reasons the country is a perfect environment for their emergence), completely shut down a whole province with a population almost the size of France and brought the rest of the country almost to a standstill because of a virus. Nor has it ever happened during that period that, in several developed countries, ICUs reached capacity in less than 3 weeks because of a pandemic.

This last point is disputed, so using French data (which is more readily available to me and makes sense given that France is one of the worst-affected countries), I want to compare how much stress the pandemic of COVID-19 is putting on hospitals compared to the flu during the past 10 years. Many people cite estimates of excess mortality caused by the flu, which are sometimes very large, to minimize or at least put in perspective what is currently happening. For instance, according to this paper, more than 22,000 deaths per year on average were attributable to the flu in Italy during the 2013/2014, 2014/2015 and 2016/2017 seasons. As they point out, this is more than the current death toll of COVID-19 in Italy, which as of April 1 was 13,155. However, as I already noted in my previous blog post, those figures are simply not comparable, because they don’t measure the same thing. It won’t be possible to estimate the excess mortality caused by the pandemic of COVID-19 until more than a year after it’s over, so there is no way to make this comparison at the moment. However, the number of people admitted in ICU is immediately observable, so we can use it to compare how much stress the pandemic of COVID-19 is putting on hospital relative to what epidemics of flu did in the past.

During the 2009-2010 pandemic of H1N1, France set up a severe influenza cases surveillance, whose goal was to collect exhaustive data on the people who had been admitted to ICU because of the flu and that was kept in place during subsequent flu seasons. According to this paper, which describes how this monitoring system was implemented, hospitals were required to report any case with “(i) a positive RT–PCR performed on a nasal swab or broncho-alveolar lavage, (ii) severe clinical influenza, likely to be caused by 2009 pandemic influenza virus according to the clinician, even in the absence of laboratory confirmation, and (iii) clinical influenza and an epidemiological link with a confirmed case of pandemic influenza”. Of course, no monitoring system is perfect and I’m sure that some cases were missed, but this seems pretty thorough and I see no reason to think significantly more severe cases were missed than for COVID-19 today. Note that what people call “the flu” is not a unique pathogen but a whole family of viruses and that on any given year several different viruses in that family are circulating. The monitoring system I just described picked up anyone in ICU who had been infected by a virus in that family and probably a bunch of people who had been infected by another pathogen that caused flu-like symptoms.

I have read the epidemiological reviews about the flu published by Santé publique France, the French equivalent of the CDC, for every flu season between 2010/2011 and 2018/2019 and made a chart that shows the number of people who were admitted to ICU because of the flu during that period:As you can see, it varies a lot, but even during the worst season, the total number of admissions to ICU for the flu never exceeded 3,000 in that period.

By contrast, there are currently 6,017 people in ICU with COVID-19 in France, so we are already way past the number of admissions to ICU caused by the flu during the worst season in the past 10 years. If you include the number of people who have died of COVID-19 in hospitals so far, namely 4,032, as well the people who have left ICU because they had gotten better, we must already be approaching and almost certainly even exceeding 10,000 admissions to ICU in less than 3 weeks. (Unfortunately, with the data published by the French government, it’s impossible to compute the cumulative number of admissions to ICU since the beginning of the outbreak in France.) Moreover, as we have seen above, this number is rising very quickly.  The French government recently announced a plan to increase the capacity to at least 14,000 ICU beds by mid-April, but if the number of admissions to ICU continues to grow at the current pace, it’s not clear that even that would be enough to satisfy the demand when the epidemic reaches a peak. Even if the rate of increase starts slowing down in a few days, which I’m hopeful it will, I don’t see any plausible scenario in which the cumulative number of admissions to ICU caused by the epidemic of COVID-19 in France won’t exceed that caused by the flu epidemic in 2017/2018 by a factor of at least 10 when all is said and done.

Of course, as I noted in my previous blog post, the numbers are probably not entirely comparable. It’s plausible that people with flu-like symptoms are more likely to go to the hospital than during a traditional flu epidemics, but it’s not clear they’re more likely to be admitted to ICU conditional on the severity of their symptoms (I even suspect the opposite is true, since doctors fear ICUs will soon be overwhelmed, which is already the case in some cases), so I doubt this has a substantial effect. Moreover, the French government has been very clear that unless you have serious difficulties breathing, you should just stay at home and wait for the illness to pass. People who don’t have acute symptoms are not even tested. Thus, in France at least, this pandemic is putting hospitals under far more stress than any epidemic of flu in the past decade. It’s possible that, in spite of that, the infection fatality rate will not be much higher than for the flu, but in terms of how much stress it puts hospitals under I think it’s already beyond dispute that COVID-19 is no flu. Moreover, although the number of people in ICU has been increasing particularly quickly in France, something very similar is probably true in several other countries.

Thus, I deny the premise of the objection to my argument that we should use the precautionary principle in this case and shut down everything until we can figure out what is going on, because I don’t think it’s true that what is currently happening has happened several times in the past 30 years. As I have argued above, it may not be as bad as we fear, but it’s definitely very unusual. Another reason why I don’t find the objection that, if we followed the logic of my argument, we’d shut everything down on a semi-regular basis, convincing, beside the fact that I don’t think it would have happened as often in the past as the people who make that objection think, is that I actually think we probably should do that more often. Shutdowns when a novel pathogen that seems potentially very dangerous appears should become like fire drills that we do every time we see enough fire. Sure, this means that we’ll lose a few GDP points over a 50-year period, but it also means that, when a real killer comes, and it will come eventually, we’ll have a much better chance of avoiding a carnage of epic proportions. I’m not going to attempt it here, but I’m pretty sure a cost-benefit analysis would vindicate this strategy.

However, if you argue that we should shut everything down until we know more about what is happening, it’s very important that you make it clear that it’s not because you know what’s going to happen, but because you don’t know, since otherwise it will durably undermine the credibility of the scientific establishment if this pandemic turns out not to be as bad as we fear and this might result in a disaster if people no longer take you seriously when something really bad comes. The problem I have with people relying too much on models is that it gives a false impression that there is much less uncertainty than there actually is and, if the predictions of those models end up being wildly off, scientists will no longer be taken seriously when they use models to guide public policy. This is not just true of models of epidemics, but also of climate models and a lot of other things. So people really should think carefully about this and, if they argue that we need to apply the precautionary principle in this case (as I think we should), they should make it absolutely clear that it’s not because we know the worst case scenario or even something close to it is going to happen and be ready to revise their view about what is happening as more data come in.

That being said, I want to conclude by talking about a behavior I’m seeing increasingly often on social media, which is really starting to annoy me and explains the title of this post. A lot of people think that this pandemic has been overblown and that it will turn out not to be as bad as most people think. I don’t think there is anything wrong with that per se and, as I have already noted, I even think that people who push back against the apocalyptic predictions are doing a public service by forcing us to look at all the data and not just those which are consistent with a pessimistic scenario. Indeed, as I’ve been arguing in this post, I think a theory of that sort is much more likely than most people assume. Most people who find this theory very likely also seem to think lockdowns are a mistake. I disagree with that, but this is also fine, I think it’s something reasonable people can disagree about and I don’t think we should vilify people who are on the other side of that issue.

Where I have a problem is when they don’t come out and say openly that they oppose lockdowns. Well, I guess even that is something I can understand, because unfortunately I think many of them have good reasons to fear that, if they did, they would get a lot of hate for it. I’ve been speaking up against silencing dissent all my life and I’m not going to change my mind about this because of this pandemic, so I want to be clear that, unless there is good evidence of intellectual dishonesty (which in many cases, but hardly all of them, there is unfortunately), this is wrong and people shouldn’t do it. However, I’m convinced there are also many people who think this is overblown and that lockdowns are a mistake, but don’t say it openly because they are positioning themselves so that, if their prediction turns out to be true, they can say they were right and that other people were irrational. For reasons I have explained above, I don’t think lockdowns will have been a mistake even if the pandemic turns out not to be as bad as we fear (it’s possible that lockdowns are not what’s causing the curve to flatten in countries that have implemented them, but I think at this point it would be unreasonable not to assume they are and act accordingly until we know more), but even if you disagree with that it’s intellectually dishonest to do what I just described.

The people who are engaged in this strategy are refusing to take a clear stance on the only question that, as far as decision-making is concerned, matters right now, but they are positioning themselves to reap the reward of their indecision later, if this pandemic turns out to be less serious than most people think. Again, I think it’s fine to think this is going to happen and it’s also fine to argue that lockdowns are a mistake, but unless you are willing to say so loudly and clearly now, when there is still a lot of uncertainty about what’s going to happen, you have no right to brag when this uncertainty has been lifted and you should just shut up if your prediction is realized, otherwise it’s just too easy. Perhaps I’m just too distrustful, but lately I’ve seen a lot of people who I suspect are playing this game, so let’s be clear about what the rule of the game are right now. If you want to be able to brag later, then you have to be willing to put your cards on the table now. But if you don’t speak out now, you’ll have to shut up later.

ADDENDUM: In this post, I mostly tried to explain why I thought it wasn’t crazy to think that COVID-19 might not be as dangerous as we fear, but as I noted, there are many things in the data that point to the opposite conclusion and that a proponent of the pessimistic scenario could bring up in support of their hypothesis. Marco Del Giudice, professor of psychology at the University of New Mexico, brought to my attention this argument that Gregory Cochran made on his blog against the idea that perhaps a much larger share of the population has already been infected:

Some guys at Oxford suggest that that A. a huge fraction, maybe 50%,. of the pop in England have already had it ( and are thus immune) , and B. that the fraction of those infected that get seriously ill is much smaller than it looks. If both things were true, the death rate numbers might work out, they say.

 

If that were the case,  you would think that any set of tests  would show  a big fraction people with the virus – which is not the case, even though the samples we’ve tested are high-biased.

 

But, some say, maybe those hypothetically numerous already-infected people have already cleared the virus and thus don’t test positive for it.  But that can’t be right either, since for a fast-growing epidemic ( which this clearly is, from the rapid increase in deaths) , the majority of all cases are very new – less than two weeks old ! So that is obviously untrue as well.

I agree that, when you look at the proportion of tests that come back positive in various countries, this sounds pretty compelling.

For instance, here is a chart that shows how this proportion has changed in Italy, where I added a 3-day simple moving average to account for the fact that it can take a few days between the time a specimen is collected and the time when the result comes back, among other things:I can think of a few rejoinders, but to be honest, I’m not sure I really believe them myself.

On the other hand, there are still the cases of Japan, Vietnam and several other Asian countries, which should have been swimming in COVID-19 for a while now, yet where very few deaths attributed to the virus have been recorded so far. Marco suggests that stochasticity in the initial spread of the virus might explain the delay and I guess it’s possible. (The large role that random factors likely played during the initial stages of the epidemic is something I kept stressing myself when people in France were saying that, since we had our first case before Italy and bodies still weren’t piling up, we were safe thanks to the amazing job our government had been doing to protect us. The truth, of course, is that our government hadn’t done shit and that we were anything but safe, as the past few days have unfortunately demonstrated.) In the case of Germany, I agree this explanation is looking increasingly persuasive (although I’m still not entirely convinced), but I have a much harder time buying this story in the case of Japan, Vietnam and other Asian countries. It just seems to me that, given how connected they are to China and how quickly COVID-19 seems to spread, there should be a lot more deaths over there already if this virus really had a fatality rate of ~1%.

But everything is so weird and makes so little sense to me right now that I also wouldn’t be that surprised if the number of deaths started to blow up over there soon. As Marco aptly noted in the brief exchange I had with him, it’s very difficult to get reliable intuitions about how stochasticity plays out in this kind of process. As he also noted, simulations could be helpful with that, but I haven’t really tried to play with stochastic models to see if random factors could explain such a delay in the timing of the moment where the epidemic blows up in different countries. Again, there is much more that could be said on behalf of both sides of this debate, but I hope to have said enough to give you a sense of where I’m coming from when I say that everything is weird and that I don’t understand what is happening.

ANOTHER ADDENDUM: This just occurred to me, but another factor that should be taken into consideration when comparing how much stress COVID-19 is putting on hospitals relative to the flu is that, when I conclude that in the best case scenario the cumulative number of admissions to ICU caused by the epidemic of COVID-19 in France will exceed that of ICU admissions caused by the 2017/2018 epidemic of flu by a factor of 10, this will only have been achieved by putting the entire country on lockdown for several weeks. Of course, I agree that we can’t know for sure that, when the number of daily admissions to ICU finally begins to go down, it will have been because of the lockdown and other social distancing measures, but it seems extremely likely at the moment. Of course, a substantial part of the population is vaccinated against the flu, whereas we have no vaccine for COVID-19. So this doesn’t necessarily speaks to the intrinsic dangerousness of COVID-19 compared to the flu, but since we have no vaccine for the moment it doesn’t really matter.