After the death of Laquan McDonald (which for what it’s worth strikes me as being murder pure and simple, but this has no bearing on what I’m going to be talking about), the Mayor of Chicago created a task force, with the job of scrutinizing the practices of the city’s police department. The task force released its report a few months ago and, brace yourself, it concluded that Chicago Police Department was plagued with racism. The media, both local and national, from the Chicago Tribune to the New York Times, reported that conclusion uncritically. But I have read the relevant section of the report and, of course, it doesn’t demonstrate anything of the sort. As every report of that sort, it has absolutely no scientific worth, but is just a political document whose only purpose is to placate the populace and prevent riots.
I want to focus on a particular fallacy that the authors of that report committed, because it’s one of the most common fallacies that people use to conclude that racism is rampant in law enforcement. (For instance, it can also be found in the Department of Justice’s report on Ferguson Police Department, which was released after the death of Michael Brown.) Moreover, I think it’s also interesting from a purely logical/statistical point of view, if you’re into that kind of things. Finally, it illustrates very nicely how the lack of political diversity in social science can result in bad consequences, such as systematic error.
The argument is nicely summarized in this article, which was published in the New York Times in 2001, showing that it’s a fallacy with a long history:
It is no longer news that racial profiling occurs; study after study over the past five years has confirmed that police disproportionately stop and search minorities. What is news, but has received virtually no attention, is that the studies also show that even on its own terms, racial profiling doesn’t work.
Those who defend the police argue that racial and ethnic disparities reflect not discrimination but higher rates of offenses among minorities. Nationwide, blacks are 13 times more likely to be sent to state prisons for drug convictions than are whites, so it would seem rational for police to assume that all other things being equal, a black driver is more likely than a white driver to be carrying drugs.But the racial profiling studies uniformly show that this widely shared assumption is false. Police stops yield no significant difference in so-called hit rates — percentages of searches that find evidence of lawbreaking — for minorities and whites. If blacks are carrying drugs more often than whites, police should find drugs on the blacks they stop more often than on the whites they stop. But they don’t.
So, to be clear, the argument roughly goes as follows:
(1) When you look at the hit rate (i. e. the proportion of people stopped who carried contraband) for whites, you find that it’s higher than for blacks.
(2) Therefore, the offending rate for blacks is lower than for whites or, at least, is not higher.
(3) Yet black people are disproportionately stopped by the police.
(4) So, by inference to the best explanation, the police is racist.
I know that sounds pretty convincing, but the inference from (1) to (2) is actually a statistical fallacy, as I’m now going to explain.
The problem is that, for (2) to follow from (1), a necessary condition is that people are stopped randomly by the police or, more generally, that the sampling methods used by cops to stop blacks and whites, together with facts about the structure of each group, make an unbiased estimator of where is the hit rate for whites, the hit rate for blacks, the offending rate for whites and the offending rate for blacks. Let’s call that hidden premise (1*), so I can refer to it quickly later. What (1*) means, roughly, is that if you looked at samples of people stopped by the police and recorded the hit rate for blacks and whites in each case, the mean of over those samples would tend to as increases.
So, in order to be able to infer (2), it is not enough that (1) be true, it must also be the case that (1*) is true. The problem is that, not only do we have no reason to believe that (1*) is true, but on the contrary there is absolutely no doubt that it’s false. Indeed, nobody questions that cops are biased against blacks, in the neutral sense that they stop blacks disproportionately. That’s basically what (3) says in the argument, which everyone agrees is true. The question is whether that bias results from a true belief that the offending rate is higher among blacks than among whites or, as (4) asserts, from prejudice on the part of cops. (I’m using “racist” to mean something like “discriminate against black people because they are prejudiced”. This may not be a good definition of “racism”, but that’s irrelevant to the point I’m trying to make, for the argument I’m attacking clearly purports to show that cops stop black people more often not because they actually commit more crimes but rather because they are prejudiced against them.) But, whether or not (4) is true, as long as (3) is true, (1*) is probably false. Indeed, as long as (3) is true, it’s likely that overestimates , quite possibly to the point that comes out as positive even though is actually negative.
Just think about it for a moment: if cops believe — rightly or wrongly, it doesn’t matter here — that blacks are significantly more likely than whites to carry contraband or engage in illegal activities, they are only going to stop whites that have “I’m a criminal” written on their face, whereas in the case of blacks they’re going to cast a much wider net. What this means is that the sample of blacks is going to be much more random than the sample of whites, which is going to contain a lot more guilty people than in the white population at large. So, as long as (3) is true, it’s entirely unsurprising that the hit rate for whites is higher than the hit rate for blacks, even if the offending rate is actually much higher for blacks than for whites. Which is why (4) doesn’t follow from (1) and (3).
It may be helpful to consider a silly example to illustrate the problem. Suppose that, in some weird country, cops are stopping people in the street to see if they have coins in their pockets, because in that country it’s illegal to own coins. The cops stop people who have a beard more often than people who don’t because, for some reason, they think people who have a beard are more likely to have coins in their pockets than people who don’t have a beard. Suppose that, in fact, the cops are right, for 10% of people who have a beard have coins in their pockets while only 5% of people who don’t have a beard do. Suppose, moreover, that among people who don’t have a beard, they only stop people who have a red shirt, while they stop people who have a beard randomly. Perhaps it’s because, for whatever reason, they are convinced that, among people who don’t have a beard, people with a red shirt are significantly more likely to have coins in their pockets. As it turns out, the cops are right again, for although only 5% of people who don’t have a beard have coins in their pockets, 15% of those who don’t have a beard and wear a red shirt do.
A sociologist observes them and notice that, not only do they stop people with a beard more often, but among the people they stop, those who have a beard are less likely to have coins in their pockets than people who don’t have a beard. Indeed, among the people they stop, only 10% of the people who have a beard have coins in their pockets, while 15% of those who don’t have a beard do. He infers that people who have a beard are less likely to have coins in their pocket and, since the cops nevertheless stop them more often than people who don’t have a beard, he concludes that they are prejudiced against people who have a beard and that’s why they stop them more often than people who don’t. Except that, as we have seen, he is wrong. People who have a beard are more likely to have coins in their pockets and, if cops stop them more often, it’s because they know it. The sociologist just committed the fallacy I explained above.
Now just pause to think about what this means exactly. It’s not just that the argument is fallacious, it’s fallacious in about the worst way an argument can be fallacious. Indeed, (4) only follows from (3) if (2) is true, but (2) only follows from (1) if (1*) is true. However, as we have just seen, if (3) is true then (1*) is probably not true. Thus, 2 of the argument’s premises — (1*) and (3) — are incompatible, which makes it a particularly defective argument, to say the least… Moreover, since I’m using “racist” in (4) to mean something like “have bias and that bias results from prejudice”, (4) entails (3). So the problem with the argument is not just that it contains a hidden premise that happens to be false, it’s that if the conclusion of the argument is true, then the argument is not valid and cannot possibly establish the conclusion! And that’s what people who use that argument have not realized: even if cops were prejudiced in that sense, you couldn’t show it by using that argument, for the reason I just pointed out.
Before I stop writing, I want to make two remarks. First, how come that argument is so common even on the part of professional social scientists, given that it’s guilty of a relatively basic statistical fallacy? Of the two authors of the NYT article, at least one of them — John Lamberth, who is a psychologist — surely knows enough statistics to understand the problem with that argument, yet he committed that fallacy all the same and, since I don’t think he is dishonest, I think it’s very likely that he really didn’t notice that his argument was fallacious. Moreover, I’m sure that I’m not the first person to notice the problem with that argument, yet I have never seen anyone point it out. Anyone who has taken an introductory class on statistics knows enough to detect that fallacy, but even people who should know better don’t notice it or, if they do, they don’t say anything.
A pretty natural explanation is that liberals are biased in favor of the hypothesis that cops are racist and the vast majority of social scientists are liberal, so either they are blind to the problem with their argument or, if they notice it (as I’m sure that some have), they don’t say anything because they’re afraid that they would be punished for it. And the truth is that they are right to be afraid, because they would be punished. The fact that this terrible argument is used over and over, even by people who should clearly know better, is a very good illustration of the damages that ideological uniformity can do. If there were more conservatives in the social sciences, enough people would have pointed out the fallacious character of that argument for people to stop using it, because they would have noticed the problem and they wouldn’t have been afraid to talk about it.
Of course, this goes beyond the scientific community, it’s a widespread problem in the culture at large. Race is such a taboo in the US, even more so than in Europe, that it’s absolutely impossible to have a honest conversation about any aspect of the racial problem in this country. And liberals are largely to blame for that state of affairs. You sure as hell aren’t going to solve any problem by burying your head in the sand and vilifying anyone who says inconvenient truths, even when by your own admission you would be incapable of showing that they are wrong. Liberals are prone to talk about the dangers of having a honest conversation about race. But they also completely ignore the dangers of not having a honest conversation about race. I have never seen a convincing argument that the former outweigh the latter. But I guess that’s a topic for another post…
The last remark I wanted to make is that, of course, even if blacks are in fact more likely to commit crimes (which they most certainly are, but that’s also a topic for yet another post), it doesn’t follow that racial profiling is morally justified. My own view of the question is that it’s a very complicated issue, much more than liberals generally assume (because they never consider the costs of abandoning every kind of racial profiling), but also one which I think is kind of pointless to discuss. Indeed, the fact of the matter is that, as long as blacks will commit significantly more crimes on average than whites, the police will engage in some kind of statistical discrimination against them. So, if you want to insist that racial profiling is unambiguously wrong, I’m happy to grant you that because it has little to no bearing on any practical issue. I think a much more interesting question is: how do we make sure that blacks no longer commit more crime on average than whites? I think progressives have potentially a lot of things to say about that and, to be fair, some of them do. But, in order to ask that question and find a solution, you must first be able to have a honest discussion about the facts, which isn’t going to happen as as long as liberals keep treating dissent like the Spanish Inquisition.
EDIT: I wrote a follow-up on this post in which I clarify a few points, so you may want to read it before you comment.