It is easier to fix biased algorithms than prejudiced people

In an article for the New York Times, Sendhil Mullainathan poses a provocative question: is it easier to recognize biases in algorithms than to "fix" biased people? We have discussed this type of problem here before in technology, which ultimately says more about the human species than about artificial intelligence itself—though that does not mean the problem is any smaller. In reality, for Mullainathan, it is larger.

The journalist cites a study published 15 years ago as a reference, in which two people applied for a job. Their resumes were similar; one was named Jamal and the other Brendan. This year, two patients sought medical care, both suffering from diabetes and high blood pressure, but one patient was white and the other was Black. In both studies, we see the issue of racial injustice: while in the first, the candidate with a name associated with a Black individual received fewer interview invitations, in the second, the Black patient received worse care.

The difference lies in a crucial element: in the first case, the problem was the hiring managers who make biased decisions, whereas in the second, the problem was a computer program. Mullainathan happens to be a co-author of both studies, and in this sense, he sees two lessons to be learned: one focused on humans and the other on algorithms.

In the first study, the researchers wanted to answer the question of whether Greg and Emily would be more employable than Lakisha and Jamal. The answer, as you might have already guessed, was yes—much more employable. "Simply having a white-sounding name increased the callbacks for job interviews by 50%," writes Mullainathan.

In the study published in October, which focused on the case of patients in the American healthcare system, the primary algorithms used became the object of study. In this case, these programs were designed to help medical centers manage the large volume of data that allows staff to know which patient is most urgent and most ill. "We found a racial bias already embedded in these algorithms. At similar levels of illness, Black patients were seen as less urgent than white patients. The magnitude of the distortion was immense: by eliminating the algorithmic bias, it would be possible to double the number of Black patients who would receive extra help. The problem lies in a subtle engineering choice: to measure the 'level of illness,' they used the majority of available data, which was healthcare spending. But because society spends less on Black patients than on equally ill white patients, the algorithm underestimated the needs of the Black patient," explains the study's author.

Now, the way to approach these two problems is different. According to the researcher, the first study felt more like an undercover operation than a truly academic study. All the logistical work of hunting for job openings, faxing resumes, and making calls to then analyze the discriminatory behaviors of a particular group—in this case, HR managers—was much more difficult than diagnosing the bias in the healthcare system's algorithm. "It was a statistical exercise, the equivalent of asking the algorithm 'what would you do with this patient?' hundreds of thousands of times and mapping the racial differences. The work was technical and routine, requiring no discretion or resources."

In other words, humans are much harder to deal with than algorithms. Our explanations for our behavior change all the time and are adapted culturally and socially. Measuring racism in people requires creating controlled circumstances in the real world where only the racial factor matters. In the case of an algorithm, however, it is possible to create controlled scenarios simply by using specific data and observing its behavior.

Furthermore, humans also behave differently regarding what to do about these biases. In the first study, the authors found it difficult to solve the problem because detecting discrimination does not necessarily mean the entire company is discriminatory, and on the other hand, changing a person's mind is not easy. At most, training might demonstrate some changes, but that is perhaps the best-case scenario.

In the case of the algorithm, the researchers developed a prototype to help companies—in this case, the American healthcare system—identify this bias and fix their management software for free. Many medical centers accepted, which proves that it was not intentional for that to occur there. And in this sense, it is possible to see how changing algorithms is easier than changing people, since software only needs an update, but people's minds are quite different from that.

But regarding algorithms, it is possible to think, for example, of a type of regulation, as proposed in a article published in 2018. "With better regulation, better algorithms can help ensure that people receive fair treatment in society, although this does not solve the deep and structural biases that continue to plague the United States. Fixing societal biases is no easier than fixing people's biases."