Not everything is worth knowing

There are more things you might not want to know besides your h-index.

AI can find patterns and make inferences using relatively little data. Only a handful of Facebook likes are necessary to predict your personality, race, and gender, for example. Another computer algorithm claims it can distinguish between homosexual and heterosexual men with 81 percent accuracy, and homosexual and heterosexual women with 71 percent accuracy, based on their picture alone.1 An algorithm named COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) can predict criminal recidivism from data like juvenile arrests, criminal records in the family, education, social isolation, and leisure activities with 65 percent accuracy.

First off, those numbers aren't very impressive. Second, they're misleading. What it's finding is people who correlate with societal expectations of sexuality, which is not the same thing as their actual sexuality at all. There is no intrisinic difference in appearance between people of different sexuality, or at least not much evidence of any. There's no way it can ever know that unless you directly identify your sexual orientation. The same can applies to the other parameters : all it can do is make a probabilistic guess. A 99% statistical chance that you're from Texas has no force against factual proof that you're actually from Greenland. A 65% chance of success is hardly evidence that it's able to discern some amazing deeper insight - it's a bad guess, not knowledge or understanding. However, this does not invalidate the central point.

Knowledge can sometimes corrupt judgement, and we often choose to remain deliberately ignorant in response. For example, peer reviews of academic papers are usually anonymous. Insurance companies in most countries are not permitted to know all the details of their client’s health before they enroll; they only know general risk factors. This type of consideration is particularly relevant to AI, because AI can produce highly prejudicial information.

Deliberate ignorance, Hertwig and Engel write, can help people to maintain “cherished beliefs,” and avoid “mental discomfort, fear, and cognitive dissonance.” The prevalence of deliberate ignorance is high. About 90 percent of surveyed Germans want to avoid negative feelings that may arise from “foreknowledge of negative events, such as death and divorce,” and 40 to 70 percent also do not want to know about positive events, to help maintain “positive feelings of surprise and suspense” that come from, for example, not knowing the sex of an unborn child.

The AI “gaydar” algorithm, for example, appears to have close to zero potential benefits, but great potential costs when it comes to impartiality and fairness. As The Economist put it, “in parts of the world where being gay is socially unacceptable, or illegal, such an algorithm could pose a serious threat to safety.”

These examples illustrate the utility of identifying individual motives for ignorance, and show how complex questions of knowledge and ignorance can be, especially when AI is involved. There is no ready-made answer to the question of when collective ignorance is beneficial or ethically appropriate. The ideal approach would be to consider each case individually, performing a risk-benefit analysis. Ideally, given the complexity of the debate and the weight of its consequences, this analysis would be public, include diverse stakeholder and expert opinions, and consider all possible future outcomes, including worst-case scenarios.
http://nautil.us/issue/61/coordinates/we-need-to-save-ignorance-from-ai?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication