Diversity matters

For all the difficulties that meritocracy is fraught with, the basic principle of hiring people based on skills and not other attributes is sound enough. It's usually implicitly accepted that if this was done, diversity would be a beneficial side-effect (so long as different groups are given equal opportunities of education, experience etc.). But it can also be an important goal in itself : groups are more than the sum of their parts. Merit doesn't just lead to diversity- sometimes it can be the other way around.

Osama Bin Laden made his declaration of war on the United States from a cave in Tora Bora in February 1996. Images revealed a man with beard reaching down to his chest. He was wearing cloth beneath combat fatigues. Today, given what we now know about the horror he unleashed, the declaration looks menacing. But an insider in the foremost US intelligence agency said the CIA "could not believe that this tall Saudi with a beard, squatting around a campfire, could be a threat to the United States of America". 

Richard Holbrooke, a senior official under President Clinton, put it this way: "How can a man in a cave out-communicate the world's leading communications society?" Another said: "They simply couldn't square the idea of putting resources into finding out more about Bin Laden and al-Qaeda given that the guy lived in a cave. To them, he was the essence of backwardness." 

Now, consider how someone more familiar with Islam would have perceived the same images. Bin Laden wore cloth not because he was primitive in intellect or technology, but because he modelled himself on the prophet. He fasted on the days the prophet fasted. His poses and postures, which seemed so backward to a western audience, were those that Islamic tradition ascribes to the most holy of its prophets.

As for the cave, this had even deeper symbolism. As almost any Muslim knows, Mohammad sought refuge in a cave after escaping his persecutors in Mecca. To a Muslim a cave is holy. Islamic art overflows with images of stalactites. Bin Laden modelled his exile to Tora Bora as his own personal hijrah, and used the cave as propaganda. As one Muslim scholar put it: "Bin Laden was not primitive; he was strategic. He knew how to wield the imagery of the Koran to incite those who would later become martyrs in the attacks of 9/11."

When a problem is complex, no one person has all the answers. We all have blind spots, gaps in our understanding. This means, in turn, that if you bring a group of people who share similar perspectives and backgrounds, they are liable to share the same blind spots. And this means that far from challenging and addressing these blind spots, they are likely to be reinforced.

I don't really have much to add except that getting the right kind of diversity of opinion is difficult. While network analysis in society is very interesting, I do wonder to what extent networks are just consequences of the procedures people choose to adopt more than they are causes of them. I suspect most are a complicated mixture of both. Still, even in those cases, the availability of who communicates with whom must play a role in fostering different ideas and getting relevant information to those who could benefit from it.

Some forms of diversity are surely better than others. For example, at least in simulations, it's been shown that a core of just 10% of diehard fanatics are enough to convince the rest, regardless of the network structure. So combine 10% who believe abortion is a mortal sin, 10% who think it's an essential right, and 80% open minded, and watch the fireworks. I'm quite confident you'll want to be at a safe distance.

Perhaps the diversity required is subtler than that. Maybe it's not about the particular issue at all, but a broader set of worldviews and different ways of approaching problems. That way you try and minimise the destructive fanaticism. You do want people who are going to stand up and disagree with the majority, but I can't think of any situations in which it makes any sense at all to disagree with the evidence.

(Caveat : irrational viewpoints sometimes get lucky. It's a bit of a dilemma as to how to incorporate metadata with direct evidence, e.g. that people have always done things a certain way tells you nothing about why they started doing that, but its persistence might suggest you should keep doing it. They may have stumbled onto a solution to a problem too complicated to fully analyse.)

EDIT : There was something nagging at me about this one, and I've realised what it was. You might think that having a bunch of specialist experts would be the best way to solve any sort of specialist technical problem, so why does diversity matter at all ? The answer, of course, is that expertise and many other attributes are uncorrelated. There's absolutely no reason* why someone of an Islamic, Christian, poor, black, white, lesbian, etc. etc. background can't become proficient in just about anything. Background by itself doesn't determine expertise or knowledge, but it does influence the way you think and give you access to different knowledge. Sometimes apparently unrelated background knowledge turns out to be more important than the professional training.

* At least in principle, leaving aside a host of complex cultural and other correlated factors.

EDIT 2 : In one of Lindybeige's excellent videos (I forget which one), he makes a throwaway comment to the effect that you wouldn't expect a female perspective on engineering to be any different than a male one. He's got a point : there's no such thing as female engineering or female physics; the Universe runs independently of gender. But this misses a more important point. Even if, for the sake of argument, male and female brains are identical, society still genderises them. Men and women and any other demographic you care to specify experience life differently, giving them different knowledge and perspectives. Thus apparently completely unrelated factors can indeed be important considerations in forming a group to tackle a specific task.

The CIA, 9/11 and collective blindness

When the Central Intelligence Agency (CIA) failed to prevent the September 11, 2001 attacks, many asked whether more could have been done. But the true reason why the agency was blind to the signs may be a diversity problem, writes Matthew Syed.