Another reason we can't have nice things

This made me go, "wwwwaaaaaaarrrghhhh ?". I might try and read the original paper because this is just properly weird. Not liking randomised testing is a bit like saying you hate facts.

In a series of surveys — conducted among the general public on paid survey-answering sites Mechanical Turk and Pollfish, and among healthcare professionals recruited through their workplace — participants were asked questions similar to the one above. Most participants agreed that it’s okay for the hospital director to print safety checklists on badges and okay for the hospital director to put a poster in each room — but 40-50% of them felt it was “inappropriate” or “very inappropriate” for the director to test which approach works better. 

The researchers asked a question very similar to the one I posed above as well as questions about “direct-to-consumer genetic testing, autonomous vehicle design, employee retirement plan enrolment nudges, recruitment of health workers in developing countries, alleviation of extreme poverty, promoting school teacher well-being, and basic income policy options.”

In all of those cases except the last one, people felt the same way. Option 1? Fine. Option 2? Fine. Random assignment between Option 1 and Option 2, for the sake of learning which works better? Not fine. 

The researchers didn’t just get these results among science-illiterate respondents, either. The researchers examined “whether educational attainment, having a science, technology, engineering, and mathematics (STEM) degree, or scientific understanding explained any of the variance in appropriateness ratings of experiments.” The results? Nope. Scientific awareness — and any other demographic factors — don’t influence whether people think that randomisation is okay.

I don't get it. Sure, maybe if your science education level is very poor, you might not understand why trials have to be random, but degree level ? That's absurd. So what the hell is it that people are objecting too ?

It presumably can't be the idea that randomisation is bad, or that experiments in general are bad, or that you have to test using large samples to avoid systematic problems. Could it be related to the fact that these are social policies, and people don't like the idea that they could be experimented on, even in a very basic and harmless way ? One would hope not, because that's akin to saying that evidence just doesn't matter at all in politics : I, for one, would welcome there to be a much greater element of evidenced-based approaches in political decisions.

The most charitable notion I can think of is that the experiments might not be disclosed, so people might think it's dishonest not to tell people when they're involved even in a harmless experiment. But even that's unsatisfying. I'm going to have to read the paper to figure out if people really are as bad as this article makes them appear.

EDIT :  I read the original paper, which is a short and easy read. It's a good paper and appears to account for possible biases, e.g. they control for how important participants think consent is in general. They examined other possible explanations : that participants should be given a choice as to which test they're subjected to, that if a single method is suggested then participants may not realise that other options are possible, that the researchers somehow know which method is more likely to succeed (thus giving half the participants deliberately inferior treatment) even though that's what the test is designed to prove, and even that people might object to experimentation in general. None of it fully accounts for the objections. It's as though everyone has different reasons, but it all conspires to produce the same strength of objection...

People are bloody mad, I tell ya.

While they note that the experimental approach and randomised trials are a relatively new thing, so they don't come naturally, they seem to me to be self-evident once someone's suggested them. Like evolution, while it takes considerable effort to realise their necessity in the first place, there's no sensible argument against them. So they're not obvious, but once their importance is realised, who could object ? About half the population, apparently.

They also note that they need more research to see if the same effect occurs in the context of other forms of experimental approaches. That would definitely be interesting, but given that this effect is present even among those with a science degree, it seems pretty frickin' weird. How in the world does half the population, even those with science degrees, escape knowing such a basic part of the scientific method ?

A shocking share of the public thinks randomized trials are immoral

Finding the best ways to do good. Made possible by The Rockefeller Foundation. Randomized trials are one of the best tools scientists have for learning about the effects of new policies. There's just one problem - the public kinda hates them.