On the poor quality of press releases, the utter lack of quality of clickbaiting, the depiction (both explicit and implicit) of scientific over-confidence (especially by science activists), why so many science activists think philosophy is useless when most real scientists think that's stupid, whether anti-science is a real thing or not, why if we insist on keeping popular scientific outreach figures we need to keep them on a much tighter leash, and all that stuff.
Which leads me on to the ugly and deformed cousin of press releases : clickbaiting. Or rather, secondary journalism where the journalist doesn't bother to check the details of the original source. "Mystery solved", "we finally know", "scientists baffled" are all common staple headlines of this so-called reporting. These are all the more damaging when you realise that most people don't even read the article, all they see is the headline.
"Scientists baffled" is not all that bad by itself. It's good to emphasise that science is about uncertainty and not knowing how the universe works, of finding things out. "Baffled", though, is a rather strong word to use so often. If we were baffled half as much as the clickbait suggests, we'd have no friggin' clue about how things work at all. And that's not right, because while science indeed doesn't know everything, it most assuredly does know some things. To claim that we're baffled by the tiniest anomaly is almost to say we've just given up. Nope, can't solve any more problems, let's let the pseudoscientists have a go instead. We're all just a bunch of incompetents.
But far, far worse than this is, "mystery solved" and lately, "we finally know". No, the mystery bloody well has not been solved, no, there's nothing "final" about the latest piece of evidence whatsoever. This is a hideous thing to say, because 99% of the time a new piece of evidence will come along and disprove the apparently solved solution. Keep doing this - keep telling people we definitely know the answer then five minutes later tell them that answer was bunk - and it inevitably leads to mistrust. I know I wouldn't trust anyone who kept insisting that they were definitely, definitely right this time even though they said that fifteen times already.
The "experts were wrong" card becomes extremely, legitimately powerful if people are told that the mystery is decisively solved when it was really nothing of the sort - and that allows them to justify any ridiculous idea they want. The danger of this should be obvious : expert opinion loses the proper weight it should be given.
Worse than this, perhaps, is the combination of "scientists baffled" and "mystery solved". It gives such a starkly black and white picture of research - either something is baffling, or it's solved and therefore interesting for five minutes until we can find something else to be baffled about
Not stressing the uncertainties is hugely damaging for public confidence in science. The, "we're really very sure about this" card is one that should be played only very rarely. What you want is to emphasise the uncertainties, not brush them under the carpet. That gives the, "everyone agrees about this" card much, much more strength when you really need to play it. There's nothing wrong with an individual scientist saying, "I think this because..." but there are a hell of a lot of problems with them saying, "I know this is true because blah blah and all of my other colleagues are just wrong". They don't even have to say that directly, they can just imply it - as they do so, all too often in press releases - by failing to acknowledge other possible interpretations.
That's the real power of a scientific consensus, that's why it's trustworthy - precisely because most of the time scientists disagree with each other, those rare occasions when there's near-unanimous agreement should not be ignored. Yet that's precisely what's happening as "journalists" (i.e. clickbait writers) and some scientists abuse the idea of certainty for quick, attention-grabbing headlines.
Would I Lie To You ?
Michael Gove's most famous quote is undeniably, "The British people have had enough of experts". But before we get carried away with this, let's remember that this is the moron who also said, "I set my personal ambition aside in my bid to become Prime Minister".
Exhibit A:
ReplyDeletesmbc-comics.com - www.smbc-comics.com/comics/1431354950-20150511.png
and B:
ReplyDelete66.media.tumblr.com - 66.media.tumblr.com/tumblr_m4fne7giiz1r1fhpso1_1280.jpg
And of course exhibit C :
ReplyDeletehttp://www.phdcomics.com/comics/archive.php?comicid=1174
... except that the university press release is usually much, much worse.
Hi, Rhys.
ReplyDeleteYour article states ...
"And everyone knows that everyone else is doing this to them, which is why the system is (to a large but by no means perfect extent) self-correcting ... Communication to the public is different. For me as a scientist it's helpful to remember just how incredibly ignorant I am as soon as I stray from my specialist area. I can tell you with some degree of expertise what the latest thinking is about gas stripping from galaxies or the formation of long tidal tails. But stellar spectroscopy ? Nope, never done that. Cratering studies ? Nope. Pulsar timing ? Nope. High redshift galaxies, even ? Nope. But when I listen, for interests' sake, to experts in these areas describing their findings, I expect them to be fundamentally truthful. I expect them to tell me what they all agree on and which bits are controversial. There's no way I can judge these for myself, I don't have the years of experience needed."
I feel that you're dancing around the periphery of a significant issue which has been more elegantly stated by Bruce Charlton in his book Not Even Trying: The Corruption of Real Science (2012). There are many counterpoints I could lodge to statements made in your article, but perhaps this one stands out:
"[M]icro-specialists are ultimately technicians and/or bureaucrats; thus they cannot even understand fatal objections and comprehensive refutations of their standard paradigms when these originate from adjacent areas of science. So long as their own specific technique has been conducted according to prevailing micro-specialist professional practice, they equate the outcome with 'truth' and assume its validity and intrinsic value.
In a nutshell, micro-specialization allows a situation to develop where the whole of a vast area of science is bogus knowledge; and for this reality of total bogosity to be intrinsically and permanently invisible and incomprehensible to the participants in that science.
If we then combine this situation with the prevalent professional reearch notion that only micro-specialists are competent to evaluate the domain of their micro-specialty -- and add [in] the continual fragmentation of research into ever-smaller micro-specialties -- then we have a recipe for permanent and intractable error.
Vast and exponentially-growing scientific enterprises have consumed vast resources without yielding any substantive progress at the level of in-your-face common sense evaluations; and the phenomenon continues for time-spans of whole generations, and there is no end in sight (short of the collapse of science-as-a-whole).
According to the analysis of classical science, science was supposed to be uniquely self-correcting -- in practice, now, thanks in part to micro-specialization, it is not self-correcting at all -- except at the trivial and misleadingly reassuring level of micro-defined technical glitches and slip ups."
In other words, simply put, specialization is in tension with self-correction. And once a person understands that specialization has been going on for a very long time now, expectations that science will naturally self-correct, due to the process itself, become dampened.
Weese's Law of Explanations: "the simplicity of the explanation varies directly with the wrongness thereof."
ReplyDeleteDan Weese Just "varies" without specifying how ? This I like very much, and will use often. :)
ReplyDeleteRhys Taylor I went back and edited it to "varies directly ", haha
ReplyDeleteDan Weese Hmmm, no, you were right the first time. Definitely.
ReplyDeleteRhys Taylor I do believe you're right.
ReplyDeleteDan Weese It's an excerpt from a larger book which was chosen for the very reason that it makes a concise point which is commonly overlooked.
ReplyDeleteYour Weese's Law could be applied to anything which is common sense, and yet, many common sense claims are surely correct.
The tension between over-specialization and self-correction is one such common sense claim which should stand at least until somebody can actually rebut it.
The value of seeking out good critics in science is that it makes our own thinking more philosophical and less insulated.
Chris Reeve There is a profound difference between Elegance and Simplicity. The former is rare. The second is as common as dirt.
ReplyDeleteCommon Sense is just so much folksy foo-foo dust sprinkled over an argument supported by dodgy evidence. I dislike the term "common sense" immensely.
If only the ignorant reporters would exhibit the (entirely necessary) hubris to actually read the papers and come to these interviews armed with a list of terms they don't understand. Or better yet, they might start by asking what chain of events and questions led the scientist to pursue this particular bit of research.
When I was a young man, I came upon a three volume set of the history of physics. The first book I got through in a few months. The second took well over a year. The third was a sort trial, Chris. I had to go back to school and take another two semesters of math - and well over a year of steady plodding. I'm not immune to the charms of simplicity. After thirty-odd years of consulting, I have long since learned to cast the Old Stink Eye upon anything trying to pass itself off as Common Sense.
Dan Weese Ok, you've deconstructed my language. I'll accept that I've chosen words which you did not like.
ReplyDeleteBut, again, you've still not offered an actual rebuttal to Charlton's claim. Can you please explain why it is that self-correction is not harmed by specialization?
Chris Reeve I am unable to connect the pronoun "It" to what has been excerpted - or even from where. Which Charlton? I've gone through Rhys' links and find no reference to him. Are you talking about Bruce G Charlton's Not Even Trying: the Death of Science ?
ReplyDeleteTo a man armed with a hammer, all the world's problem look like nails: I shall speak of what I know. AI / Machine Learning / Expert Systems. Neural networks can be over-trained. In the profession, this condition is called "brittle". We see it in people, too, who have trouble accepting new evidence which might undermine their long-held positions. When a neural net gets within a rough approximation of where I want it, I stop training it: I understand my training data set isn't complete enough. No data set is, by the way.
Once I turn it loose on the Real World, for which I trained it, I then watch it operate for a while. Most of my stuff implements policy decisions. It renders "verdicts", yes, that's what they're called - to underwriters. But there's another AI, waiting on the far side, often it's not even AI, it's just some service sitting there, listening to the verdicts being accepted or rejected by the underwriters.
These underwriters gave me the policy fundamentals. But it's the same underwriters who override the verdicts, every so often. They sure liked how the network behaved when it was operating on test data - but the Real World proves to be something quite different.
Self-correction presumes the self can become aware of its incorrect postures and conclusions. Without feedback, this is impossible. I can train a network, I can periodically take its measure, back it up, restore it if necessary. But training can only go forwards, it's essentially impossible to go backwards, training a neural net. Rules-based systems likewise, once I start in with enormous exception handling routines, the verdicts become arbitrary.
Dan Weese Re: "Self-correction presumes the self can become aware of its incorrect postures and conclusions. Without feedback, this is impossible."
ReplyDeletePeople are more complex than neural nets. Change in light of evidence is part of the reason why science is considered superior to other forms of reasoning.
Specialists are not today practiced in viewing observations through competing worldviews. Science programs today test students' recall of problem set solutions (exemplars).
Problem sets assume the validity of the dominant model. The approach to modeling today is that the exemplars are extended to fit newer observations.
Yet, Rhys stated ...
"I expect them to be fundamentally truthful. I expect them to tell me what they all agree on and which bits are controversial."
From where does the expectation come that a specialist will actually understand the challenges to their exemplar approach?
Which part of their training prepares them to become experts in challenges to the dominant worldview?
The scientific community does not even systematically document challenges to their dominant theories. When I talk to geologists, for instance, they exhibit no knowledge at all of the various challenges to their claims.
The very concept of specialization implies a person who is only aware of a thin slice of a domain within a single (dominant) worldview.
So, how can self-correction be assumed to occur when we are not even generating individuals who exhibit a breadth of understanding?
Is the idea that the combination of all of the respected specialist publications will be equivalent to one broad-minded generalist?
Effective self-correction seems to assume some reservoir of generalists who can help specialist scientists to question their core assumptions.
I don't see any evidence that this is a mainstream pattern. In fact, the defining feature of modern science today is that specialist scientists should not cross domains. This is not a system which has perfected the questioning of assumptions and documentation of controversies. Honestly, we've not yet even begun those processes.
There exists much room for improvement in these areas, and we should temper any assumption that self-correction is common in science, as it stands.
Chris Reeve I question just how fundamentally different humans are from neural nets. As I said, machines are better at policy, that's why I get paid to write these little beasts. I have no patience for claims of human superiority: it's just so much Homunculus Woo, patently unscientific.
ReplyDeleteI am not a professional scientist. Rhys is an astrophysicist. As for how professional he is or how amenable he is to competing worldviews, I believe I'll defer to him in the realm of astrophysics, a knowledge domain in which he puzzles through problems of which I am only dimly aware. And I'll bet my socks he deals with competing views of galaxy formation every day.
Dr. Taylor is, in short, a specialist. And so am I. Everyone knows one or two things better than anyone else who ever lived. As a consultant, it is my job to find such people. It is the great miracle of my job, that I have never encountered a situation where such a person cannot be found.
It's been my observation, the more I learn the dumber I feel. I suspect every intelligent person feels the same way. Here's where you're completely and utterly wrong: you presume the world is made up of Hedgehogs and Foxes *, that specialists are incapable of understanding challenges to their "exemplar approaches", that expertise in one domain thereby only creates arrogant pedants.
Let's suppose Dr. Taylor encountered a challenge to some fundamental aspect of astrophysical science. Let's say it was mere quackery, someone with an imperfect understanding of spacetime who wrote a dumb paper. Does this quack deserve anything but correction? You suppose the quack will be self-correcting. Science doesn't work like that. When Einstein proposed spacetime, he also proposed a test for it. That made all the difference.
And who, precisely, composes this Scientific Community? A bunch of telegenic weasels, a few of whom are capable of intelligent conversation, reduced to spouting five second factoids upon the nature of the Schwarzchild Radius. I am sorry to disabuse anyone or make them fell ignorant, but the Schwarzchild Radius cannot be explained without explaining relativistic mass of light. It can't be done. And that process involves opening some cans of worms which cannot be reduced to five second factoids.
* http://www.blogs.hss.ed.ac.uk/crag/files/2016/06/the_hedgehog_and_the_fox-berlin.pdf
Dan Weese Re: "Here's where you're completely and utterly wrong: you presume ... that specialists are incapable of understanding challenges to their 'exemplar approaches', that expertise in one domain thereby only creates arrogant pedants."
ReplyDeleteI do not invent my claims. What I do is seek out the best critique I can find -- preferably, when possible, by insiders and academic whistleblowers. Like this one ...
Martín López Corredoira: Cosmologist / Astrophysicist / Philosopher / Published 50 Academic Papers, Often as Lead
The Twilight of the Scientific Age
"Creativity is blocked. It seems that the system gives the message that no ideas are needed. It seems the system, through its higher authorities, is saying that science only needs to work out the details. It is accepted that the basis of what is now known is correct, that present-day theories are more or less correct and only manpower is needed to sort out some parameters of minor importance. A Copernican revolution is totally unthinkable within the current system."
Or this ...
Science: The Glorius Entertainment (1967)
Jacques Barzun on Specialization
"It is true that no outsider, not even the closest student of a science or art, will have the same familiarity with its difficulties as the practitioner. The outsider is often wrong and sometimes unjust; for lack of perfect fluency in the speech of the guild, he can sound ridiculous even when he is right. But none of this makes him any the less necessary. Science itself would never have made headway against the theological monopoly if the advocates of science had not gone 'out of their field' to criticize philosophy and religion as amateurs. It is the very familiarity with his own shop that prevents the professional from being critical of it or contemplative about it"
Or:
How Superstition Won and Science Lost: Popularizing Science and Health in the United States (p250, 1987)
John Burnham
"All of the signs came to point in the same direction: American scientists, as a whole, were behaving like mere technicians -- competent in a narrow, technical field but without the vision or identity with a larger calling in society such as would move them to make a public profession of naturalism and skepticism ...
... One of the marks of the narrow technician was his or her unwillingness to go beyond facts -- again reflective of the cultural regard for 'information' as well as specialization. Because of specialization, complained a biologist in 1930, too many science courses perpetuated the worst of old-fashioned teaching and were still taught at the most primitive levels, that is, emphasizing the factual and authoritarian, without the enlightenment of what investigation meant ...
"... In succeeding decades, other commentators watched the population of scientists increase and their specialization intensify and remarked that the result was not only narrowness but mediocrity. In a highly fragmented, technical system, people flourished professionally who in another day would have been handicapped by insufficiency of breadth, to say nothing of their lacking the culture and calling of Victorian scientists who argued for science because it was culture ..."
In fact, well-spoken critiques of specialization are actually reasonably easy to find. I've compiled a number of quite good ones here ...
https://plus.google.com/+ChrisReeveOnlineScientificDiscourseIsBroken/posts/QtKVXCKte2C
On the topic of the problems of science journalism, there is a spectacular analysis by John Burnham of the origins of today's science journalism here in these two pieces:
Chris Reeve Yeah, yeah. Science mostly speaks in equations, not so much in words. Can't read the equation? Back to school with yez.
ReplyDelete