Let us say you have 1000 boxes, and your hypothesis is that there is a valuable diamond in one of the boxes. You can open a box, look into it, and reliably determine whether there is a diamond in that box or not. Let us say you have opened two boxes, and found no diamond. The probability of this result if there is no diamond at all (the null hypothesis) is 1. The probability of this result if there is one diamond is 0.998, so the likelihood ratio is 0.998 - the 2 trials are very weak evidence that there is no diamond...
SETI researchers argue (persuasively, I think), that they haven't opened very many of the boxes yet. Perhaps one day, after a much more exhaustive tour of the search space, it will be less than 1, and we can start meaningfully wondering about the eerie silence. Until then, the search continues and no one should be too discouraged.
I agree regarding civilisations comparable or less technologically advanced than our own, but the Fermi paradox remains.
When is absence of evidence = evidence of absence?
You often hear the old mantra "absence of evidence is not evidence of absence," but I think that this is an oversimplification. The truth is, sometimes it is, and sometimes it isn't. It depends on the experiment, and also how well you understand the implication of the results.
This is a strange one. I expect it from woo peddlers, but I also hear it ("you can't prove a negative") relatively frequently from people who should know better. Anyone who has successfully completed an introduction to logic course should understand this.
ReplyDeleteWhat I find dismaying is that people confuse "proof" and "evidence".
ReplyDeleteProof is neither here nor there. It is irrelevant.
Also, the article is clearly not up to it.
ReplyDeleteAbsence of evidence is ALWAYS evidence of absence.
People who don't understand that have no understanding of what evidence is.
Maxx D The other thing I find strange is that people routinely claim you can't prove a theory (proof, overwhelming evidence, whatever you want to call it). Which of course you can, and not just in mathematics either... if you have a theory that says the reason no-one ever comes out of a room is because there's a hungry tiger in the room that eats them, and you look in the room and find a tiger, bam, theory proven.
ReplyDeleteAndreas Geisler "Absence of evidence is ALWAYS evidence of absence."
ReplyDeleteWell not really... if I go into a cave and I see a tiger but I don't find any tiger poo, the absence of tiger poo does not constitute evidence that there wasn't a tiger.
Rhys Taylor or "confirmed".
ReplyDeleteBecause all you've done is add more evidence.
Rhys Taylor If you see a tiger, there's no absence of evidence of tigers.
ReplyDeleteBut if you neither see, hear, smell or feel any tigers, nor trace of tigers, then you have an absence of evidence of tigers.
ReplyDeleteAnd usually, when I have such an absence of evidence of tigers, I effortlessly assume there are no tigers around.
Now, if I had access to a record of evidence of tigers (sightings reports for the area, for instance) my prior for "there could be tigers around" would be as high as indicated by that evidence, and the localized lack of evidence would be less important (because I have evidence that tigers can be stealthy).
The argument from ignorance (or argumentum ad ignorantiam and negative proof) is a logical fallacy that claims the truth of a premise is based on the fact that it has not been proven false, or that a premise is false because it has not been proven true. This is often phrased as "absence of evidence is not evidence of absence".
ReplyDeleterationalwiki.org - Argument from ignorance - RationalWiki
Absence of evidence is often evidence of a flawed investigator, or a difficult to assess phenomenon.
ReplyDeleteAt the extreme, if you never even look for the tiger you don't even have evidence of absence of evidence :D
ReplyDeleteNow, you absolutely could say, "Presense of negative evidence is evidence of absence"
ReplyDeleteExactly ! Not looking and consequently not finding anything means exactly nothing. Looking and not finding anything gives you a detection limit.
ReplyDelete"Absence of evidence is evidence of an underfunded science department" (true story)
ReplyDeleteMaxx D No, it is always evidence of absence.
ReplyDeleteHow strong it is, that depends on how hard you've been looking.
But funnily enough, it's possible to Prove that absence of evidence is evidence of absence:
All hypotheses of equal complexity and evidential support should have equal probability, right?
And for any given situation, there's an infinite number of possible hypotheses, most of which are unsupported by evidence. So, for any unsupported hypothesis, there's a very very large number of equally complex and equally unsupported hypotheses.
Meaning their probability will be very close to zero, just based on their not being supported by evidence (at best it will be 1/n, where n is the number of hypotheses, this being for the special case that ALL hypotheses are unsupported).
This means that their being unsupported by evidence is the reason why we set their probability to close to zero.
And that's literally what "evidence of absence" is : Reason to think something isn't the case.
Andreas Geisler Absence of evidence (not looking for a tiger) isn't the same as negative evidence (looking for a tiger but not finding any). If you haven't conducted any observations, you have no evidence at all. You can't infer absence of anything without any evidence.
ReplyDeleteBut "Not looking" doesn't change as much as you'd think.
ReplyDeleteFor that special case, you have a literally infinite number of possible hypotheses (or potentially an infinity of infinities), meaning that all hypotheses will have infinitesimal odds.
This means you don't have cause to believe anything.
So if you do believe something, despite this, then you're wrong (the odds of your choice being correct is infinitesimal, it is OK to round off that to zero).
Rhys Taylor Um, "absence of evidence" is not "We haven't looked".
ReplyDeleteThere's no such thing as "negative evidence", at most there can be evidence of contradictory hypotheses, but that's still not "negative evidence".
Anyway, just take Bayes Theorem, and put all the Es to zero and see what happens.
ReplyDeleteAndreas Geisler "Um, "absence of evidence" is not "We haven't looked"."
ReplyDeleteSure it is, by definition of "absence".
As for negative evidence, that would be when you do look for something but don't find it. That constitutes evidence that thing doesn't exist in your search region down to your detection limit.
ReplyDeleteRhys Taylor You're confusing a special case with the general.
ReplyDeleteWhat you are talking about is "The absence of any evidence at all is evidence of the absence of any given thing" (Which as I pointed out is correct)
The general case is "The absence of evidence for hypothesis X [in the present body of evidence] is evidence that hypothesis X is false" - which is also always correct.
If you have an empty, dark cave, there's no reason to think any specific thing else is in there, if there's no sign of it.
It's that simple. There could be a tiger, an elephant, a wolf, a snake, a scorpion, a christmas tree full of razors, a microscopic black hole, a dragon, etc. etc.
If you're rational, you set their probability to zero if there is no sign of them. Exactly because the possibilities are endless.
Rhys Taylor There's no such thing as "negative evidence", you're just distorting things needlessly.
ReplyDeleteAndreas Geisler I'm not trying to distort anything, just presenting my honest interpretation.
ReplyDelete"Absence" is an ambiguous word. It could mean that no search has been performed at all, or it could equally mean that a search was performed but nothing was found.
If we take it to mean the former, then clearly an absence of evidence means absolutely nothing, and the "absence of evidence is not evidence of absence" thing is entirely correct. If you never look in the cave, you have no idea what's in it so you can't assign any probabilities. You don't set the probabilities to zero, you don't set them at all.
If we take it to mean the latter, then it's different. Now we have not merely a lack of evidence, but evidence in favour of something not existing. I think it would be fair to call that "negative evidence", i.e. evidence supporting the non-existence of a particular thing within the specified search parameters. If you have a quick glance in the cave and don't see anything, you can say with some confidence that there's nothing there, but your confidence would increase the longer and more carefully you looked.
Rhys Taylor At any given time, you have a Body of Evidence.
ReplyDeleteIf in that BoE, there's no evidence for X, there is an absence of evidence for X, meaning you should not think X is the case.
But as I pointed out, the adage is always correct, even in the special case of no BoE.
Andreas Geisler
ReplyDeleteBut as I pointed out, the adage is always correct, even in the special case of no BoE.
So in your view, if you don't look into the cave, you somehow have evidence that there's nothing inside it ?
EDIT : scratch that. Think I've got myself into a right muddle.
A total absence of evidence means you should set all hypotheses to infinitesimal probability. That's the correct choice, clearly, meaning that absence of evidence is evidence of absence even in that extreme case.
ReplyDeleteAndreas Geisler Well, I simply disagree with that. Not looking at all is qualitatively different to looking but not finding anything. "There could be anything in there !" is different to "we have evidence that there's nothing in there". You don't have evidence without doing a search - that would require magic. Or not even that, because even using magic would be a form of searching.
ReplyDeleteRhys Taylor Not looking at all means "never having had any sensory input whatsoever".
ReplyDeleteDo you agree that it's irrational to posit "There are elephants" based on never having had any experience with the universe you're in?
Andreas Geisler I agree that with zero sensory input, it's irrational to posit the existence of elephants.
ReplyDeleteRhys Taylor OK, so, given that at any point we're talking about a BoE, and the presence or absence therein of evidence for a given hypothesis, can you give an example where the absence of evidence for X is not "reason not to posit X"?
ReplyDeleteThe real clincher is that we run out of resources if we try to evaluate all hypotheses. We literally won't ever get anywhere if we don't limit ourselves to what seems might be the case (i.e. hypotheses supported by at least some evidence).
Andreas Geisler Sticking with the cave analogy, cos I like analogies and get tangled up if I reduce it to generalisations, if I find a cave I don't sit around evaluating what might be inside it. If I have no evidence of what's inside, I don't have a body of evidence of any sort. So I proceed without any prejudice beyond the absolute fact that the cave must either contain something or nothing, but there's absolutely no reason to say that the cave is more likely to be empty or full of tigers. The probability simply can't be evaluated, so instead of trying to work out what's there, I just go in and have a look. Easy peasy.
ReplyDeleteIf I do have evidence of something inside, there's still no need for me to waste my finite resources evaluating every possibility. I just evaluate whatever hypotheses seem most likely. If they're disproved when I eventually go in, then so be it. No need to assign every possibility some probability value - I just don't factor them in to my assessment at all. I might be wrong, but I think it would be silly to say there's evidence against things I have no evidence for. I just ignore those possibilities until I have evidence for or against. Easy.
Rhys Taylor So, you're not going to do anything at all with the fact that you can't hear any creatures, can't smell any creatures, can't see signs of life, etc. etc?
ReplyDeleteYou're just going to go all a priori?
Andreas Geisler Yeah, pretty much. Apart from the not seeing any signs of life, which would constitute evidence. :P
ReplyDeleteNot seeing any signs of life is to notice an absence of evidence.
ReplyDeleteI mean, you can try to define the saying to be meaningless, but it will work even then.
There's literally no scenario where the best bet isn't one for which there is evidence.
Or a scenario where it's not irrational to posit something one doesn't have evidence for.
Do you see what I mean?
Never go all a priori : you'll die.
On the contrary, even if I'm a trained zoologist and spend several hours searching the exterior of the cave and find nothing, the safest, most rational course is to assume the cave might contain danger. There's no way to estimate the probability, but I can assume it's something significant and non-zero. Then, instead of wandering in singing, "la la la !", I'll proceed with extreme caution - despite having good evidence that the cave contains nothing.
ReplyDeleteOn the other extreme, if I don't know anything about the signs of animal life to look for, my search will be utterly meaningless anyway. So still no evidence, but the best course is to assume danger. Even a large animal could have wandered in a few minutes ago without leaving any signs. Snakes and spiders would be much more difficult even for an expert.
Similarly, imagine the cave is now an opening on perfectly flat, level ground. The Sun is at a low angle so I can't see in it. It would be silly to say, "I have no evidence that this is more than 50 cm deep, so I'll assume it's that deep and jump in". Far better to assume it's much deeper than that, even though there's no evidence for this.
So yes, there are lots of situations where absence of evidence doesn't mean evidence of absence, and where it's even safer to act against the evidence when available.
Rhys Taylor So, you'll also assume the cave contains radioactive materials, as well as biohazards. Full hazmat suit?
ReplyDeleteFunnily enough, the same conclusion (given the flawed heuristic of "absence of evidence != evidence of absence") is true of every taxi cab you enter.
The funny thing is, I am willing to bet that you live your life exactly as if absence of evidence is evidence of absence - because humans literally could not live without that heuristic.
But for some reason, you've been tricked into thinking this crucial heuristic is bad.
Andreas Geisler To proceed with maximum caution in every situation is foolish. But to proceed with no caution at all is foolish and dangerous. There are excellent evolutionary reasons for being overly-cautious, because you're far more likely to end up dead from being under-cautious.
ReplyDeleteI don't go around wearing a hazmat suit, but neither do I wander into caves (or indeed any and all darkened spaces, including my own home) without at least looking where I'm going and (in some situations) using a torch. I bet you do the same, precisely because you have no evidence of what's ahead. OK, you might be able to rule out dangerous animals and radioactive waste, but you can't rule out every danger.
Have you ever tried walking with your eyes shut in a wide open space ? Personally, I can't go for more than a few seconds without developing great trepidation, even though I "know" there's no danger. But the brain says I've replaced the known world for the unknown world, and the brain is very sensible. Maybe there was something as innocuous as a small pothole that I missed. "Better safe than sorry" is a perfectly sensible approach as long as you don't go to extremes.
Rhys Taylor Well, you have evidence for the cave. The cave walls and floor present a clear but controllable danger.
ReplyDeleteSo, again, if you're in a dark cave, will you:
1) Move around carefully, so as to not collide with walls, floors or stalactites?
or
2) Wish you had powered ABC armor to protect yourself from the unseen, unheard beasties that are quite possibly about to attack you?
Again, my bet is on 1. Because lots of things that are quite possible are also exceedinly unlikely.
Are you kidding ? I'd LOVE a suit of power armour ! :D
ReplyDeleteBut anyway, I already said I'd go for option 1. Because the dangers ahead are unknown. The cave might have flat walls and a soft sandy surface, but it might become rocky and sharp further in. Without actually looking or walking in, I have no way of knowing. So my approach is based on the lack of evidence for what's inside. The absence of evidence hasn't given me any hint of evidence of absence of sharp rocks up ahead, because those (unlike animals) won't leave any detectable traces at all.
ReplyDeleteRhys Taylor Yeah yeah, but that's defaulting to the body of evidence of what caves can be like (or darkened rooms, even).
ReplyDeleteThe fact that you don't worry about your lack of heavy armor is clear indication that you're reassured by the lack of signs of large animals.
Andreas Geisler I still have no evidence at all for what's actually in this cave. I know what they're generally like, but that doesn't constitute evidence that this cave is dangerous. It's only evidence that it could be dangerous.
ReplyDeleteRhys Taylor Any unfamiliar place with low visibility is a risky place to move about in. That's why you move with care.
ReplyDeleteBut you're not worried about large predators.
Which is because the lack of evidence of large predators is pretty damn good evidence that none are present.
But if you're an Indian, and you live near a rainforest known for it's stealthy man-eating tigers? Well, that reputation is EVIDENCE for invisible, unhearable predators. So when you go in that forest, you will be worried.
Yes. I move about with care because I don't know what's there. Very simple.
ReplyDeleteRhys Taylor Sigh. The animals. Are you afraid of the animals you can't see or hear?
ReplyDeleteIf not, then you're relying on the absence of evidence.
Basically, if you apply a heuristic you simultaneously decry, then you have a problem, and I can at most point you in the right direction for your solution.
In this case, the right solution is to accept that absence of evidence is always evidence of absence.
What trips people up is that they don't remember that evidence can exist without being conclusive, and that there's often evidence of things that are false.
It's not about the animals ! It's about the fact that there could be any sort of danger there. I'm afraid of the unknown because it's either a) dangerous or b) not dangerous. Yes, I'm afraid of the things I can't see. Not everything constitutes any kind of useful evidence.
ReplyDeleteI got into a cave and I'm careful because I don't know what's there. I've got no evidence of what's in the cave. This isn't complicated !
Rhys Taylor You just described NOT being afraid of the things you see no sign of.
ReplyDeleteNow you're trying to rationalize yourself back into the absurd position you've decided you (ought) to hold to.
That's doing it wrong.
Andreas Geisler Honestly, you're not making sense any more. You have your philosophy, I'll have mine, and we'll both explore caves very carefully. With a torch. And possibly a crowbar.
ReplyDeleteTry the link I gave.
ReplyDeleteWell, I might look at it later, when my brain re-crystallises.
ReplyDeleteOK, but remember, if you don't look, you'll have to assume it's super convincing :D
ReplyDeleteSo, I read it, but I'm sticking to my original perspective. Absence of evidence is sometimes evidence of absence, but not always. I just find the idea that no evidence somehow equates to having evidence to be very silly. Sorry !
ReplyDeleteRhys Taylor What about the null hypothesis? Yea or nay?
ReplyDeleteWell, the null hypothesis is a thing. Maybe it's a good thing, maybe not. Dunno. Got a deadline to meet though, so that can wait for another time.
ReplyDeleteRhys Taylor The null hypothesis is actually just the effect of "absence of evidence is evidence of absence". Meaning, the null hypothesis is not actually a thing : the reason we need evidence for hypotheses is that without evidence, they're counterindicated (by the lack of evidence).
ReplyDelete