Every time I read this particular paper I find something else wrong with it. Doesn't matter which paper (maybe in a future post) except that it's a simulation of a famous hydrogen tail from a spiral galaxy. I'm keeping a detailed file on this one. The main points thus far :
- Their figures comparing the simulations and observations are set to the same horizontal but different vertical scales, making it look as though the simulations are in far better agreement with the observations than they actually are.
- The spiral galaxy is already known to be unusually gas rich, but their model would make it even richer - exceptionally so. They simply avoid any remarks on this.
- The gas in the spiral would have been significantly more extended than in reality and with a profile that doesn't match the observations. That might not be so bad except that the real galaxy's gas distribution is quite normal. Again they simply avoid noting this.
- The mass of their simulated spiral is about a factor 2 lower than the real one. They cunningly conceal this by stating only the mass (which is derived from the observations) rather than the circular velocity (which is what's actually observed) and seemingly ignore the fact that circular velocity must be corrected for inclination.
- A major structure in the spiral galaxy is not reproduced at all.
- They identify one real object which could be the cause of the interaction, state that there are others but don't state which ones.
- Very little is said quantitatively about the structure of the tail (which would not have been that difficult to do) so we only have their figures to compare with observations.
This isn't even the most determined effort to hide results that I've seen, but it's a serious contender for "using the widest variety of different ways to massage data". They aren't even small problems of detail either - potentially the main result is simply wrong. It's got 39 citations.
[Later I tested their results through a large series of numerical simulations. Their main conclusion is likely correct, in that the mechanism they present can explain certain observations in general. Whether it can explain the specific observations they discussed, though, is in my opinion very much less clear.]
Of those I've read, none.
ReplyDeleteI am infuriated by people that go around discarding data like that. Too clever to be doing it, but they do it anyway. Unfortunately, I see it so much of that in politics and in people that should know better. And that graph scale thing is either incompetence or deliberate lying in my opinion. Do they mention the change in vertical scale at all?
ReplyDeleteFred Beckhusen I'm forced to the conclusion that the paper is thoroughly strange. It's also probably wrong, but I'm running my own simulation using realistic parameters to check that. Could be it'll turn out to be correct after all, though other results lead me to think that possibility is remote. What their model actually shows, as far as I can tell, is that their mundane explanation isn't sensible for the origin of this particular weird object. They've used a galaxy with the absolutely best possible parameters that should allow the object to form, and it still doesn't work. That should be seen as an exciting result which would win them fortune and glory, but for some reason they seem determined to twist the data to show the exact opposite. They don't mention the scale on the graphs at all.
ReplyDeleteThe thing is there's an awful lot of work gone into the paper and they take a lot of trouble to describe their setup quite precisely - in enough detail for an interested reader to be able to re-create it. But when you actually try this you find that things are not really as they've presented them. It's really very odd.
Well that is ... interesting. And that is the less obviously broken of the attempts to simulate that famous hydrogen tail!
ReplyDeleteI think they're both equally broken, but one of them read the O'RLY? guide more carefully. :)
ReplyDeleteAlso worth noting that the simulation and observation figures are on different pages so it's not at all obvious that the scales are different. The mass of the spiral is a particularly fine bit of obfuscation : until I actually needed to know the circular velocity of the galaxy, I never though to check it. I just assumed their mass would be equivalent to something close to the observational circular velocity, but after correcting for inclination they're a whole 100 km/s different.
Then there's the fact that their stream isn't kinky enough, but that's a whole other issue...
Paul Carr This rant from Rhys Taylor looks like it would make a really fine interview for the Unseen Podcast.
ReplyDeleteBob Calder we don't do a lot of interviews on Unseen, but it would be great to have him as guest panelist.
ReplyDeleteWere the observations made with GIGO? :-)
ReplyDelete