THE STATE OF SCIENCE: Fraud is the exception not the rule in science, but it happens, as one recent high-profile case showed. How does this occur, and what can mathematics bring to the equation? Jon Borwein and David H. Bailey explain.
From time to time, the scientific community is rocked by cases of scientific fraud. Needless to say, such incidents do little to instill confidence in a public that’s already predisposed to be skeptical of inconvenient scientific findings, including biological evolution and human-induced global warming.
One notable case of fraud came to light in 2002 when Bell Labs researcher Hendrik Schön, once described as a “rising star” in the field of nanoelectronics, was accused of fraud by a review panel consisting of several prominent scientists, including physicist Malcom Beasley of Stanford University.
Most of the 25 papers in question were published in prestigious journals such as Science, Nature and Applied Physics Letters.
More recently, in 2008 a Science article started:
“The only two peer-reviewed scientific papers showing that electromagnetic fields (EMFs) from cell phones can cause DNA breakage are at the center of a misconduct controversy at the Medical University of Vienna (MUV). Critics had argued that the data looked too good to be real, and in May a university investigation agreed, concluding that data in both studies had been fabricated and that the papers should be retracted.”
This month, Netherlands psychologist Diederik Stapel was accused of publishing “several dozen” articles with falsified data.
Stapel’s papers were certainly provocative. One claimed that disordered environments such as littered streets make people more prone to stereotyping and discrimination. After being challenged in an “editorial expression of concern” in Science, Stapel confessed that the allegations were largely correct.
How could such frauds have happened? Firstly, scientific investigation is premised on open enquiry and treating every new result as a potential fraud is both antithetical and destructive.
In general, false findings such as the cell phone case are easier to uncover than “prettifying” — which in some cases comes from enthusiastic assistants “cleaning” the data to assist the case.
This culture is certainly cultivated by media reports in which every advance must be a “breakthrough”. In Stapel’s case, he was able to operate for so long because he was “lord of the data".
He did not make this data available for other researchers, a practice Jelte Wicherts of the University of Amsterdam termed “a violation of ethics rules established in the field”.
Counting on maths
It’s worth examining the role of mathematics in general, and statistics in particular, in the disclosure of these frauds. In the case of Stapel’s work, researchers found “anomalies" including suspiciously large experimental effects and a lack of outliers – observations that appear to deviate markedly from other members of the sample – in the data.
A lack of outliers and unlikely distributions are tell-tale signs of poorly-constructed artificial data – a situation similar to what is known in the trade as Benford’s law.
Even setting aside outright fraud, statistical sloppiness pervades some fields. This is especially true in clinical medical research and in the social sciences, where many of the researchers are poorly trained quantitatively.
In an analysis published this year by Hekte Wicherts and Marjan Bakker of the University of Amsterdam, about half of 281 psychology journal papers examined contained some statistical error, and about 15% had at least one error that would have changed the reported finding, “almost always in opposition to the authors’ hypothesis”.
Let us emphasise here – in case it’s not completely obvious – that scientific fraud is the exception, not the rule. Our cursory search of Science’s archive showed about half-a-dozen headline cases in the past ten years. Business, politics or law would not fair as well.
In any event, it’s clear that:
(a) more care needs to be taken in using statistical methods in scientific and mathematical research.
b) statistical methods can and should, to a greater extent, be used to detect fraud and manipulation of data (deliberate or not).
Perhaps the considerable attention drawn to recent incidents will lead to more rigorous analyses, and more circumspect behaviour by scientists. We shall see.
A version of this article first appeared on Math Drudge.
This is the seventh part of The State of Science. To read the other instalments, follow the links below.
- Part One: Does Australia care about science?
- Part Two: What’s a scientist – a poker or a puffin?
- Part Three: Science can seem like madness, but there’s always a method
- Part Four: Express yourself, scientists – speaking plainly isn’t beneath you
- Part Five: Science is imperfect – you can be certain of that
- Part Six: Why do people reject science? Here’s why …
- Part Eight: Get real: taking science to the next generation of Einsteins
- Part Nine: Critically important: the need for self-criticism in science
- Part Ten: Please, sirs, can we have some more? Aussie scientists need fuel, not gruel
- Part Eleven: Scientists and politicians – the same but different?
- Part Twelve: Tweed or speed … a day in the life of a modern scientist
- Part Thirteen: Selling science: the lure of the dark side
- Part Fourteen: Way off balance: science and the mainstream media
Join the conversation
Comments (11)
Tim Scanlon
(logged in via Facebook)
Maths and statistics in science are really the test of logic. But I see two problems with statistics in science: the lack of understanding of more complex methods ("black box" stats) and how to communicate the complex maths to the broader public.
I've had my own work argued with by senior scientists because they didn't understand the statistical techniques used, ironically my biometrician was a co-author. I am often amazed at the number of scientists who don't understand statistics, nor can they…
show full comment
Rod Lamberts
(Deputy Director, Australian National Centre for Public Awareness of Science at Australian National University)
I think it's always important when talking about exposing scientific misconduct to point out that it's actually good we occasionally catch people.
It's often portrayed as being seen by "the public" as an indictment of science and scientists. But it's also an endorsement that the sciences do check each other, and that ne'er-do-wells can, and do, get caught.
Neat article, by the way - thanks!
Doug Cotton
(IT Manager)
There must be a fine line between fraud and all the "minor" things found in the Climategate emails ... lucky? No, predictable that whitewash would be successful:
.
■The Climate Research Unit at East Anglia had profited to the tune of at least $20 million in “research” grants from the Team’s activities.
■The Team had tampered with the complex, bureaucratic processes of the UN’s climate panel, the IPCC, so as to exclude inconvenient scientific results from its four Assessment Reports, and to influence…
show full comment
Matthew Thredgold
Software Engineer/Secondary Teacher (logged in via email @xtra.co.nz)
Doug, You're really flogging a dead horse with the so called "climategate", and after reading about all your "research" into climate science you seem to not have understood the first bit of any of it. I would suggest that if you did write an op-ed piece for the Conversation it would fail the most rudimentary peer-review and you would be seriously laughed at. You're out of your depth and have little, or nothing to contribute. My experience has been IT Managers are managers because they can't code. Perhaps you could be a climate science lab manager too.
The PropheticKleenex
(logged in via Twitter)
Where do they find so many little men?
Doug Cotton
(IT Manager)
i would not go so far as to say that AGW theory set out to be a fraud. I'm sure there were genuine and excusable mistakes made in the early stages and people were understandably concerned about the temperature rises late last century.
No body of influence seemed to realise that they could have been due to other causes, such as mentioned below.
However, as Climategate has shown, there has been doubt in the minds of people like Judy Curry and even Trenberth himself. And the clear-cut change in…
show full comment
The PropheticKleenex
(logged in via Twitter)
Doug , I looked at your website and its clear to me you put alot of time into researching global warming.
Would you be interested in writing an article for The Conversation?
I'd like to see an alternative view to the government party line.
Doug Cotton
(IT Manager)
Of course I'd be interested, but you'd be joking regarding the possibility of acceptance, right? What I have written is the equivalent of several articles anyway.
Some are helped, more so in the medical area I guess - see http://theconversation.edu.au/safety-advisory-on-antidepressant-turned-adhd-drug-risk-of-rising-blood-pressure-and-heart-rate-4156
Michael J. I. Brown
(ARC Future Fellow and Senior Lecturer at Monash University)
An interesting commentary on science gone wrong is http://retractionwatch.wordpress.com/, which discusses papers that have been retracted recently.
The PropheticKleenex
(logged in via Twitter)
Alot the falsehoods of science i've discovered are deliberately perpetrated through compartmentalisation.
Where one branch of science has to take on trust the "facts" of another branch of science, and then draw conclusions from them.
This article of course is dancing around the recent climategate scandal.
As are all the recent articles from government apologists here.
This was a conspiracy, not contrived by the ego of one particular man, as in the example above.
But to facillitate the excuse to create a global government.
By their fruits shall ye know them.
Felix MacNeill
Felix MacNeill (logged in via email @grapevine.com.au)
How could the article be 'dancing around the recent climategate scandal'? I don't think it's possible to dance around something that doesn't exist.