A recent, rather flattering, article on acupuncture on this website holds a mirror to a broader problem in the world of acupuncture research. A problem that goes to the heart of the most fundamental scientific principles.
There’s no doubt that acupuncture is gaining traction on the grounds that it holds up under scientific interrogation. But does it really?
Let’s go back to basics. The scientific method involves proposing a theory based on plausible principles, and then trying to disprove it. Let’s say the theory proposes that a particular treatment is effective for a certain condition.
First, we ask whether it’s based on plausible principles. And, if so, we design studies to rule out every possible explanation for the observed effect except the explanation captured in the theory. If the effect remains after all reasonable controls have failed to remove it, we conclude that, on balance, it’s probably real.
Shaky foundations
Acupuncture is based on implausible principles. The concepts of Chi – an invisible, unmeasurable life force, flowing through meridians (unobservable pathways with no known anatomical correlation) - have both an allegorical quality and the appeal of an ancient provenance. But these aren’t held up by anything the scientific method has revealed.
Perhaps despite its implausible principles, acupuncture point combinations are based on centuries of practice and consistent observations of this effect. Very unlikely – we have previously reflected on this. Even limiting the number of needles that are inserted to four (in order to develop crude evidence that specific points are effective for specific ailments) requires an astronomical number of tests.
Let’s presume, for instance, that in China at the time when the first acupuncture tomes were generated with their 400 or so acupuncture points, the average lifespan was 50 years. In order to get all the needle combinations tested, every member of the Chinese population would have had to suffer from the same condition throughout their lifespan, and then receive eight separate treatments a year for every year of their life.
For a number of reasons, this estimate should be considered wildly conservative – and this is for just one condition. Acupuncture is recommended for an astounding range of afflictions.
Research evidence
Ensuring robust double-blinding studies of acupuncture is a tricky business. Nonetheless, good quality trials across a range of clinical conditions and outcomes, overwhelmingly show that acupuncture fails to outperform sham.
What’s more, when the studies are good, it appears that it doesn’t matter where one inserts the needles, how deeply they are inserted, whether or not they are manipulated once in situ, and, crucially, whether they are inserted at all.
So, according to scientific method, the theory has been disproved. In more classical scientific parlance, the key hypotheses that arise from the founding principles of acupuncture have been refuted.
Intriguingly, despite this the studies keep coming. A recent meta-analysis of individual patient data received a great deal of media attention as it was suggested to provide compelling evidence that acupuncture works. It found that while acupuncture was better than no treatment controls, there was a small, clinically trivial but statistically significant benefit of acupuncture over sham.
It took the power of a large sample to demonstrate that effect, and the authors conclude that it represents the active ingredient of real acupuncture. But as we have already seen, in imperfectly blinded trials, one would expect to see some small difference simply from the resulting bias.
So which interpretation is more plausible? Have the authors really fulfilled their obligation to falsify more plausible alternative explanations?
Fundamental problems
There’s a great deal of energy being devoted to unpicking the potential mechanisms of acupuncture. We’re told that acupuncture causes local tissue changes or activation differences in certain areas of the brain. We suggest that it’s hardly surprising that inserting needles into the skin causes a reaction in the tissues or a change in the brain. Indeed to not find such changes would be considered genuinely revolutionary.
These findings neither validate acupuncture nor provide a cogent mechanism for its therapeutic action. Indeed, we think this approach reflects some fundamental problems with much acupuncture research. The retrofitting of physiological mechanisms to explain a non-existent effect is scientifically upside down. Designing experiments to demonstrate that something works or hunting down an elusive mechanism for a cherished idea is not scientific.
We would suggest that if experiments that control for all possible explanations except acupuncture clearly show no benefit, but experiments that don’t control for other possible explanations clearly do show a benefit, then we should be investigating those other possible explanations rather than acupuncture.
There’s a thriving industry of acupuncture research that will no doubt continue to optimistically mine pockets of subtle or manufactured uncertainty. This is understandably human - what else would one do if one absolutely knew something was true but could not prove it?
But from a scientific viewpoint, acupuncture shouldn’t work, and comfortingly, when tested properly, it doesn’t.
The sharpest experimental probes have dismantled the principles on which acupuncture is based but have failed to puncture the balloon of naïve acceptance that floats aloft in the popular media, the public consciousness and in the acupuncture research community. A re-acquaintance with the most basic scientific principles may just be the sharp prick required.