Dilute Science

This part of the essay starts by outlining a common criticism levelled at the most common form of homeopathy practised in the US and UK.  This calls homeopathy unscientific because:

“[…] in many homeopathic remedies, the original substance has been diluted out of molecular existence, detractors claim belief in homeopathy has no basis in science as ‘nothing cannot do something’.”

So, can apologists for homeopathy point to serious scientific work which shows that nothing can do something?  Milgrom’s approach is to cite recent claims invoking concepts from materials science and physical chemistry to suggest that:

 “[…] homeopathy’s method of remedy preparation leads to modifications in the dynamic long-range supra-molecular ordering of solvent molecules; an effect called the ‘memory of water’”^*.

Real science

Before examining the evidence Milgrom marshals, it is worth reflecting on what we should expect to see if it is really science.  This is a complex topic and there are many different descriptions of what, in practise, science is.

However, there is practically universal agreement that science is based on the formulation and testing of hypotheses.  This means that to be scientific an idea must be testable.  It also puts careful experimental practise at the core of science.

Also, as we all tend to become attached to our own ideas, even when there is evidence against them, scientific methods include strong precautions against scientists fooling themselves.  This is vital, as Feynman (1974) remarked, “The first principle is that you must not fool yourself – and you are the easiest person to fool.  So you have to be very careful about that.”

Central to this is the honest search for reasons why an idea might be wrong.  Feynman (1974) described the process as:

“[…] a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty – a kind of leaning over backwards.  For example, if you’re doing an experiment, you should report everything that you think might make it invalid – not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked – to make sure the other fellow can tell they have been eliminated.”

This includes testing hypotheses in ways that might break them.  It also means carefully comparing new ideas and results against previous work and established theory.

Scientific investigation also includes methods for limiting the influence of personal biases.  The advent of automated measurement systems means that the mistake of Blondlot  – imagining that he was seeing scintillations from non-existent N-Rays, because he was so personally invested in his ‘discovery’ – can be avoided. 

Where an experimenter or subject can subconsciously influence the result – medical trials being a good example – a real scientist takes proper precautions.  For instance, the investigators and subjects not knowing if they are receiving a new drug or a indistinguishable dummy (blinding) prevents them from being tempted to see ‘expected’ improvement where there is none.   Making sure that if two groups are being compared they are as similar as possible, through assigning participants to them at random (randomization) is another precaution against being misled. 

Where measurements can vary because of factors outside of the scientist’s control, repeated measurements are made and statistically tested to see if two samples, for instance, are really different or if a measured difference is just happenstance. 

Finally, the use of controls – like making the same measurements on pure solvent from the same bottle used to make test solutions, for example – helps ensure that scientists are not fooled by the vagaries of the real world. 

Many experiments produce anomalous results.  In real science they are carefully examined in well-controlled experiments.  Detailed measurements are made and possible causes of error are progressively excluded.  Anomalies that survive this scrutiny may well go on to challenge current theories, but most melt away.  So real scientific investigators don’t leap to wild conclusions; they carefully seek the simplest explanation. 

So if Milgrom is really offering up examples of scientific investigations that support the basic plausibility of homeopathy, they will demonstrate these basic scientific virtues.  I expect to see careful, well documented experiments; possible flaws pointed out and explored; evidence of blinding and randomization, where appropriate; the presentation of statistical data where the experimental system produces varying results.  I also would not expect to see wild interpretations made of anomalous results; particularly if simpler explanations are possible. 

Homeopathic science

The evidence Milgrom provides on the topic of materials science relies on two publications involving Rustum Roy, an eminent materials scientist with an enviable publications record.  The only problem is that latterly he appears to have hitched his wagon to the alternative health movement.  This seems to have compromised his objectivity.

In Roy et al. (2005) we see a review of the many amazing and occasionally anomalous properties of water.  This work argues that the physical properties of water support the idea that homeopathic preparations can ‘remember’ what solute was originally added and diluted out of existence and, through structural changes, communicate this to patients. 

The problem is that the gap between what can be measured and that which is merely asserted to justify homeopathy is never closed.  The exercise is a lengthy non sequitur:  just because water has some anomalous properties doesn’t mean that it can remember what used to be dissolved in it. 

In reality, this paper is a collection of straws that are desperately clutched at.  Nowhere is this clearer than in the discussion of the potential for contamination in experiments purporting to study high-dilution remedies: 

“Obviously chemical contamination from the container material could itself serve as a “remedy”.

This is both desperate and not obvious.  If contamination from containers could be the homeopaths actual remedies, then the remedies are uncontrolled and bear no resemblance to their claimed ingredients, or their supposed therapeutic effects.  The ‘remedies’ will be different each time and will vary between homeopaths.  A medical practise based on accidental contamination cannot be in any sense plausible, let alone ’scientific’.

The biggest problem with Roy et al. (2005) is that it confuses real measurable physical phenomena (electric and magnetic fields, for instance) with the immeasurable “subtle energies” of the CAM practitioner.  This beggars any claim to be a real scientific publication.   Citing silly papers that claim to be able to show the measurable effect of “human intention” and “qi” on chemical systems provides the final nail in the coffin.  This is not science.  It may have the appearance of science, but it lacks proper content.  A real scientific paper would critically examine paranormal claims, not just accept them at face value.

So, how is it that it appears to have been published in a scientific journal?  Actually, it’s not.  It’s published in Materials Research Innovations , Rustum Roy’s own journal; a publication that rejects peer review of papers in favour of reviewing the authors.  One consequence appears to be that if you have published some good work in the past, as Roy has, then you can publish any old nonsense in the future; as Roy and his co-workers demonstrate.

Next, the essay references Rao et al (2007).  This deeply flawed paper, published in the homeopathic vanity press, which claims to show that homeopathy is plausible because they came up with some spectrographic measurements that appeared to show differences between homeopathic remedies and their solvents.

As Kerr et al (2008) pointed out: the spectrum contained in the paper purporting to be ‘pure ethanol’ does not look like ethanol of any recognised degree of purity.  Further, from the paper, there is no way to know whether the reported differences between the spectra were the result of using solvent containing different levels of impurities.

Worryingly, it contains no statistical information, so no conclusions can be drawn as to whether the remedies were actually different. Finally, one graph was reproduced twice, with the authors claiming that it showed different things each time.

The author’s reply (Rao, 2008) failed to address any of these serious concerns.  Sticking your head in the sand when serious flaws are identified in your work is not doing real science.  There again, this was published in the journal Homeopathy, which is clearly not a real science journal.

Let’s get physical

So, no real science so far: just an enthusiasm for sloppy work and the paranormal.  What about the evidence that Milgrom sees coming from physical chemistry? 

Samal and Geckler (2001) is cited because this reports evidence that water molecules ‘clump’ around solutes; tending to form bigger clumps at lower solute concentrations.  How does this help the argument that there is a scientific rationale supporting the effect of solutions where the solute is highly unlikely to have survived successive dilutions?  Quite simply: it does not.  Any structures formed around solute ions will be finite in number and diluted out of solution: just as any finite solute concentration of anything will be.  At least this paper is real science; but it says something that it definitely does not support the claims of homeopathy.

The next anomaly that is cited to show the scientific plausibility of homeopathy is provided by Rey (2003).  In this study a technique called Thermoluminescence was used to study samples of frozen heavy water (D₂O); some of which were the result of diluting preparations of lithium chloride and sodium chloride beyond the point where any molecules of these salts could be expected to be found in the solution.  This process replicated a homeopathic method of ‘remedy’ preparation: successive 1:100 dilutions with vigorous mechanical shaking (succussion) at each step.

Thermoluminescence is the, “emission of light from some minerals and certain other crystalline materials” as their temperature is raised.  The energy of this emission is, “derived from electron displacements within the crystal lattice of such a substance caused by previous exposure to high-energy radiation.”  Heating the material, “enables the trapped electrons to return to their normal positions, resulting in the release of energy.”  (Encyclopædia Britannica, 2009)

This is a proper scientific technique, generally used to date archaeological artefacts and minerals.  Rey appears to be pioneering its application to the study of frozen solutions – assuming that structures in the liquid phase will be preserved by freezing. 

Although the Rey (2003) claims that, “despite their dilution beyond the Avogadro number, the emitted light was specific of the original salts dissolved initially” it provides few details for a paper advancing such a radical hypothesis.  The investigation of the ‘highly dilute’ samples (C15 LiCl, C15 NaCl and C15 D₂O) did not include an obvious control: the unsuccussed solvent (D₂O) making it impossible to separate the putative influence of the (non-existent) salt ions from changes made by shaking the samples.

Neither did Rey provide any statistical information, so there is no way of telling if the differences measured were real, or just down to chance.  As a contributor on this blog has noted:

“[…] the scientific basis of the emissions recorded – “what do these kind of readings tell us?” […] is actually pretty obscure. The technique relies on freezing the sample, irradiating it in one of a number of ways, and then watching the thermoluminescence emissions while the sample re-warms. The general message is that the emissions depend upon “structural irregularities” in the crystal lattice of the frozen sample, but the details, to repeat, are poorly understood.”

If there really are differences between the C15 ‘dilutions’ then what might be the cause?  Rather than grasping at implausible explanations, a real scientific approach eliminates the mundane.  For instance, could it be that shaking the solutions has changed their physical characteristics?  Rey (2007) has gone on to explore the possibility that vigorous shaking causes the formation of ‘nanobubbles’ in the solutions, and that these, when frozen, provide the structures associated with the thermoluminescence spectra.  The investigation looked at the spectra generated by samples “succussed” under the standard laboratory atmosphere, pure oxygen and vacuum.  The spectra appear to be different but, again, no statistical data was included to allow a reader to be sure.

Construing this work as evidence of a scientific basis for the claims of homeopaths is unwarranted.  Yes, it is science of a sort; it’s not without flaws and obvious biases.  Neither is it very convincingly reported.  However, all it provides is evidence for the presence of not very well understood structures in heavy water ice.  There is some evidence that they are associated with the shaking of the solutions, but the missing control (unsuccussed D₂O) limits the conclusions that can be drawn.  In fact, I would say that Rey’s contention that the spectra were specific to the salts goes too far, as he was not in a position to separate the implausible influence of the absent chemicals from that of the process by which his solutions were made.

One of the key values in science is that of replication.  If independent scientists, working in other places, can get the same result from the same experiment then this helps separate knowledge from happenstance.  Milgrom claims that the findings in Rey (2003) were replicated by van Wijk et al. (2006).  They were not.

This work did try and replicate and extend the findings of Rey (2003).  Using Lithium Chloride (LiCl) as the solute and heavy water as the solvent (D₂O) they also explored the possible influence of, “time between preparation of substance and time of experimentation, and […] time between irradiation and thermoluminescence recording”.

Unlike Rey (2003) van Wijk et al. (2006) presented a statistical analysis of their measurements.  Their like-for-like attempted replication failed:

“We report here differences in thermoluminescence between C15 D₂O and C15 LiCl, which correspond with the observations reported by Rey (2003).  However, the difference from all of these recordings of these substances was not statistically significant.”

In the replication experiment (A) the difference between C15 LiCl and C15 D₂O did not reach statistical significance (p = 0.059, ANOVA t-test).  In the experiments that looked at the influence of the time between sample preparation and freezing, or the time between irradiation and thermoluminescence measurement (B and C) There wasn’t even a hint of a significant difference (p = 0.72, and p = 0.63, respectively, ANOVA t-tests).

However, they did report some statistically significant differences.  When the data were processed differently the result suggested that LiCl C15 differed significantly from C15 D₂O in experiment A (p = 0.0128); but not in experiments B and C. (p = 0.60 and 0.73, respectively).

The best evidence of a difference between samples was seen in the comparison between the succussed (C15 D₂O) and unsuccussed (D₂O) solvent for experiment C.  This used the maximum time between sample preparation and freezing (12 weeks), and between irradiation and thermoluminescence measurement (3 weeks).  The result was statistically significant under both data processing methods (p<0.0001, and p<0.0004, respectively).

It’s clear that van Wijk et al. (2006) did not replicate the findings of Rey (2003), as Milgrom and, indeed, Rey (2007) has claimed.  There might be an interesting anomaly to pursue here, but there is no real evidence of absent solutes being the cause.  Van Wijk et al. (2006) does show that the experimental system is a ‘noisy’ one and that careful statistical analysis is required.

“The point must be stressed that we obtained a very good qualitative reproducibility of the thermoluminescence pattern, but the quantitative reproducibility was rather poor, and p-values should be interpreted in the sense of descriptive statistics.”

This emphasises that the lack of statistical data in Rey (2003) is a serious flaw: most of van Wijk et al.’s comparisons did not show statistically significant differences.  

The differences reported between the succussed and unsuccussed solvents also strengthen the impression that any differences are more likely the work of nanobubbles and not a watery memory of long gone solutes.  Again, this highlights Rey’s oversight in not using unsuccussed solvent as a control.  In a delightful irony even Roy et al. (2005) show they understand the potential importance of this omission:

“It is important to emphasize that the proper control solutions include not only untreated, unsuccussed solvent, but also succussed solvent without the initial addition of any remedy source materials to address possible artifacts generated by the shaking of the liquid per se within the test container itself.”

So, van Wijk et al. (2006) seems to contain science, but it doesn’t help the homeopath’s cause.

Next the essay sees scientific support in the work of Elia et al. (2006).  This is another attempt to find physically measureable differences between homeopathic solutions that do not contain any of the original solutes and their solvents.  It’s also based on very different physical principles to thermoluminescence.  This adds to the impression that this is just chasing after anomalies.  I don’t have access to this paper, so I’ll not comment further.  However, in 2007, the same author (Elia, 2007) published a review of the evidence they had accumulated. 

It’s not very impressive: no statistical data are provided to help the reader understand if any differences are significant or not.  Neither is there any indication of how many times (if at all) measurements were repeated.   It also contains an odd confession:

“It is important to emphasise that, from the studies so far conducted, we cannot derive reproducible information concerning the influence of the different degrees of homeopathic dilution or the nature of the active principle (solute) on the measured physicochemical parameters.”

If different concentrations of homeopathic preparations cannot be distinguished, then it casts serious doubt on any claims to be able to differentiate between homeopathic preparations.   This is really clutching at straws^**.

The attempt to show that homeopathy is grounded in science peters out from here.  An irrelevant theoretical speculation on Quantum Electrodynamics (QED) is thrown in to the mix (Arani et al., 1995).  Martin Chaplin’s fascinating website on the properties of water is also referenced.

Milgrom also resorts to an inappropriate analogy:

“Just as two physically contrasting substances, such as diamond and graphite, are composed of exactly the same carbon atoms arranged into different molecular structures, so it is not the composition of an ultra-diluted homeopathically-prepared solution that is different from plain diluted solvent, but its dynamic supra-molecular structure.”

Well, diamond and graphite both have structure – being solids – liquid water does not (Teixeira, 2007).  Clearly, there is no “just as” about it!  The rest is just unsupported opinion. 

The same can be said of the reference to Hankey (2004) who provides evidence and data-free hand waving of a distinctly unscientific variety.  Here is a sample:

“In this model, all vibrational medicines are quantized fluctuations, of mineral, vegetable, animal, mental, psychic, or spiritual origin. Succussion and dilution potentize the first; correct formulation of phytomedicines, the second; while the last four are all involved in various levels of healing. For example, in Maharishi Vedic Vibration Technology (Nader et al., 2001), use of a mantra develops the specific healing vibration within the technician’s nervous system, for transferal to the patient.”

The appeal to some kind of quantum theory is bogus^***.  The rest either has no meaning or is paranormal.  This is not science and by citing it Milgrom eloquently debunks his own argument.

A poor memory

And that is it^†; Milgrom presents this as a refutation of the claim that homeopathy has no scientific basis.  At best his argument rests on a few anomalous experimental results (Rey, 2003; Elia et al, 2006), which are likely to be explained by very ordinary causes: bubbles causing by shaking, chance readings in noisy experimental systems, contaminated samples, etc. 

The better work he refers to doesn’t help either.  Van Wijk et al. (2006) fails to provide the replication of Rey (2003) that Milgrom (and Rey, 2007) claims.  Samal and Geckeler (2001) does not provide a way for homeopaths to cheat Avogadro.

Much of the rest actually provides an elegant confirmation of the critics’ accusation by ignoring scientific values and asserting the reality of imaginary ‘energies’. (Roy et al., 2005; Hankey, 2004)

It is clear that Milgrom believes that water has a ‘memory’.  Unfortunately this is not just unsupported by scientific evidence; it is contradicted by it (Teixeira, 2007).  The quality of the evidence Milgrom has marshalled here bears witness to that fact.

If Milgrom really wants to turn to science, then he needs to rediscover its essential integrity:

“[…] it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in Cargo Cult Science.”  (Feynman, 1974)

Of course, this would entail leaving the Cargo Cult Science of the homeopathic apologist behind.  From the evidence on display here, I don’t think that’s very likely.

Next, I’ll have a look at what Milgrom has to say about homeopaths, the pharmaceutical industry and money.

Also in this series

A homeopathic refutation – part one – evidence.

A homeopathic refutation – part two – How deadly is homeopathy?

Disclaimer

I try to make sure that what I write is both accurate and fair.  If you think that I have got anything wrong please let me know.  If you are right I will happily change what I have written.

Notes

* See here for my summary of the “Memory of Water” issue of the pseudo-journal Homeopathy.

**Interestingly Elia et al. (2008) published a conductivity study purporting to show the effect of aging on homeopathically prepared solutions.  This was strongly criticised by Corti (2008) who asserted that: the equipment used is not capable of taking measurements of the sensitivity reported; samples were stored over time in brown glass bottles, which are known to leak conductive ions such as Fe and Ni; the shifts in conductivity over time were cyclical over the period of a year and best explained by annual shifts in temperature.  The criticisms were strongly rejected by Elia (2008).

*** See here for a demolition of Hankey’s “physics”.

†For completeness, the final reference in this part of the essay [33] is to Collins JC: Water: The Vital Force of Life. Molecular Presentations. New York, Kinderhook, 2000.   This is out of print, but Amazon carries some details. 

References

“thermoluminescence.” Encyclopædia Britannica. 2009. Encyclopædia Britannica Online. 25 Sep. 2009 Available from: http://www.britannica.com/EBchecked/topic/591643/thermoluminescence

Arani R, Bono I, Del Giudice E, Preparata G. QED COHERENCE AND THE THERMODYNAMICS OF WATER. International Journal of Modern Physics B. 1995;9(15):1813–1841. Available from: http://dx.doi.org/10.1142/S0217979295000744

Corti H. Comments on “New Physico-Chemical Properties of Extremely Dilute Solutions. A Conductivity Study at 25 °C in Relation to Ageing”. Journal of Solution Chemistry. 2008 December;37(12):1819–1824. Available from: http://dx.doi.org/10.1007/s10953-008-9335-6.

Elia V, Elia L, Cacace P, Napoli E, Niccoli M, Savarese F. ’Extremely diluted solutions’ as multi-variable systems. Journal of Thermal Analysis and Calorimetry. 2006 May;84(2):317–323. Available from: http://dx.doi.org/10.1007/s10973-005-7266-7.

Elia V, Napoli E, Germano R. The ’Memory of Water’: an almost deciphered enigma. Dissipative structures in extremely dilute aqueous solutions. Homeopathy. 2007 July;96(3):163–169. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.007.

Elia V, Napoli E, Niccoli M, Marchettini N, Tiezzi E. New Physico-Chemical Properties of Extremely Dilute Solutions. A Conductivity Study at 25°C in Relation to Ageing. Journal of Solution Chemistry. 2008 January;37(1):85–96. Available from: http://dx.doi.org/10.1007/s10953-007-9215-5.

Elia V. Response to Comments on a “New Physico-Chemical Properties of Extremely Dilute Solutions. A Conductivity Study at 25°C in Relation to Ageing” by Horacio R. Corti. Journal of Solution Chemistry. 2008 December;37(12):1825–1826. Available from: http://dx.doi.org/10.1007/s10953-008-9345-4

Feynman RP. Cargo Cult Science. Engineering and Science. 1974 June; pp. 10–13. Available from: http://calteches.library.caltech.edu/51/2/CargoCult.pdf.

Hankey A. Are We Close to a Theory of Energy Medicine?  Journal of Alternative and Complementary Medicine. 2004 February;10(1):83–86. Available from: http://dx.doi.org/10.1089/107555304322848995

Kerr M, Magrath J, Wilson P, Hebbern C. Comment on “The defining role of structure (including epitaxy) in the plausibility of homeopathy”. Homeopathy : the journal of the Faculty of Homeopathy. 2008 January;97(1). Available from: http://dx.doi.org/10.1016/j.homp.2007.10.004

Milgrom LR. Under Pressure: Homeopathy UK and Its Detractors. Forsch Komplementmed. 2009 September;16(4):256–261. Available from: http://dx.doi.org/10.1159/000228916

Rao M, Roy R, Bell I, Hoover R. The defining role of structure (including epitaxy) in the plausibility of homeopathy. Homeopathy. 2007 July;96(3):175–182. Available from: http://dx.doi.org/10.1016/j.homp.2007.03.009.

Rao M. Authors’ reply to Kerr et al. Homeopathy. 2008 January;97(1):45–46. Available from: http://dx.doi.org/10.1016/j.homp.2007.11.011

Rey L. Thermoluminescence of ultra-high dilutions of lithium chloride and sodium chloride. Physica A: Statistical Mechanics and its Applications. 2003 May;323:67–74. Available from: http://dx.doi.org/10.1016/S0378-4371(03)00047-5.

Rey L. Can low-temperature thermoluminescence cast light on the nature of ultra-high dilutions?  Homeopathy. 2007 July;96(3):170–174. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.004.

Roy R, Tiller WA, Bell I, Hoover MR. The Structure Of Liquid Water; Novel Insights From Materials Research; Potential Relevance To Homeopathy. Materials Research Innovations. 2005;9(4):577–608. Available from: http://www.rustumroy.com/Roy_Structure%20of%20Water.pdf

Samal S, Geckeler KE. Unexpected solute aggregation in water on dilution. Chemical Communications (Cambridge, England). 2001 November;(21):2224–2225. Available from: http://dx.doi.org/10.1039/b105399j.

Teixeira J. Can water possibly have a memory?  A sceptical view. Homeopathy. 2007 July;96(3):158–162. Available from: http://dx.doi.org/10.1016/j.homp.2007.05.001.

van Wijk R, Bosman S, van Wijk EP. Thermoluminescence in ultra-high dilution research. Journal of Alternative and Complementary Medicine (New York, NY). 2006 June;12(5):437–443. Available from: http://dx.doi.org/10.1089/acm.2006.12.437.

Acknowledgements

dvnutrix for pointing me at this nonsense.

DrAust for his insightful comments on thermoluminescence.

Philippe Leick for his comments on Rey (2003) and van Wijk et al. (2006)

gnu and Acleron for their comments on Elia et al. (2007) during the Homeopathy journal club run at Badscience.net. 

Edits

None yet!

[BPSDB]

How deadly is homeopathy?

Milgrom starts with a bit of distraction: “The claim that homeopathy is deadly has never been substantiated, primarily because it cannot be proved anyone has died as a direct result of taking a homeopathic remedy.” 

This is entirely irrelevant; no critical discourse that I have come across has made the claim that the remedies themselves are toxic^*.  As I pointed out in my last post: the problem is not in the pills, but in their uselessness; and the attitudes of some homeopaths.  He then moves to the actual concerns of sensible critics: 

“The claim arises over concerns that those taking homeopathic remedies might forgo ‘life-saving’ drugs. This is a false perception: many who come to homeopathy do so only after conventional treatments have failed.”

And this is not right either: those who are able to turn to homeopathy after conventional treatment has failed are not going to be suffering from life-threatening illnesses.  They are generally people who are suffering from chronic complaints for which modern medicine has no good treatments (such as certain kinds of back pain, stress, medically unexplained fatigue, and modest viral illness – Goldacre, 2007).  The danger, such as it is, lies in choosing homeopathy instead of proper medicine for serious illness. That many will be using homeopathy to treat illnesses that are not life-threatening doesn’t mean that all users (or practitioners) of homeopathy are as conservative.

Milgrom’s rejection of any suggestion that homeopathy can harm is disappointing.  There are documented cases of people choosing homeopathy, or having it chosen for them, and dying as a direct result.  The number of fatalities appears to be low, but denying that there is any problem at all is rash. 

For instance, Gloria Thomas died at nine months of age, from sepsis, after her homeopath father ‘treated’ her eczema with homeopathy instead of seeking proper medical aid.  A UK GP, Dr Marisa Viegas was eventually struck-off after a she advised a patient with idiopathic dilated cardiomyopathy to take homeopathic treatments instead of the drugs she needed.  The patient died as a result. 

An example of the dangers of the disregard that some homeopaths show for conventional medicine and evidence is seen in the untimely death of Russell Jenkins, a CAM practitioner.  He took the advice of homeopath Susan Finn, who suggested that he treat an electrical burn with Manuka honey.  As a result of this improper treatment, he died from gangrene.  This is a different form of harm, but no less dangerous. 

Add to this the documented incidents of UK homeopaths advocating homeopathy for malaria prophylaxis and the activities of homeopaths in developing countries who believe they can treat AIDS and malaria: there are real risks. 

Again, I would not want to over-state the problem; but it is inappropriate for Milgrom to ignore it. 

Other homeopathic harms

Not all the harms of homeopathy are directly deadly ones.  A risk analysis that only focuses on extreme outcomes is too simplistic: there are real harms that don’t kill.  For instance, Goldacre (2007) identified a range of other risks associated with homeopathy.  These include medicalisation, “the reinforcement of counterproductive illness behaviours, and promotion of the idea that a pill is an appropriate response to a social problem, or a modest viral illness.” 

Also, by knowingly prescribing placebos medical practitioners can undermine the notions of informed consent and patient autonomy.  

As Milgrom’s essay shows, homeopaths are apt to denigrate conventional medicine.  This attitude can also lead some homeopaths to undermine public-health campaigns, like those promoting vaccination. 

Finally, as Milgrom shows, homeopaths have a tendency to misrepresent scientific evidence, undermining the public understanding of both science and medicine. 

It is important to recognise that even placebo medicine has a range of risks associated with it. 

What are homeopaths for?

Milgrom then moves on to flirt with the placebo effect.  After making what we have seen is the unjustifiable assertion that there, “is evidence to support homeopathy is more than a placebo response.”  He notes that, “homeopaths like other health practitioners, responsibly encourage expectation of positive outcomes”.  This is fair enough.  The evidence shows that any benefit that homeopaths deliver through the therapeutic encounter is due to expectation effects (Shang et al, 2005).  If homeopaths were open about this then, perhaps, there could be a role for them in a clinical setting. 

However, there is a sizable fly in the ointment.  As Milgrom says, health practitioners, “responsibly encourage expectation of positive outcomes.”  Proper medical practitioners can deliver specific effects through their interventions, along with non-specific expectation effects.  They also have the advantage of diagnostic training and don’t disparage other medical disciplines.  Given this, who needs homeopaths? 

Neither does encouraging positive expectations reduce the risks involved with homeopaths pretending to treat malaria, AIDS or other dangerous diseases.  This line of attack has little relevance to the matter at hand. 

Wouldn’t that be NICE?

Similarly irrelevant are the author’s claims about Prozac: a particular pharmaceutical being either ineffective or unsafe doesn’t mean that homeopathy is either effective or safe.  Anyway, Milgrom’s analysis is problematic in its own right: 

“One of the world’s top-selling drugs, the anti-depressant Prozac, was recently shown to be no better than placebo [22]. Yet, with an effect size of only d ~ 0.3 (the National Institute for Health and Clinical Excellence – NICE – recommends d = 0.5 for clinical efficacy), there are no urgent calls for Prozac’s withdrawal through ‘lack of efficacy’.”

His reference [22] is to Kirsch et al. (2008) and it does not say what he claims it says.  First, this paper looks at what evidence was available before Prozac was licensed, not the totality of the data.  As Ben Goldacre has observed^**:

“It is common for quacks and journalists to think that the moment of licensing is some kind of definitive “it works” stamp of approval. It’s not, it’s just the beginning of the story of a drugs’ evidence, usually.”

So this paper does not show what the best evidence is for the efficacy, or otherwise, of Prozac (fluoxetine) for the treatment of depression.

Milgrom has also confused the result for a specific drug, fluoxetine, with a pooled analysis of all the drug groups against their placebo groups (Table 2, Model 3a).  The paper says that the drug group: 

“[…] does not meet the three-point drug–placebo criterion for clinical significance used by NICE. Represented as the standardized mean difference, d, mean change for drug groups was 1.24 and that for placebo 0.92, both of extremely large magnitude according to conventional standards. Thus, the difference between improvement in the drug groups and improvement in the placebo groups was 0.32, which falls below the 0.50 standardized mean difference criterion that NICE suggested.”

So, this is where Milgrom’s “d ~ 0.3″ comes from.  It does not relate specifically to fluoxetine, but rather to a pooled analysis for all the drugs covered in this review.  In fact, the mean difference between the drug and placebo groups, “easily attained statistical significance.” 

The paper does show that the drugs studied achieved both statistically and clinically significant improvements, compared to placebo, for the most severely depressed.  As Figure 3 shows (below), they also exceeded the NICE criterion for these patients (the green bit).

Neither does the essay contain any mention of the weaknesses of this study, or the criticisms that have been levelled at it^***.

Milgrom’s argument is further weakened by the fact that NICE have, since 2004, taken the position that:

“Antidepressants are not recommended for the initial treatment of mild depression, because the risk–benefit ratio is poor.”

So, for cases where the evidence does not support the use of drugs like Prozac, NICE recommends they are not used.

Here Milgrom overstates the scope of Kirsch et al. (2008) by implying that it is a definitive assessment of the efficacy of Prozac.  He also turns the argument into a simple binary choice: either the drug works or it doesn’t.  Reality is more complicated: the effectiveness of this (and other) drugs varies with the severity of the depression.  Whilst they may not be justifiable treatments in some cases, they are in others.

By arguing that drugs which don’t meet the NICE criteria should be withdrawn, he is also setting the bar too high for homeopathy.  It’s notable that he provides no “d” values for any single homeopathic treatment.  Prozac may not be very useful for treating all but the most severe cases of depression, but there is no evidence that homeopathy can help at all.  After reviewing the literature Pilkington et al (2005) concluded:

“Evidence for the effectiveness of homeopathy in depression is limited because of a lack of high-quality clinical trials.”

I am sure that it’s possible to argue that anti-depressants are over-prescribed and their benefits are at times over-stated.  However, their limitations appear to be appreciated by the medical community and strategies are in place to align their use with the available evidence.  There may be legitimate controversy here, but it is clear that drugs like fluoxetine have some benefit, compared to placebo: not the zero benefit Milgrom alleges.  Furthermore, Milgrom appears to have misunderstood the paper he has cited: ascribing the results for a pooled analysis of number of drugs to a single drug.  He also has missed the guidance from NICE to limit the use of anti-depressants based on their risk-benefit ratio.

This essay is meant to be making the case for homeopathy.  In this context the discussion of Prozac is irrelevant.  This section of the essay is also meant to be overturning the notion that homeopathy is dangerous; again, a flawed analysis of Kirsch et al. (2008) does not contribute to this objective.

Real medicine has risks … but are they this big?

Then again, neither does his next argument, which claims that:

“Those who denounce homeopathy as ‘deadly’ should consider conventional medicine’s safety record; something recently scrutinised by the UK’s House of Commons Public Accounts Committee [23]. Including fatalities, this committee found that in 2006 alone, at least 2.68 million people were harmed by conventional medical interventions; representing 4.5% of the UK population …”

Milgrom’s reference [23] is this report:

Leigh E: A safer place for patients: Learning to improve patient safety. 51st report of session 2005–06 report, together with formal minutes, oral, and written evidence. House of Commons papers 831, 2005–06, TSO (The Stationery Office). July 6, 2006.

It can be found here.  The first thing that stands out is that it was published in early July 2006: so it’s obvious that it cannot provide data for “2006 alone”.  This raises some suspicion about the rest of the claims.

As does the fact that I cannot find Milgrom’s figures in this report.  What Leigh (2006) examines is how patients can be treated more safely.  It looks at the incident reporting systems in the NHS, along with how it can better learn lessons when things go wrong.  It does not provide any estimates of the number of people “harmed by conventional medical interventions”.  It’s concerned with episodes of unintentional harm: medical accidents.  On this specific topic it quotes a previous report which, “estimated that one in ten patients admitted to NHS hospitals are unintentionally harmed”.   This is consistant with a recent report on patient safety from the House of Commons Health Committee (Barron, 2009).

So, if we take this rate of harm and apply it to 2006, how close do we get to Milgrom’s figures?  For the year 2005/06 the NHS Hospital Episode Statistics count 12,678,628 admission episodes.  If one in ten of these admissions resulted in harm, this implies around 1.3 million incidents of harm.  The report cites nothing which would allow us to assess the number of people harmed, or include medical accidents that may occur to patients who were not admitted to hospital (under the care of a GP or treated as an out-patient, for example). This report only provides information on harm done to hospital in-patients; and a crude estimate at that.

Terry et al(2005) also points out that injuries due to falls ranks as one of the most common causes of these incidents.  To place these events at the feet of conventional medicine, rather than the process of caring for the sick, would seem to be harsh.

Finally, it is, of course, not valid to relate this figure to the UK population as the data refers to incidents not individuals.

Either I’m missing something (always a possibility) or Milgrom’s figures must come from elsewhere.  It might be that they are correct and that this is just a mis-citation.  In any case, he should make clear where these numbers actually came from.

Ultimately this is another irrelevance.  The rate of harm resulting from conventional medicine must be set against the benefits it delivers.  We have seen that homeopathy’s magic pills and potions may offer no risk in themselves – because they don’t contain anything – but neither do they provide any benefit. 

It must also be recognised that conventional hospitals often care for desperately sick people with complex conditions: under these circumstances mistakes are more likely.  However, Milgrom’s simplistic analysis makes no attempt to do this, rendering it useless.  Further, it has no relevance to concerns about the safety of the practise of homeopathy.  It’s just a bit of tu quoque.

Ultimately, all healthcare providers should be striving to do less harm.  The rate of medical accidents within the UK’s NHS – whilst comparable to that in other developed countries – is still too high.

However, safer medical practise should also include stopping homeopaths treating serious medical conditions and the abandonment of medical interventions that incur risk without benefit. 

Critics refuted?

So has Milgrom managed to refute the charge that, “Homeopathy is deadly and those who practice it are at best purveyors of a placebo effect”?  I don’t think so.   This part of the essay is very weak.

It  steadfastly ignores the documented incidents of harm caused by homeopaths treating serious diseases.  His concept of homeopathic damage is simplistic, limited to deaths which are not acknowledged.

It may be that homeopaths, “responsibly encourage expectation of positive outcomes”.  But that is all they have.  Real medicine can offer this and more: effective treatments.

Going on the offence, Milgrom offers nothing more than two examples of ill-founded tu quoque.  First, nasty critics call homeopathy a placebo, so he says Prozac is nothing more than a placebo.  Of course, that’s not really true; Milgrom has mis-read the evidence, not acknowledged its limitations and ignored the effectiveness of anti-depressants under particular circumstances.  He also invokes NICE criteria when discussing Prozac, even though it’s clear that no homeopathic intervention could meet this standard, unlike the drug he disparages.

Similarly, critics say homeopathy is dangerous, so Milgrom says conventional medicine is too.  However, the source of Milgrom’s figures is obscure – they do not come from the report he cites.  Neither are they relevant: the failings of conventional medicine don’t make homeopathy any better.  And even with its failings conventional medicine is massively more successful than homeopathy can ever be.

All-in-all this is an empty attempt to justify an empty practise.  The scholarship is slap-dash and the arguments are flawed.  It is hard to imagine that this essay was subject to any meaningful review.

Next, I’ll look at Milgrom’s attempted refutation of “The claim that homeopathy is unscientific”.

Also in this series

A homeopathic refutation – part one – evidence.

Disclaimer

I am not a doctor.  This does not constitute medical advice.  If you need that consult a properly qualified and registered medical practitioner.

These are just my opinions, but I try to make sure that what I write is both accurate and fair.  If you think that I have got anything wrong please let me know.  If you are right I will happily change what I have written.

Notes

*Not all homeopaths use non-existent ‘medicine’.  Some of these could be toxic.

**Ben Goldacre made some interesting observations on this paper in a piece published in the Guardian, “A quick fix would stop drug firms bending the truth” (blog version)

***The Pyjamas in Bananas blog has looked at this paper in detail and provides a good reference for this controversy.  This topic is clearly more complex than the caricature provided in this essay suggests.

References

Barron K, (chairman). Patient Safety – Health Committee – Sixth Report of Session 2008-09 – Volume I: Report, Together with Formal Minutes. London: The Stationery Office Limited; 2009. Available from: http://www.publications.parliament.uk/pa/cm200809/cmselect/cmhealth/151/151i.pdf.

Goldacre B. Benefits and risks of homoeopathy. The Lancet. 2007 November;370(9600):1672–1673. Available from: http://dx.doi.org/10.1016/S0140-6736(07)61706-1.

Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration. PLoS Med. 2008 February;5(2):e45+. Available from: http://dx.doi.org/10.1371/journal.pmed.0050045.

Leigh E, (chairman). A safer place for patients: Learning to improve patient safety. 51st report of session 2005/06.  Report, together with formal minutes, oral, and written evidence. London: The Stationery Office Limited; 2006. Available from: http://www.publications.parliament.uk/pa/cm200506/cmselect/cmpubacc/831/831.pdf

Milgrom LR. Under Pressure: Homeopathy UK and Its Detractors. Forsch Komplementmed. 2009 September;16(4):256–261. Available from: http://content.karger.com/ProdukteDB/produkte.asp?Aktion=ShowAbstract&ArtikelNr=228916&Ausgabe=248719&ProduktNr=224242

Pilkington K, Kirkwood G, Rampes H, Fisher P, Richardson J.  Homeopathy for depression: a systematic review of the research evidence. Homeopathy : the journal of the Faculty of Homeopathy. 2005, July; 94(3):153–163. Available from: http://view.ncbi.nlm.nih.gov/pubmed/16060201.

Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al. Are the clinical effects of homoeopathy placebo effects?  Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005;366(9487):726–732. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67177-2.

Terry A, Mottram C, Round J, Firman E, Step J, Bourne J. A safer place for patients: learning to improve patient safety. London: National Audit Office; 2005. Available from: http://eprints.whiterose.ac.uk/3427/.

Acknowledgements

dvnutrix for pointing this nonsense out to me.

Edits

None yet!

In this post I shall examine Milgrom’s opening and his comments on the evidence for homeopathy.  I will be examining his arguments around: the scientific nature of homeopathy, its risks, the role of the profit motive and the influence of philosophy, in subsequent posts. 

Sitting comfortably? 

The summary starts with a familiar defence: “homeopathy has been in successful and continuous use for well over 200 years”.  This makes the usual mistake of conflating two different arguments: efficacy and popularity.  It is a common mistake to assume that the two go hand in hand.  History tells a different story. 

For example, medical bloodletting was both popular and ineffective for almost two-thousand years.  In fact, it was positively harmful. 

Milgrom starts to identify what he sees as the main accusations levelled against homeopathy, that it’s ‘unproven’, ‘unscientific’, and even ‘deadly’.  These seem to be a fair representation of the problems with this delusion.  The context within which he sees these attacks being made is a bit odd, “the globalised pharmaceutical industry which is itself in crisis, and a succession of UK governments seemingly supine in the face of legislation originating from the European Union.”

Of course, the pharmaceutical industry is a major bug bear of alternative ‘medicine’.  I was surprised to see the UK government and the EU attract the ire of a homeopath.

Homeopathy in crisis?

Milgrom starts the essay proper with a lament.  National Health Service (NHS) spending on homeopathic prescriptions fell by nearly 50% between 2005 and 2007.  One of the five homeopathic hospitals funded by the UK taxpayer has been earmarked for closure.  The, “flagship Royal London Homeopathic Hospital required an Early Day Motion and a debate in the House of Commons to temporarily guarantee its continued existence.”

There is a tone of entitlement in this piece: homeopathy has been available free on the NHS since its inception, so, by implication it should always be.

An easily confused homeopath

Now comes some confused whinging.  Accusations that, “homeopathic remedies are ‘deadly’, yet no better than sugar pills” are “confusing”.  It is surprising that Milgrom is so easily confused.  As the offending articles he cites make clear, the reason is simple: taking entirely ineffective sugar pills for a serious illness can be dangerous or, even lethal.  It’s really very simple: dangerous illness need real medicine.

In making this contention Milgrom fails either to understand, or fairly represent, the views of the authors he cites.   For instance, he references an article by Nick Cohen.  This is very clear as to why sugar pills can be lethal; the comments were made in the context of claims by named homeopaths to be able to treat AIDS.  Here are the opening two paragraphs:

“On 1 December, faith healers will meet at Roots & Shoots in south London to discuss how to treat Aids with magic pills. They won’t call themselves faith healers, of course, or shamans or juju men. They will present themselves as ‘homeopaths’: serious men and women whose remedies are as good as conventional medicine. 

According to the advance publicity, Hilary Fairclough, a homeopath endorsed by no less than Jeanette Winterson, will describe the ‘impressive’ results from her clinic in Botswana. Harry van der Zee, co-founder of the Amma Resonance Healing Foundation, will say that ‘in just a few days or weeks’ African Aids patients he treated became ’symptom-free and able to return to their jobs and schools or to look after their children again’. All in all, the Society of Homeopaths promises to provide ‘fascinating insights’ for World Aids Day.”

I would say that’s very easy to understand and not at all confusing.  Next Goldacre (2007) is cited.  This very balanced article is similarly clear as to where the problems lie (references omitted):

“There are also more concrete harms. A routine feature of homoeopaths’ marketing practices is to denigrate mainstream medicine. One study found that half of all homoeopaths who were approached advised patients against the measles, mumps, and rubella vaccine for their children. A television news investigation found that almost all homoeopaths who were approached recommended ineffective homoeopathic prophylaxis for malaria, undermined medical prophylaxis, and did not even give simple advice on bite prevention. Undermining medicine is a wise commercial decision for homoeopaths, because survey data show that a disappointing experience with mainstream medicine is one of the few features to regularly correlate with a decision to use alternative therapies. But it might not be a responsible choice. 

Homoeopaths can undermine public-health campaigns; leave their patients exposed to fatal diseases; and, in the extreme, miss or disregard fatal diagnoses. There have also been cases of patients who died after medically trained homoeopaths advised them to stop medical treatments for serious medical conditions.”

Again, it’s hard to see what is confusing about Goldacre’s position.  Milgrom also manages to overlook Goldacare’s discussion as to how homeopathy might be clinically useful – he’s not such a nasty sceptic after all.

Milgrom’s final reference in support of his contention that critics of homeopathy are effectively trying to ‘have their cake and eat it’ is an article by Samarasekera (2007).  As expected, the source of Milgrom’s confusion is elusive.  The piece quotes Michael Baum, making a very clear point, “People say homoeopathy cannot do any harm but when it is being promoted for HIV then there is a serious problem”.  And, after mentioning the Sense About Science exposé of homeopathic clinics and pharmacies who were willing to sell homeopathic pills to protect against malaria, David Colquhoun is quoted being equally clear, “Making false claims about treating colds is one thing but it is quite another thing to make false claims about malaria”.

One of Goldacre’s other contentions is that homeopaths have a tendency to cherry-pick and misrepresent evidence (Goldacre, 2007).  Unfortunately Milgrom falls into these traps when he starts discussing the evidence for homeopathy.

Un, deux, trois, nous irons au bois.  Quatre, cinq, six, ceuillir des cerises 

Milgrom’s first line of attack to refute the charges against homeopathy is to argue that, “Apart from several hundred years of clinical case histories, there are many good quality scientific trials and meta-analyses showing that homeopathy can demonstrate clinically observable effects over and above placebo”.

This is not a promising start.  First, case histories are low-grade evidence.  Subject to the vagaries of observer bias, expectation effects and the natural history of a disease – case histories are starting points not destinations.  Second it’s suspicious that instead of citing these “good quality […] trials and meta-analyses” Milgrom cites two cherry-picking reviews put together by advocacy organisations.

European Network of Homeopathy Researchers: An overview of positive homeopathy research and surveys, March 2007. www.homeopathy-soh.org/whats-new/documents/POSITI.PDF

Alliance for Natural Health: Homeopathy. Modality: Homeopathy. www.anhcampaign.org/practitoners/homeopathy.

What do the meta-analyses really say?  Homeopaths often refer to Kleijnen, Knipschild and Ter Riet (1991) but tend to accentuate the positive and ignore the caveats.  They concluded: 

“At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias. This indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.”

This is in no way a demonstration that homeopathy is effective.  Linde et al (1997) is another favourite of homeopaths, who often cite the first sentence of the conclusion but omit the second. 

“The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homoeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homoeopathy is clearly efficacious for any single clinical condition. Further research on homoeopathy is warranted provided it is rigorous and systematic.”

As Milgrom is contending that the evidence supports the clinical use of homeopathy, the authors’ caution against drawing such a conclusion is important: 

“Our study has no major implications for clinical practice because we found little evidence of effectiveness of any single homoeopathic approach on any single clinical condition.”

Re-analyses of the data demonstrated that this work had been overconfident about its ability to account for publication bias and assess the methodological quality of included studies.  Subsequently, many of the same authors revisited their data (Linde et al, 1999) and concluded that: 

“[…] in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results. Because summarizing disparate study features into a single score is problematic, meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome.”

Finally, Cucherat et al (2000) also showed that the evidence just does not support using homeopathy, as opposed to researching it:

“[…] sensitivity analysis showed that the P value tended towards a non-significant value (P = 0.08) as trials were excluded in a stepwise manner based on their level of quality. […]

Conclusions: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results”

So, Milgrom spectacularly fails to refute the central charge levelled at homeopathy.  His use of reviews that are cherry-picked, written by advocacy groups and exist outside of the peer-reviewed medical literature is deeply flawed.  The major meta-analyses of the literature provide no justification for using homeopathy in clinical practise.

The cherry-picking continues as the essay ventures into the area of what passes for basic homeopathic science.  He cites the discredited Nature paper on basophil degranulation.  (Davenas et al., 1988) and an alleged replication^* by Belon et al (2004).

In typical cherry-picking style he omits a failed attempt at replication published in Nature during 1993 (Hirst et al., 1993).  He also neglects to mention another failed replication by Guggisberg et al (2005).  This concluded:

“We were not able to confirm the previously reported large effects of homeopathic histamine dilutions on basophil function of the examined donor. Seemingly, minor variables of the experimental set up can lead to significant differences of the results if not properly controlled.”

This seems to get to the heart of the matter: poor experimental control.  Comments made by one of the authors of the debunked Nature paper (Beauvais, 2008) strengthen this impression:

“[...]The main issue was that in some circumstances, “effect” and “no effect” were randomly distributed regardless their origin (negative or positive samples) [...] the results of blinded samples were almost always at random and did not fit the expected results: some “controls” were active and some “active” samples were without effect on the biological system [...]“

In other words once the experimenters were not aware what the ‘expected’ result was, the results were random.  All the experimenters were measuring was bias; their own.

No homeopathic rant would be complete without an attack on Shang et al. (2005):

“This has been shown to be thoroughly biased [14–17], a view reinforced by two recent studies further demonstrating the Lancet meta-analysis as seriously flawed [18, 19]. In addition, this meta-analysis broke the Lancet’s own stringent guidelines on methodological and publication transparency [20], leading one to question why it ever appeared in such an eminent journal.”

Milgrom’s first set of  references [14 – 17] all come from the CAM literature.  As the appearance of this deeply flawed essay in a peer reviewed CAM journal shows, they don’t seem to be particularly careful in what they publish: so I’ll not waste my time on them.  The truth of the matter is that various critics had letters published in the Lancet (Fisher et al.,2006; Linde  and Jonas, 2006; Walach et al., 2006; Dantas, 2006).  The authors’ reply (Shang et al, 2006) addressed the concerns raised.  No substantial criticism remained unaddressed^**.

Homeopaths continue to make all sorts of criticisms of this work – popular myths include that it ignored key papers or that the authors never disclosed which trials made it into their final sub-group.  These are generally the result of a failure to understand^*** (or sometimes even read) the paper and the authors’ reply. (For further details see various posts^‡ on this, or Paul Wilson’s blog)

The two recent studies that Milgrom cites do not deliver what he claims.  Lüdtke and Rutten (2008) actually came to a rather mild conclusion, “Because of the high heterogeneity between the trials, Shang’s results and conclusions are less definite than had been presented.”  This is hardly evidence of deep flaws.  What they actually found was that if you make post-hoc choices about inclusion criteria and analysis methods you can get different results.  This is not surprising and is why research studies should have their analysis methods set beforehand – otherwise researchers might unconsciously influence the results of their studies by selecting methods which give them the result they want. 

A careful analysis by David Gorski shows that this paper actually confirms one of Shang’s key findings: evidence of bias was found in the sub-set of higher quality studies that was missing from their final set of eight most reliable trials.  The point of Shang’s work was to arrive at a set of least biased trials on which to make judgements of efficacy.  He concluded: 

“[…] whenever one investigator “reanalyzes” the dataset of another investigator, they virtually always have an axe to grind. That doesn’t mean it isn’t worthwhile for them to do such reanalyses or that they won’t find serious deficiencies from time to time, but you should always remember that the investigators doing the reanalysis wouldn’t bother to do it if they didn’t disagree with the conclusions and weren’t looking for chinks in the armor to blast open so that they can prove the study’s conclusions wrong. In this, Lüdtke and Rutten failed.”

Milgrom’s second paper (Rutten and Stolper, 2008) is actually a more homeopath-friendly  - therefore less objective – report of the same reanalysis: a bit of double-counting.  The excellent Paul Wilson critiqued this on his blog (here and here); this was subsequently published (Wilson, 2009).  The authors’ reply (Rutten and Stolper, 2009) signally failed to engage with his substantial criticisms. 

Shang et al (2005) has been subjected to a barrage of criticism.  Some of it is fair; most is partisan and ill-informed.  A re-analysis by homeopaths failed to show any errors.  The most that can be said is if you do it differently you can get different results.  This is an entirely trivial conclusion. 

As for Milgrom’s allegations of lack of appropriate transparency, he had a point.  The original paper did not fully disclose the identities of the trials analysed.  This should have been included in the original paper.  However, the error was put right in December 2005.  One thing that Lüdtke and Rutten (2008) showed is that it is possible to go back, reconstruct Shang’s analysis and get the same results.  Any lack of transparency, regrettable as it is, does not change what they found: 

“Biases are present in placebo-controlled trials of both homoeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.” 

Guilty as charged

Critics of homeopathy rightly point to lack of credible evidence that it has specific clinical effects.  As we have seen, this is consistent with the evidence provided by major meta-analyses.  It is also clear that the basic research offered up by homeopaths has been shown to be unreliable. 

Some sceptics have also made it very clear that homeopaths advocating the use of ineffective homeopathic treatments for serious illnesses (like AIDS and malaria) poses risks.  They have been precise about both the illnesses and homeopaths involved: the risk is not in the pills, but in their uselessness; and the attitudes to medicine promoted by some homeopaths (like discouraging vaccination). 

All Milgrom offers in this essay is a spurious smokescreen of “confusion”, lists of studies cherry-picked by advocacy organisations and an insubstantial critique of Shang et al (2005). 

It is baffling that a man of education and intelligence could think that this was any kind of serious refutation of the charges.  Equally, it is revealing that such an error-strewn analysis could make it into a peer-reviewed journal.  Then again, it’s a CAM journal. 

Next, I’ll look at Milgrom’s attempted refutation of “The claim that homeopathy is deadly”.

Also in this series

A homeopathic refutation – part two – How deadly is homeopathy?.

Disclaimer

I am not a doctor.  This does not constitute medical advice.  If you need that consult a properly qualified and registered medical practitioner.

These are my opinions, but I try to make sure that what I write is both accurate and fair.  If you think that I have got anything wrong please let me know.  If you are right I will happily change what I have written. 

Notes

*See The Great DBH Rant blog for a devastating critique of this work.

**Contrary to popular homoeomythology the authors subsequently made available a complete list of all the trials analysed in the study, including which made it into various sub-groups.

*** I have had a go at explaining what the authors actually did here.  If you are a homeopath, or apologist for homeopathy, please read this before offering any criticism.

‡In Dangerous delusions Jayney Goddard repeats the myth that “that they didn’t disclose the identities of the final eight studies.”  In Making your own reality I tried to get an article corrected in which Dana Ullman claimed that the study “did not include any of David Reilly’s research”.  In part two of this post I dealt with Ullman’s further criticisms of Shang.  In Spying on Shang I took apart a rather silly criticism by a homeopath named Clive Stuart.  Shang’s secret – the hydra of homoeomythology finds Milgrom pretending in 2008 that the authors had not disclosed the identity of the studies they analysed, although they did this in December 2005!  Homeopathy and the Absence of Evidence deals (among other things) with Dr Damien Downing’s misconceptions.  Finally, The Myth of The Secret Eight, was my first foray into the debate over this meta-analysis. 

References

Beauvais F. Memory of water and blinding. Homeopathy. 2008 January;97(1):41–42. Available from: http://dx.doi.org/10.1016/j.homp.2007.10.001.

Belon P, Cumps J, Ennis M, Mannaioni PF, Roberfroid M, Sainte-Laudy J, et al. Histamine dilutions modulate basophil activation. Inflamm Res. 2004 May;53(5):181–188. Available from: http://dx.doi.org/10.1007/s00011-003-1242-0.

Cucherat M, Haugh MC, Gooch M, Boissel JP. Evidence of clinical efficacy of homeopathy. A meta-analysis of clinical trials. HMRAG. Homeopathic Medicines Research Advisory Group. European journal of clinical pharmacology. 2000 April;56(1):27–33. Available from: http://view.ncbi.nlm.nih.gov/pubmed/10853874.

Dantas F. Are the clinical effects of homoeopathy placebo effects?  The Lancet. 2006 January;366(9503):2083. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67880-4.

Davenas E, Beauvais F, Amara J, Oberbaum M, Robinzon B, Miadonna A, et al. Human basophil degranulation triggered by very dilute antiserum against IgE. Nature. 1988 June;333(6176):816–818. Available from: http://dx.doi.org/10.1038/333816a0.

Fisher P, Berman B, Davidson J, Reilly D, Thompson T. Are the clinical effects of homoeopathy placebo effects?  The Lancet. 2006 January;366(9503):2082–2083. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67879-8.

Goldacre B. Benefits and risks of homoeopathy. The Lancet. 2007 November;370(9600):1672–1673. Available from: http://dx.doi.org/10.1016/S0140-6736(07)61706-1.

Guggisberg AG, Baumgartner SM, Tschopp CT, Heusser P. Replication study concerning the effects of homeopathic dilutions of histamine on human basophil degranulation in vitro. Complementary Therapies in Medicine. 2005 June;13(2):91–100. Available from: http://dx.doi.org/10.1016/j.ctim.2005.04.003.

Hirst SJ, Hayes NA, Burridge J, Pearce FL, Foreman JC. Human basophil degranulation is not triggered by very dilute antiserum against human IgE. Nature. 1993 December;366(6455):525–527. Available from: http://dx.doi.org/10.1038/366525a0.

Kleijnen J, Knipschild P, Ter Riet G. Clinical trials of homoeopathy. BMJ (Clinical research ed). 1991 February;302(6772):316–323. Available from: http://view.ncbi.nlm.nih.gov/pubmed/1825800.

Linde K, Jonas W. Are the clinical effects of homoeopathy placebo effects?  The Lancet. 2006 January;366(9503):2081–2082. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67878-6.

Linde K, Scholz M, Ramirez G, Clausius N, Melchart D, Jonas WB. Impact of study quality on outcome in placebo-controlled trials of homeopathy. Journal of clinical epidemiology. 1999 July;52(7):631–636. Available from: http://view.ncbi.nlm.nih.gov/pubmed/10391656.

Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, et al. Are the clinical effects of homeopathy placebo effects?  A meta-analysis of placebo-controlled trials. Lancet. 1997 September;350(9081):834–843. Available from: http://view.ncbi.nlm.nih.gov/pubmed/9310601.

Lüdtke R, Rutten AL. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. Journal of Clinical Epidemiology. 2008 December;61(12):1197–1204. Available from: http://dx.doi.org/10.1016/j.jclinepi.2008.06.015.

Milgrom LR. Under Pressure: Homeopathy UK and Its Detractors. Forsch Komplementmed. 2009 September;16(4):256–261. Available from: http://content.karger.com/ProdukteDB/produkte.asp?Aktion=ShowAbstract&ArtikelNr=228916&Ausgabe=248719&ProduktNr=224242

Rutten ALB, Stolper CF. The 2005 meta-analysis of homeopathy: the importance of post-publication data. Homeopathy : the journal of the Faculty of Homeopathy. 2008 October;97(4):169–177. Available from: http://dx.doi.org/10.1016/j.homp.2008.09.008.

Rutten ALB, Stolper CF. Reply to Wilson. Homeopathy. 2009 April;98(2):129. Available from: http://dx.doi.org/10.1016/j.homp.2009.01.002.

Samarasekera U. Pressure grows against homoeopathy in the UK. The Lancet. 2007 November;370(9600):1677–1678. Available from: http://dx.doi.org/10.1016/S0140-6736(07)61708-5.

Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al. Are the clinical effects of homoeopathy placebo effects?  Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005;366(9487):726–732. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67177-2.

Shang A, Jüni P, Sterne JAC, Huwiler-Müntener K, Egger M. Are the clinical effects of homoeopathy placebo effects?  Authors’ reply. The Lancet. 2006 January;366(9503):2083–2085. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67881-6.

Walach H, Jonas W, Lewith G. Are the clinical effects of homoeopathy placebo effects?  The Lancet. 2006 January;366(9503):2081. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67877-4.

Wilson P. Analysis of a re-analysis of a meta-analysis: in defence of Shang et al. Homeopathy : the journal of the Faculty of Homeopathy. 2009 April;98(2). Available from: http://dx.doi.org/10.1016/j.homp.2008.12.003.

Acknowledgements

dvnutrix for pointing this nonsense out to me, and the writer of The Great DBH Rant blog for their insights into Belon et al (2004).

Edits

13^th September 2009. Link to A homeopathic refutation – part two – How deadly is homeopathy? added

I asked, 

“You claim that homeopathy provides, “Remedies that will cure ulcers cold sores etc”.  Do you actually have any proper evidence to support this claim?  To be quite frank I don’t [believe] that you do, but I’m always happy to be surprised.”

I quickly received two replies from their Secretary:

“Dear Doubting Thomas,  sorry Adrian

The best evidence you will ever find is from the patients treated with homeopathic remedies.  Next time you have a patient, or better, yourself, with a mouth ulcer, you know they usually last 3 days. Buy some Feverfew and chew a harmless useless tablet and repeat two hourly. There will be your evidence.

Some unfortunate patients get bouts of mouth ulcers at regular intervals. Do as I say and the intervals between bouts gets longer until they no longer have any.

Be brave and try. You can do no harm and you might even surprise yourself. Do chew or suck as they do not work if just swallowed

Cheers

[…] ( Do keep in touch. You might even want to join BHDA)”

So, no evidence from any trials at all, let alone good ones.  All this offers is an appeal to anonymous testimonies.  Hardly the rational, scientific and dare I say professional reply that I would expect from an officer of a serious medical association.  There are all sorts of strangeness in this reply, but my favourite bit is: 

“[…] chew a harmless useless tablet […]“

Now that’s an accurate description of homeopathy.  I got a follow-up reply a little while later.  Clearly the BHDA didn’t think that their first reply was very persuasive: 

The only evidence that we have is experience, in my case 48 years of patients telling me that it works, and, of course my own experience of remedies working for me.  Mouth ulcers. Next time you have one chew Feverfew and the ulcer will go in a few hours, when normally it takes 3 days. People whe hace [sic] regular crops of ulcers, the bouts will get further apart taking Feverfew until they cease altogether.

How open is your mind to experience? 

Which covers the same ground: I should trust the alleged experience of this homeopath dentist and his patients.  It seems that he cares little about the potential for being mislead by recall bias and confirmation bias; let alone the natural history of the condition. 

Speaking of which, the UK NHS Choices website points out that minor ulcers normally heal within ten to fourteen days, whilst major ulcers, “heal more slowly, over a period of several weeks.”^** It goes on to say that most ulcers do not require specific treatment and usually heal naturally.  So the ulcers that this homeopathic dentist is talking about seem to be at the (very) minor end of the spectrum if he expects them to be gone in three days anyway. The natural history of these ulcers clearly has the potential to fool this homeopath.

Finally, as both stress and anxiety may be responsible for at least some ulcers, the placebo effect cannot be ignored either.

Given the potential influence of these factors, personal experience in the presence of belief, or the absence of proper experimental controls, is not a reliable guide. 

It’s fun to point out careless phrasing.  It’s no fun at all to see a qualified dentist with no grasp of what constitutes medical evidence and no appreciation of the limitations of his observations.   This attitude harks back to the pre-scientific practise of medicine. 

So, is there any evidence listed by reputable sources?  Searching PubMed for “feverfew AND ulcer” along with “feverfew AND mouth” returned a single reference.  Abebe (2003) provided “an overview of the utilization of herbal supplements with particular emphasis on possible interactions with oral health drugs and oral manifestations.”  It mentions feverfew in the context of the herbal treatment of gingival bleeding and swelling; so no evidence here.  Using the same searches in the Cochrane Library revealed no evidence. 

Searching the NHS Evidence website turned up nothing relevant.  There is some discussion of the use of the herb for treating arthritis, fibromyalgia and migraines.  To put this in context, a Cochrane review by Pittler and Ernst (2004) concluded that: 

“Five trials were identified that assessed the efficacy of feverfew (taken as an oral preparation) compared with placebo. Results from these trials were mixed and did not convincingly establish that feverfew is more effective than placebo for preventing migraine.”

Also, Pattrick, Heptinstall and Doherty (1989) reported a small trial (n=41) that found, “no apparent benefit from oral feverfew in rheumatoid arthritis.”  A review by Soeken, Miller and Ernst (2003) found no other trials. 

The evidence for the use of feverfew as a herbal remedy is summed up by NCCAM: 

• Some research suggests that feverfew may be helpful in preventing migraine headaches; however, results have been mixed and more evidence is needed from well-designed studies.
• One study found that feverfew did not reduce rheumatoid arthritis symptoms in women whose symptoms did not respond to conventional medicines. It has been suggested that feverfew could help those with milder symptoms.
• There is not enough evidence available to assess whether feverfew is beneficial for other uses. 

Perhaps a bit more generous on migraine prophylaxis that Pittler and Ernst (2004), but the message is clear: feverfew is of no proven use as a herbal treatment.  

So it matters not a jot whether the BHDA had in mind the sort of homeopathy that uses actual rather than imaginary doses of alleged therapeutic agents.  Even if its present in the pill there is no reason to think that it will do any good, particularly for mouth ulcers.  In fact, NCCAM cite “canker sores” (mouth ulcers) as a possible side effect from using this herb.  

Maybe this is where the BHDA got their ideas of using it as a remedy for mouth ulcers from, though it doesn’t seem that other homeopaths share their views.^*** 

For instance, the BHDA link to abcHomeopathy as a source of additional information.  This site has an on-line remedy finder and a homeopathic store.  The only mention of feverfew on the site is a comment on their forum.  This is an experience of one person taking a homeopathic preparation of feverfew for tinnitus. 

‘Googling’ “homeopathic feverfew” also reveals a short article by one Dana Ullman.  This recommends treating Headache with feverfew; though Ullman appears to be advocating herbal rather than homeopathic treatment (“Make an herbal tea of it, or simply take a feverfew capsules.”) 

It seems that feverfew as a treatment for mouth ulcers isn’t something that’s much recommended in the homeopathic world.  Anyway, I thought that, as the BHDA say, “homeopathy treats the person, rather than the disease.”  So what are they doing making a blanket recommendation for me? 

Also, here is the BHDA’s secretary – a practising homeopathic dentist – making grand claims for a treatment of a disease, not person, with no evidence.  This is a treatment that has nothing to back it up, even if the feverfew in the pills hasn’t been diluted out of existence.  This treatment doesn’t even seem to make sense in the weird world of homeopathy. 

What do I take from this brief correspondence? I’ll never trust my teeth to a homeopathic dentist and their harmless, useless tablets. 

Disclaimer

I am not a doctor (or a dentist for that matter!) This does not constitute medical advice.  If you need that consult a properly qualified and registered medical practitioner.

I try to make sure that what I write is both accurate and fair.  If you think that I have got anything wrong please let me know.  If you are right I will happily change what I have written.

Notes

*This correspondence took place between the 17^th and 26^th November 2008.

** Not well phrased.  Fourteen days is, of course, a couple of weeks.

***I know, even if other homeopaths used feverfew to treat mouth ulcers, this still would not make sense in the real world.

References

Abebe W. An overview of herbal supplement utilization with particular emphasis on possible interactions with dental drugs and oral manifestations. Journal of Dental Hygiene : JDH / American Dental Hygienists’ Association. 2003;77(1):37–46. Available from: http://view.ncbi.nlm.nih.gov/pubmed/12704968.

Pattrick M, Heptinstall S, Doherty M. Feverfew in rheumatoid arthritis: a double blind, placebo controlled study. Annals of the Rheumatic Diseases. 1989 July;48(7):547–549. Available from: http://view.ncbi.nlm.nih.gov/pubmed/2673080.

Pittler MH, Ernst E. Feverfew for preventing migraine. Cochrane Database of Systematic Reviews. 2004;(1). Available from: http://dx.doi.org/10.1002/14651858.CD002286.pub2.

Soeken KL, Miller SA, Ernst E. Herbal medicines for the treatment of rheumatoid arthritis: a systematic review. Rheumatology. 2003 May;42(5):652–659. Available from: http://dx.doi.org/10.1093/rheumatology/keg183.

Acknowledgements

Thanks to Robert Carnegie over at the Bad Science Forum for pointing out the apparent lack of support for the use of feverfew for mouth ulcers within the homeopathic world.  Without the prompt of this thread I would have left this correspondence languishing in my inbox.

Edits

None yet!