You’ve probably heard a lot about the drug hydroxychloroquine. It’s been heralded by some as a definitive cure for COVID-19. I don’t want to dwell on the politics of hydroxychloroquine. Instead, I want to use the story of the French physician Didier Raoult and his hydroxychloroquine studies as a case study on causality and pharmacology. [This is part of Raptor Pharm & Tox’s Scientific Integrity Investigations and Causal Analysis Practice Areas].
So the basic question boils down to this: do Didier Raoult’s studies show that hydroxychloroquine is actually causing people who have COVID-19 to get better?
The answer is no, for a number of reasons, but the big reason really boils down to causality. And that’s where we’ll begin.
Causality
When most of us here or read the word causality, we have a pretty good idea of what it means. It’s dealing with causes of something or some type. Something is causal if it causes an effect. If I set-up a bunch of dominoes and knock over the first one, it will cause the next to fall, and so-on — sometimes it’ll even make a pretty picture if I set them up right. We know that the first domino is the cause of the rest of the dominoes falling.
But how do we know that the first domino is really the cause? No, I’m not going to get completely philosophical here, talking about metaphysical dominoes in a metaphysical plane, being metaphysically bumped and metaphysically falling.
If we witnessed the first domino pushing into the second domino, causing it to fall, and the second pushing into the third, and so on, then we know the first domino caused the rest to fall.
But, what if we didn’t see the first domino fall? What if what we found was all of the dominoes laying on the ground in some pattern? How could we state that the first domino caused the rest to fall?
In order for us to say definitively that the first domino is the root cause, we need to also be able to say that if it weren’t for the first domino falling, the rest wouldn’t have fallen. In the law, this is sometimes called the “but-for” test, as in “but for the first domino falling, the rest wouldn’t have fallen.” These are called counterfactual statements and they are the bedrock of causal analyses.
So why is the counterfactual statement so important? It really boils down to the fact that in order for us to prove that the first domino is the cause of the rest falling, it also must be true that had the first domino not fell, the rest wouldn’t have fallen. Coming back to the dominoes, let’s think about this counterfactual statement.
It could very well be that something happened that caused the second domino to fall, which set off the chain reaction. If the first domino remained standing, then clearly the first domino is not the cause.
It could also be the case that the first domino fell oddly, missing the second domino, yet the second domino still fell — this has happened to me countless times when putting together domino art (usually it’s a cat that plays some role).
Better examples of why the counterfactual are important actually come from pharmacology (study of drugs) and toxicology (study of poisons).
Sugar Pills, Counterfactuals and Causality
Sugar pills — I remember when I first heard about them. I was close to 7 years old, and I overheard my mom and dad talking about something in the news. My dad said a phrase that was new to me: “placebo effect”.
I asked my dad, “What’s a placebo?”
His response, “A sugar pill…” and I think he probably said more than that, but I was fascinated that a physician could prescribe candy to people. And I immediately wondered why my physician never prescribed me candy.
Placebos are actually kinda important in pharmacology. Sadly, they’re not always sugar pills, sometimes they’re water pills, they could be other formulations. The important thing is what a placebo lacks — the active ingredient, the drug, the thing that’s supposed to make someone feel or get better.
If we want to know if a drug actually causes an effect, we perform an experiment. If the purpose of the drug is to treat, let’s say, an infection, then clearly we need to do an experiment where some people with the infection get the drug.
Let’s say that people are on the drug for 7 days and their infection has cleared up. Does that mean that the drug worked?
Let’s start with what most people think is the definition of causality: Event A is causal for Event B if Event A occurs and thus causes Event B to occur. And you could extend this out to Event A causes Event C if Event A occurs, causing Event B to occur, causing Event C to occur — these are all examples of the causal chain definition of causality.
So, under the causal chain definition, yes indeed, the drug worked and the drug is what caused the infection to clear up. Thank you very much, now go approve my drug!
But wait, that’s not right at all. How do we know that the infection wouldn’t have cleared up had someone not taken the drug? For instance, what if I just sat at home, watching the Price is Right, drinking water and eating right (maybe some chips and sugary snacks) and I got better? I never took the drug. Isn’t that possible? Shouldn’t someone control for that in the drug trial — shouldn’t someone examine what happens to people who don’t actually get the drug?
So, what I’m saying is, for the drug to really, truly be causal in treating the infection, then this counterfactual statement must be true: if I hadn’t taken the drug, I’d still have an infection.
Think about that for a minute. If I didn’t take the drug, and my infection is gone in the same 7-day time-frame, did the drug really cause me to get better? The answer is: no.
And that is why we require placebos in randomized clinical trials to show that a drug has an effect. Also, in randomized clinical trials we require that people be randomly assigned to treatment groups. We want to minimize investigator bias as much as possible, and we want to try to ensure that every person has an equal opportunity to be assigned to treatment and placebo/control groups. In an ideal situation, the placebo would have the same inert ingredients as the drug under study (meaning, it’s the same pill except that the placebo is only missing the drug active ingredient).
So, About Those Didier Raoult Studies
I’m going to focus Raoult’s study in Travel Medicine and Infectious Disease, technically I should call it Gautret et al; however, Raoult is the senior author, and the one ultimately responsible for the study.
What they did: Gautret and colleagues treated 80 individuals who came to their hospital with verified COVID-19 infections with hydroxychloroquine and azithromycin. Then Gautret and colleagues measured what happened to these patients, including how long they stayed in the infectious disease unit (IDU).
What they found: All but 2 patients survived, and they were “rapidly discharged from IDU with a mean length of stay of five days…We believe there is urgency to evaluate the effectiveness of this potentially-life saving therapeutic strategy at a larger scale, both to treat and cure patients at an early stage before irreversible severe respiratory complications take hold and to decrease duration of carriage and avoid the spread of the disease.”
What’s missing: Gautret and colleagues didn’t use a placebo control.
What does this all really mean: Here’s the thing — they gave 80 people a drug combination of hydroxychloroquine and azithromycin. They say these people got better. But did they really?
They didn’t actually use a placebo. No one in this study got a placebo. So, can we say, “but for the hydroxychloroquine and azithromycin the patients wouldn’t have recovered”? No, we can’t say that — because Gautret and colleagues never looked at the counterfactual — they never looked at individuals who just got the placebo.
So really, is it fair for them to say that hydroxychloroquine and azithromycin is actually making people better? Is it really fair for them to say that there is promise in this treatment? Absolutely not!
In order to say that a treatment actually causes an effect you need to have the counterfactual present. Else, you can’t answer the but-for question.
Is an observational study really worth it in this case? My ethics stance on this is a clear no — in this case, where a placebo is an easy thing to do, and you are dealing with an unproven treatment, then there is no reason to perform the observational study. Keep in mind, giving a placebo is relatively straightforward, especially in a hospital that is used to performing clinical trials. I cannot actually think of too many reasons, ethically, why I would say an observational study of a drug is warranted in humans.
Are there situations where observational studies are warranted in general? Yes there are. Surveillance studies are a great example of observational studies. For instance, Phase 4 clinical trials, where a drug company is performing post-market surveillance (i.e., the drug is approved and now being used by members of the general public) — observational/surveillance studies are great for identifying potential side effects in the larger population that may be missed in preclinical studies or Phase 1, 2 or 3 clinical trials.
Observational studies are also appropriate for environmental surveillance of indigenous species for an increased rate of toxicity.
The thing to remember is that observational studies are not going to give you any causal information. Instead, observational studies are what scientists do to understand what’s going on in the world around them. Based on observational studies, scientists then perform experiments, or causal investigations, to understand what drugs are effective in combating a disease, which chemicals are driving toxicity in the environment, and how these causal factors operate within the environment and the biological system.
Remember this: when looking at scientific studies, always convert the finding into the but-for. When the statement is X caused Y, the but-for is “but for X, Y would not have occured.” If you can actually answer that question with the information given, then the authors have satisfied the first step of making a causal argument.
Raptor Pharm & Tox, Ltd Scientific Integrity Investigations and Causal Analysis
This work was supported by Raptor Pharm & Tox’s Scientific Integrity Investigations and Causal Analysis Practice Areas. Our Scientific Integrity Investigations focus on rooting out problematic science, raising awareness, and working to get it corrected or retracted if necessary.
Our Causal Analysis Practice Area is focused on using our causal analysis framework to ascertain if studies, assessments, or bodies of knowledge are sufficient to make causal arguments, draw causal conclusions, and create causal chains of evidence. We also provide study design consultations in this practice area to other scientists who want to ensure their studies are optimally designed and can be used in causal arguments.
Contact us today if you’re interested in finding out our Scientific Integrity Investigations and Causal Analysis Practice Areas can help you.
