The controversial study gave hope to the world. That was about three seconds before scientists found the trial suffered worrisome flaws.

The controversial hydroxychloroquine (HCQ) study gave hope to the world. That was about three seconds before scientists found the trial suffered flaws so grievous as to render the results meaningless. Whether the drug helped people heal faster, this study never could tell.

Reviewing the paper, one finds problems that reach such extremes it’s hard to understand how the paper got published. How could this research have seen peer review and come out — like this?

Therein lies the story. Ultimately, the study showed us one truth to a dramatic effect: Studies may be so troubled as to defy belief in their publication and still mislead the public as if the results had been pristine.

An ethics committee green-lighted the 14-day study on March 6th, 2020. If the trial began the next day, then March 20th was the earliest date by which they could have finished. Researchers presented early trial results, not yet reviewed by outsiders, for the first time on March 16th via YouTube.

Also on March 16th, the team submitted the findings to the International Journal of Antimicrobial Agents. The journal accepted by March 17th, publishing on March 20th. That leaves a 1–2-day period for the peer-review process, which may take months ordinarily.

The strangest aspect of this paper’s journey to publication may be that once results were uploaded to the preprint servers where anyone anywhere could access them, a more thorough external review appears not to have taken place.

Many of the 40,000+ COVID-19 preprint studies have been published quickly, sought review, and published formally a month or two later with the needed corrections.

Peer review means someone with expert-level knowledge has read and critiqued a paper. If you’re lucky, it will be someone who disagrees with you; they will see every weakness in your arguments. Asking someone outside the field would be like asking a classical pianist what the strange noises from your car mean. Reviewing it yourself would be like grading your homework.

If peer-review happened as the journal claims, the job was so poor as to be undetectable. That isn’t said metaphorically. The paper submitted for review shows no differences from the original.

Scientists, who began reviewing the document before it was published, compared the earliest version to the journal’s published version. The earlier draft came from a Google Drive version that had circulated among scientists before March 16th. They had this to say:

The International Society of Antimicrobial Chemotherapy stated in April 2020:

The ISAC concluded:

The concrete evidence shows that nothing changed. Was the paper given to someone uniquely unskilled in peer review? The evidence suggests not.

The Editor-in-Chief of the publishing journal is also the author of the study. The issue featuring the research paper also included an editorial penned by the study authors, including one Editor-in-Chief. The letter gives a sparkling review, making statements not reasonably made from the results of a single study, even if the drug had helped people (sic):

Had the results confirmed as the quote states, it would not have shown the drug would “become one of the simplest and easiest to treat and prevent among infectious respiratory diseases.” The study as it was, and not as the authors presented it, never had the potential to deliver firm answers. Partly, the controversy with this study comes from discussing two separate factors at once:

The final draft shows 20 patients receiving treatment, 6 less than at the beginning. Looking at the data analysis, one would assume that the missing 6 patients dropped from the trial. When people drop from a trial, we “censor” the data, which stops dropouts from influencing the results.

That is not what happened.

Three ended up in the ICU, one died, one could not tolerate the side effects from treatment, and the last person left the hospital. The authors analyzed the study as if these critical or dead patients had simply stopped taking the drug.

Effectively, all patients who conflicted with the authors’ conclusion ended up excluded from the data. Whether that was the intention is another issue entirely, but it was the result. The percentage of people who died or were hospitalized is almost exactly what one expects to see without treatment, with 15 to 20% severe cases and a 1% death rate overall.

The study bases much of its claims on results taken from the 6th day of treatment, but that was not a planned checkpoint for the study. They switched endpoints, citing the promising nature of the results. That’s a problem.

The PLoS Clinical Trials Editorial Board highlighted endpoint switching.

The clinical trial registry says they planned to check patients on Day 1, Day 4, Day 7, and Day 14. Six does not appear among the planned days. It begs what happened on day 7. The paper submitted on March 16th, so it’s reasonable to expect they include that in the preprint.

The results appeared as if a control group took part, but one author fairly openly doesn’t believe in controlled trials. Some patients assigned themselves rather than random sorting, and they received treatment at different locations. Any differences might explain why one group fared better.

The paper claims that all patients were older than 12 years, but the report appears to have several under that age. Some patients go negative before positive again. It’s unclear how this test either shows anything about the drug’s ability to help. The authors neglected to describe these aspects fully and show no other factors could explain the fantastic results.

COVID-19 patients are already at risk for acute cardiac injury, myocarditis, and cardiac arrhythmias — all heart-related afflictions.

In response, Dutch scientists detailed a litany of valid and serious concerns. “Even if a larger randomized controlled trial (RCT) showed that the combination of (hydroxy)chloroquine and azithromycin would be effective in patients with COVID-19, safety would still be an issue.” The criticism isn’t strictly American.

The toxic potential of the drug — in all things, the dose makes the poison — increases the risk of death. While it may not have mattered much in malaria patients, it matters for COVID-19 patients whose hearts may be directly infected.

According to a cardiology report from JAMA:

The question isn’t, “what have we got to lose?” It’s “How certain are we that a drug with a known ability to harm the heart will help patients infected with a virus that also increases their risk for heart problems?” The answer is, you better be damn sure.

It took less than a day for the too-good-to-be-true claims to catch the eye of elected officials and a world desperate for a cure.

The public heard that a cure was soon to come, leaving scientists trapped in the difficult position of delivering the bad news. Officials and the public may not have understood the problems in the study. This highlights the responsibility scientists have to honestly represent their work, and for politicians to defer analyzing scientific results to qualified parties.

To date, the authors have dismissed studies to the contrary as “crappy” while not applying those same design standards to their study. At least one author has a history of outright fabricating data, and insiders have claimed that fear of that author has led to silence.

The dogmatic claims from the authors, taken up by political figures, quickly polarized public opinion on a situation that even those who study the field would need time to fully understand.

Quickly, many dismissed scientists who raised valid concerns as politically motivated, but the evidence for that has yet to be supplied. On the contrary, a welling sea of evidence shows the fears were justified. One cannot dismiss that which conflicts with their chosen perspective by taking any plausible rationale and passing it off as fact. That appears to be the root of this story.

Upton Sinclair once noted:

Clinging to belief that a cure was within our grasp has much more appeal than that the study authors misled us from the bottom all the way to the top. Whatever the truth of the study’s results, it no longer mattered once media outlets ran with it. The ill-fated effort to help people understand centered on evidence, but for some, there could be no sufficient evidence to the contrary. Scientists failed to see that opinions that come to pass without evidence likely won’t be dissuaded by more of it.

Monolithic public pressure meant scientists had to repeat the studies ad nauseam. A lack of coordination and centralized response meant “the effort [was] marked by disorder and disorganization, with huge financial resources wasted.”

Whether the drug had been helpful, a single study could not and should not be used to draw that broad conclusion. There are too many potential variables, even with thorough research. For the maybe-wonder-drug hydroxychloroquine, follow-up studies boomed out of control. STAT reported in July 2020 that 1 in every 6 drug trials was chloroquine related.

That means fewer resources went to other potential therapies, a loss felt more strongly now that hydroxychloroquine appears to increase mortality and mixed results on benefits.

Evidence had little to do with what followed the media and political frenzy. We fulfilled precisely that which we knew would happen and could have avoided.

Kai Kupferschmidt, a noted science journalist and molecular biologist, wrote in a book from 2018, “New plagues will come. Whether they will lead to catastrophe, will depend mostly on what wins out in the end: empathy and innovation or ignorance and selfishness.” With that, few could disagree, but I would add that humility seems to underpin success or failure.

In 2014, the Director-General of the WHO gave a keynote speech that was nothing short of prophetic. Yet another outbreak had just shaken the heads of state who now cared to hear the messages they had long ignored, messages that might have saved their countries from the devastating loss they now faced.

The Director-General mentioned in this address the World Bank’s shocking estimation that 90% of economic losses in a pandemic come from irrational and uncoordinated efforts.

The US response to the crisis has been nothing if not irrational and uncoordinated, though we could have avoided it. Our results tell the story of our choices, as well as the predictable reasons we made them.