Commentary on NCCET statement on ivermectin

We have considered the extracts quoted below from the current National Covid Clinical Evidence Taskforce (NCCET) statement regarding the use of ivermectin in Covid-19. Our responses and commentary to these statements follow.

The current recommendation regarding ivermectin is as follows:

“Despite some early suggestions that ivermectin may provide both prophylactic and therapeutic benefit, the available research evidence does not yet provide reasonable certainty to recommend for or against the use of ivermectin.”

And a specific critique asserts:

“Some widely discussed meta-analyses of ivermectin studies (e.g. The British Ivermectin Research Development (BIRD) Group meta analysis) have significant weaknesses, for example they include a large trial which has been discredited and retracted (Elgazzar et al.). Even in these reviews, when patient populations are separated by severity and comparisons to active treatments removed, no meaningful effect is found.”

A.   Overall assertion.

The available research evidence from (i) randomised controlled trials, (ii) observational trials, (iii) clinical success of multiple unrelated clinicians in many parts of the world, (iv) the phenomenology of whole country effects with both temporal correlation to introduction of ivermectin, and the contrasting experimental control of states or other administrative divisions with differing public health policies, all point overwhelmingly to the efficacy of ivermectin in both the prevention and management of Covid-19 [1].

The phrase “reasonable certainty” is undefined and vague, and no declaration as to what level of certainty would be regarded as “reasonable” is given. It is not a “level of certainty” recognised in formal meta-analysis.

The formal review of Bryant et al. [2] found “moderate certainty” evidence which is normally considered more than sufficient for regulatory approval of existing drugs in a new indication. For example, corticosteroids have become a standard of care for inflammatory stage Covid-19 on the basis of a single RCT of dexamethasone [3], on what is generally considered as “moderate certainty” evidence. The review of Bryant et al. [2] found “moderate certainty” evidence over 24 RCTs, not just one.

The prophylaxis trials were assessed as “low certainty” but report quantitative results in prophylaxis fully consistent with much larger observational trials, some very large [4].

“Low” certainty evidence in the past has been sufficient for the inclusion of ivermectin on the WHO Essential Medicines (Children) (EMLc) List in the indication of scabies [5] where measures of effect were in fact inferior to the previously recommended drugs.

On the basis of prior decisions in Covid-19, and for ivermectin in an anti-parasitic indication, the continued hesitancy of regulatory authorities worldwide with respect to ivermectin in Covid-19 is completely anomalous.

“Reasonable” is not recognised in formal meta-analysis, according to PRISMA guidelines [6], which recognise very low, low, moderate, and high certainty, typically from appraisals of Risk of Bias in contributing studies. There is always a measure of subjectivity in such appraisals but allocation of grades and conclusions of “levels of certainty” follow strict rules.

“High” certainty evidence is rare, confined to strong effects in very large clinical trials or meta-analyses pooling several such large studies.

“Moderate” certainty evidence is generally considered extremely powerful, and more than sufficient for regulatory approval of existing medicines in new indications.

“Low” certainty evidence has led to prior regulatory approvals to meet clear clinical needs. We address subsequent critiques of [2] below, under (B).

Much of the evidence was summarised as early as November 2020 by Kory et al. and now published in their narrative review in the American Journal of Therapeutics [1] (May- June issue).

The formal systematic review and meta-analysis by Bryant et al. [2] (July-August issue of same journal) was an exercise in support of the narrative review of Kory et al. [1], but restricted by deliberate choice to Randomised Controlled Trials (RCTs) only, as conventionally considered the highest quality of medical evidence.

For example, the review protocol excluded by policy notable studies such as the ICON study [7] demonstrating strong advantage in overall mortality in a large propensity-matched retrospective study, with obvious confounders addressed, simply because the patient allocation was not randomised. The most pronounced benefits were seen in severe disease.

Similarly in prophylaxis the very large trial of Behera et al. [4] with well over 3000 participants was excluded for the same reasons, though delivering quantitative measures of Risk Reduction (for infection) very close to the meta-analysis of the RCTs.

Including high-quality observational trials was found to lead to results just as reliable as RCTs in the synthesis of Anglemyer [15]. Adding the many known observational trials to the meta-analysis of Bryant et al. [2] is likely only to strengthen the findings further.

In any serious scientific appraisal, the evidence presented by these non-randomised trials cannot be dismissed as of no account, just because they lacked certain formal constraints, being part of the experience of hard-working clinicians in stressed circumstances.

_______________________

(Authorship note: To pre-empt widespread misunderstandings, what is called “the BiRD group” or more accurately the British Ivermectin Recommendation Development panel (not “Research”) was an ad hoc panel of clinicians, researchers and other stakeholders, with international representation, convened for an “Evidence to Decision” framework event on 20 February 2021 to hear the evidence summarised in an earlier version of reference [2].

The BiRD panel published its recommendation quite separately from Bryant et al. [2]. The authors of Bryant et al. [2] comprise: two members of the steering group (who did not vote), four ordinary members of the BiRD panel (consumer representative, health economist and two active clinicians), and one professional systematic reviewer who did not take part in the BiRD panel but contributed extensively to the research.

Hence the authors of Bryant et al. [2] are not congruent with the membership of the BiRD panel, a much larger group, and include one major contributor who remains uninvolved with BiRD.)

_______________________

B.   Subsequent critiques of [2]:

Some widely discussed meta-analyses of ivermectin studies (e.g. The British Ivermectin Research Development (BIRD) Group meta analysis) have significant weaknesses, for example they include a large trial which has been discredited and retracted (Elgazzar et al.). Even in these reviews, when patient populations are separated by severity and comparisons to active treatments removed, no meaningful effect is found.

These claims are categorically false, though regularly asserted by those with an agenda driven independently of the actual evidence.

1/ The claim of “significant weakness” in [2] is confined entirely to the inclusion of the disputed trial of Elgazzar [8]. The review of [2] was exhaustive of all RCTs found at the review closure and the first anywhere to follow strict PRISMA guidelines [6]. At the time of publication of [2], there was no reason to doubt the veracity of Elgazzar [8]; indeed it would have been a protocol violation to exclude it.

It is untrue to state that the study has been “retracted”. Prof. Elgazzar has retracted nothing, asserts defamation and has intimated legal action. The server ResearchGate has withdrawn the preprint in response to a complaint, without giving Prof Elgazzar the right of reply. Whether or not the study is “discredited” remains to be determined.

Notwithstanding these uncertainties, a “Letter to the Editor” of Am. J. Therap. [9] concerning the Elgazzar dispute has been accepted for publication and should appear shortly. We show explicitly the consequences of deleting the disputed trial in the leading mortality outcome, and in prophyalxis (Elgazzar [8] contributed arms to both outcomes). Whilst the quantitative result inevitably changes, the mortality outcome remains clear, demonstrating a 49% reduction in favour of ivermectin (aRR=0.51, 95% CI 0.27 – 0.95).

Similarly, the prophylaxis outcome remains in quantitative effect virtually unchanged, and in fact slightly improved in that the point estimate for reduction in Covid-19 infection increases from 86% to 87% (aRR=0.13, 95% CI 0.08 – 0.21), with similarly tight 95% Confidence Intervals again fully consistent with the larger observational trials of ivermectin prophylaxis.

NCCET: “When patient populations are separated by severity and comparisons to active treatments removed, no meaningful effect is found.”

This assertion lacks any logic. Removing comparison to active treatments would be a pointless exercise. The pragmatic and pre-specified inclusion of “active” treatment comparators is a strength, not a weakness, of Bryant et al. [2] and would lead to under-estimation of the effect of ivermectin, not over-estimation. In other words, Bryant et al. [2] is conservative by design, against the effect of ivermectin. The fact that consistent positive effects are observed makes the results more convincing, not less.

Separation by severity has been dealt with explicitly by Neil and Fenton [10] who apply a Bayesian meta-analysis to the full set of trials in Bryant et al. [2], with an explicit separation of disease severity between “severe” and “mild-moderate”. The study of Niaee [11] was excluded because disease severity was not distinguished. A “leave one out” sensitivity analysis is performed systematically on the entire data set, including the disputed trial of Elgazzar [8]. Again the conclusions remain robust to the removal of particular studies. For some studies with known heterogeneity the results are actually improved.

Neil & Fenton [10] find for severe disease a 90.7% posterior probability that the risk ratio favours ivermectin, and for mild/moderate Covid-19 there is an 84.1% probability the risk ratio favours ivermectin. They conclude that the results support the conclusions of Bryant et al. [2] over other claims such as that of Roman et [12]. The removal of Elgazzar [8] (Niaee [11] already excluded) provides the worst reduction in evidence but still result in a Bayesian posterior probability of effective risk reduction of 77%.

Other meta-analyses have been accepted for publication [12], in spite of demonstrated reporting errors available at pre-print stage, with very similar titles to [2] but asserting the opposite conclusions. Roman et al. [12] make a limited selection ( 1173 patients over 10 trials compared to 3406 patients over 24 trials in [2] ) of the trials reviewed in [2]. The assertions in [12] commit the elementary fallacy of supposing that lack of statistically significant evidence (in their highly selective survey) is the same thing as a positive demonstration of no benefit. These claims of Roman et al. [12] were dismissed by Neil & Fenton [13], an earlier version of [10].

Similar assertions have been made by propagandists in news media [14] but are simply untrue, as demonstrated explicitly in [9].

The context where essentially all studies are referenced to placebo (or non- pharmaceutical precautions) is prophylaxis. As previously mentioned, the prophylaxis effect reported in [2] is actually slightly improved by the removal of Elgazzar [8], and consistent with large non-randomised trials of ivermectin prophylaxis. There is no question of categorising by severity in the prophylaxis context and virtually all studies are referenced against no active comparators. The reduction in infection risk by 87% cannot be said to constitute “no meaningful effect”. It is a very strong effect, achieved with ivermectin alone (or in one trial, combined with topical iota-carageenan nasal sprays).

Moreover, there has been no credible challenge to the prophylaxis results. It is not credible that ivermectin should achieve a prophylactic effect (by whatever mechanism) and fail to achieve a therapeutic effect, at least in the initial (viremic) phase of the illness.

The authors are principals of Evidence Based Medicine Consultancy Ltd., in Bath, England

References

1. Kory, P., Meduri, G.U., Varon, J., Iglesias, J., & Marik, P.E. (2021). Review of the emerging evidence demonstrating the efficacy of Ivermectin in the prophylaxis and treatment of Covid-19. J. Therapeutics, 28(3), e299-e318 DOI: 10.1097/MJT.0000000000001377
2. Bryant, A., Lawrie, T. A., Dowswell, T., Fordham, E. J., Mitchell, S., Hill, S. R. & Tham, T. C. (2021). Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines. American Journal of Therapeutics, 28, e434–e460. doi: 1097/mjt.0000000000001402
3. Horby, P., Lim, W. S., Emberson, J., Mafham, M., Bell, J., Linsell, L., et al. (2020). Effect of Dexamethasone in Hospitalized Patients with COVID-19: Preliminary Report. New England Journal of Medicine, doi: 10.1056/NEJMoa2021436
4. Behera, P., Patro, B. K., Padhy, B. M., Mohapatra, P. R., Bal, S. K., Chandanshive, P. D., et (2021). Prophylactic role of ivermectin in SARS-CoV-2 infection among healthcare workers. Research Square preprint. doi: 10.21203/rs.3.rs-208785/v1
5. Cantey, P. (2018), ‘WHO Expert Committee on the Selection and Use of Essential Medicines: Application for inclusion of ivermectin on the WHO Model List of Essential dicines (EML) and Model List of Essential Medicines for Children (EMLc) in the indication of Scabies. WHO Expert Committee Application. https://who.int/selection_medicines/committees/expert/22/applications/s6.6_ive rmectin.pdf
6. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D. et (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372. doi: 10.1136/bmj.n71 Accessed 22 July 2021.
7. Cepelowicz-Rajter, J., Sherman, M. S., Fatteh, N., Vogel, F., Sacks, J. & Rajter, J.-J. (2020). Use of Ivermectin is Associated with Lower Mortality in Hospitalized Patients with COVID-19 (ICON study). Chest, 159(1), 85-92. DOI: 1016/j.chest.2020.10.009
8. Elgazzar, A., Hany, B., Youssef, S. A., Hafez, M., Moussa, H. & Eltaweel, A. (2020). Efficacy and Safety of Ivermectin for Treatment and prophylaxis of COVID-19 Pandemic. Research Square preprint doi: 10.21203/rs.3.rs-100956/v2 Accessed 22 July
9. Bryant, , Lawrie, T. A. & Fordham, E. J. (2021). Letter to the Editor, Am. J. Therapeutics,

accepted, to appear (August 2021).

10. Neil, M. & Fenton, N. E. (2021). Bayesian Meta Analysis of Ivermectin Effectiveness in Treating Covid-19 (with sensitivity analysis to account for possibly flawed studies). Research Gate doi: 10.13140/RG.2.2.19713.58723 Accessed 10 August 2021.
11. Niaee, M. S., Namdar, P., Allami, A., Zolghadr, L., Javadi, A., Karampour, A., … Gheibi, N. (2021). Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: A randomized multi-center clinical trial. Asian Pacific Journal of Tropical Medicine, 14(6), https://doi.org/10.4103/1995-7645.318304

12. Roman, Y. M., Burela, P. A., Pasupuleti, V., Piscoya, A., Vidal, J. E. & Hernandez, A. V. (2021). Ivermectin for the treatment of COVID-19: A systematic review and meta- analysis of randomized controlled trials. Clinical Infectious Diseases, doi: 1093/cid/ciab591
13. Neil, M. & Fenton, N. E. (2021). Bayesian Meta Analysis of Ivermectin Effectiveness in Treating Covid-19 Disease. Research Gate preprint doi: 10.13140/RG.2.2.31800.88323 12 July Accessed 22 July 2021
14. Davey, M. (2021). Huge study supporting ivermectin as Covid treatment withdrawn over ethical concerns. The Guardian, 15 July. https://theguardian.com/science/2021/jul/16/huge-study-supporting-ivermectin- as-covid-treatment-withdrawn-over-ethical-concerns Accessed 22 July 2021.
15. Anglemyer, A., Horvath, H. & Bero, L. (2014). Healthcare outcomes assessed with observational study designs compared with those assessed in randomized

trials. Cochrane Database of Systematic Reviews, DOI: 10.1002/14651858.MR000034.pub2