As phase III trials of covid-19 vaccines reach their target enrolments, officials have been trying to project calm. The US coronavirus czar Anthony Fauci and the Food and Drug Administration leadership have offered public assurances that established procedures will be followed.1234 Only a “safe and effective” vaccine will be approved, they say, and nine vaccine manufacturers issued a rare joint statement pledging not to prematurely seek regulatory review.5
But what will it mean exactly when a vaccine is declared “effective”? To the public this seems fairly obvious. “The primary goal of a covid-19 vaccine is to keep people from getting very sick and dying,” a National Public Radio broadcast said bluntly.6
Peter Hotez, dean of the National School of Tropical Medicine at Baylor College of Medicine in Houston, said, “Ideally, you want an antiviral vaccine to do two things . . . first, reduce the likelihood you will get severely ill and go to the hospital, and two, prevent infection and therefore interrupt disease transmission.”7
Yet the current phase III trials are not actually set up to prove either (table 1). None of the trials currently under way are designed to detect a reduction in any serious outcome such as hospital admissions, use of intensive care, or deaths. Nor are the vaccines being studied to determine whether they can interrupt transmission of the virus.
Characteristics of ongoing phase III covid-19 vaccine trials
Evaluating mild, not severe, disease
In a September interview Medscape editor in chief Eric Topol pondered what counts as a recorded “event” in the vaccine trials. “We’re not talking about just a PCR [polymerase chain reaction test]-positive mild infection. It has to be moderate to severe illness to qualify as an event, correct?” he asked.8
“That’s right,” concurred his guest, Paul Offit, a vaccinologist who sits on the FDA advisory committee that may ultimately recommend the vaccines for licence or emergency use authorisation.
But that’s not right. In all the ongoing phase III trials for which details have been released, laboratory confirmed infections even with only mild symptoms qualify as meeting the primary endpoint definition.9101112 In Pfizer and Moderna’s trials, for example, people with only a cough and positive laboratory test would bring those trials one event closer to their completion. (If AstraZeneca’s ongoing UK trial is designed similarly to its “paused” US trial for which the company has released details, a cough and fever with positive PCR test would suffice.)
Part of the reason may be numbers. Severe illness requiring hospital admission, which happens in only a small fraction of symptomatic covid-19 cases, would be unlikely to occur in significant numbers in trials. Data published by the US Centers for Disease Control and Prevention in late April reported a symptomatic case hospitalisation ratio of 3.4% overall, varying from 1.7% in 0-49 year olds and 4.5% in 50-64 year olds to 7.4% in those 65 and over.13 Because most people with symptomatic covid-19 experience only mild symptoms,14 even trials involving 30 000 or more patients would turn up relatively few cases of severe disease.
In the trials, final efficacy analyses are planned after just 150 to 160 “events,”—that is, a positive indication of symptomatic covid-19, regardless of severity of the illness.
Yet until vaccine manufacturers began to release their study protocols in mid-September, trial registries and other publicly released information did little to dispel the notion that it was severe covid-19 that the trials were assessing. Moderna, for example, called hospital admissions a “key secondary endpoint” in statements to the media.15 And a press release from the US National Institutes of Health reinforced this impression, stating that Moderna’s trial “aims to study whether the vaccine can prevent severe covid-19” and “seeks to answer if the vaccine can prevent death caused by covid-19.”16
But Tal Zaks, chief medical officer at Moderna, told The BMJ that the company’s trial lacks adequate statistical power to assess those outcomes. “The trial is precluded from judging [hospital admissions], based on what is a reasonable size and duration to serve the public good here,” he said.
Hospital admissions and deaths from covid-19 are simply too uncommon in the population being studied for an effective vaccine to demonstrate statistically significant differences in a trial of 30 000 people. The same is true of its ability to save lives or prevent transmission: the trials are not designed to find out.
Zaks said, “Would I like to know that this prevents mortality? Sure, because I believe it does. I just don’t think it’s feasible within the timeframe [of the trial]—too many would die waiting for the results before we ever knew that.”
Stopping transmission
What about Hotez’s second criterion, interrupting virus transmission, which some experts have argued17 should be the most important test in phase III studies?
“Our trial will not demonstrate prevention of transmission,” Zaks said, “because in order to do that you have to swab people twice a week for very long periods, and that becomes operationally untenable.”
He repeatedly emphasised these “operational realities” of running a vaccine trial. “Every trial design, especially phase III, is always a balancing act between different needs,” he said. “If you wanted to have an answer on an endpoint that happens at a frequency of one 10th or one fifth the frequency of the primary endpoint, you would need a trial that is either 5 or 10 times larger or you’d need a trial that is 5 or 10 times longer to collect those events. Neither of these, I think, are acceptable in the current public need for knowing expeditiously that a vaccine works.”
Zaks added, “A 30 000 [participant] trial is already a fairly large trial. If you’re asking for a 300 000 trial then you need to talk to the people who are paying for it, because now you’re talking about not a $500m to $1bn trial, you’re talking about something 10 times the size. And I think the public purse and operational capabilities and capacities we have are rightly spent not betting the farm on one vaccine but, as Operation Warp Speed [the US government’s covid-19 vaccine plan] is trying to do, making sure that we’re funding several vaccines in parallel.”
Debating endpoints
Still, it’s fair to say that most of the general public assumes that the whole point of the current trials, besides testing safety (box 1), is to see whether the vaccine can prevent bad outcomes. “How do you reconcile that?” The BMJ asked Zaks.
Safety and side effects
History shows many examples of serious adverse events from vaccines brought to market in periods of enormous pressure and expectation. There were contaminated polio vaccines in 1955, cases of Guillain-Barré syndrome in recipients of flu vaccines in 1976, and narcolepsy linked to one brand of influenza vaccine in 2009.1819
“Finding severe rare adverse events will require the study of tens of thousands of patients, but this requirement will not be met by early adoption of a product that has not completed its full trial evaluation,” Harvard drug policy researchers Jerry Avorn and Aaron Kesselheim recently wrote in JAMA.20
Covid-19 vaccine trials are currently designed to tabulate final efficacy results once 150 to 160 trial participants develop symptomatic covid-19—and most trials have specified at least one interim analysis allowing for the trials to end with even fewer data accrued.
Medscape’s Eric Topol has been a vocal critic of the trials’ many interim analyses. “These numbers seem totally out of line with what would be considered stopping rules,” he says. “I mean, you’re talking about giving a vaccine with any of these programmes to tens of millions of people. And you’re going to base that on 100 events?”8
Great uncertainty remains over how long a randomised trial of a vaccine will be allowed to proceed. If efficacy is declared, one possibility is that the thousands of volunteers who received a saline placebo would be offered the active vaccine, in effect ending the period of randomised follow-up. Such a move would have far reaching implications for our understanding of vaccines’ benefits and harms, rendering uncertain our knowledge of whether the vaccines can reduce the risk of serious covid-19 disease and precluding any further ability to compare adverse events in the experimental versus the placebo arm.
“It’ll be a decision we’ll have to take at that time. We have not committed one way or another,” Moderna’s Tal Zaks told The BMJ. “It will be a decision where FDA and NIH will also weigh in. And it will be probably a very difficult decision, because you will be weighing the benefit to the public in continuing to understand the longer term safety by keeping people on placebo and the expectation of the people who have received placebo to be crossed over now that it has been proved effective.”
RETURN TO TEXT“Very simply,” he replied. “Number one, we have a bad outcome as our endpoint. It’s covid-19 disease.” Moderna, like Pfizer and Janssen, has designed its study to detect a relative risk reduction of at least 30% in participants developing laboratory confirmed covid-19, consistent with FDA and international guidance.2122
Number two, Zaks pointed to influenza vaccines, saying they protect against severe disease better than mild disease. To Moderna, it’s the same for covid-19: if its vaccine is shown to reduce symptomatic covid-19, it will be confident it also protects against serious outcomes.
But the truth is that the science remains far from clear cut, even for influenza vaccines that have been used for decades. Although randomised trials have shown an effect in reducing the risk of symptomatic influenza, such trials have never been conducted in elderly people living in the community to see whether they save lives.
Only two placebo controlled trials in this population have ever been conducted, and neither was designed to detect any difference in hospital admissions or deaths.23 Moreover, dramatic increases in use of influenza vaccines has not been associated with a decline in mortality (box 2).26
Not enrolling enough elderly people or minorities
A vaccine that has been proved to reduce the risk of symptomatic disease by a certain proportion should, you might think, reduce serious outcomes such as hospital admissions and deaths in equal proportion.
Peter Marks, an FDA official with responsibility over vaccine approvals, recently stated as much about influenza vaccination, which “only prevents flu in about half the people who get it. And yet that’s very important because that means that it leads to half as many deaths related to influenza each year.”24
But when vaccines are not equally effective in all populations the theory breaks down.
If frail elderly people, who are understood to die in disproportionate numbers from both influenza25 and covid-19, are not enrolled into vaccine trials in sufficient numbers to determine whether case numbers are reduced in this group, there can be little basis for assuming any benefit in terms of hospital admissions or mortality. Whatever reduction in cases is seen in the overall study population (most of which may be among healthy adults), this benefit may not apply to the frail elderly subpopulation, and few lives may be saved.
This is hard to evaluate in the current trials because there are large gaps in the types of people being enrolled in the phase III trials (table 1). Despite recruiting tens of thousands, only two trials are enrolling children less than 18 years old. All exclude immunocompromised people and pregnant or breastfeeding women, and though the trials are enrolling elderly people, few or perhaps none of the studies would seem to be designed to conclusively answer whether there is a benefit in this population, despite their obvious vulnerability to covid-19.
“Adults over 65 will be an important subgroup that we will be looking at,” Moderna’s Zaks told The BMJ. “That said . . . any given study is powered for its primary endpoint—in our case covid-19 disease irrespective of age.”
Al Sommer, dean emeritus of the Johns Hopkins School of Public Health, told The BMJ, “If they have not powered for evidence of benefit in the elderly, I would find that a significant, unfortunate shortcoming.” He emphasised the need for “innovative follow-up studies that will enable us to better determine the direct level of protection immunisation has on the young and, separately, the elderly, in addition to those at the highest risk of severe disease and hospitalisation.”
One view is that trial data should be there for all target populations. “If we don’t have adequate data in the greater than 65 year old group, then the greater than 65 year old person shouldn’t get this vaccine, which would be a shame because they’re the ones who are most likely to die from this infection,” said vaccinologist Paul Offit.8 “We have to generate those data,” he said. “I can’t see how anybody—the Data and Safety Monitoring Board or the FDA Vaccine Advisory Committee, or FDA decision-makers—would ever allow a vaccine to be recommended for that group without having adequate data.”
“I feel the same way about minorities,” Offit added. “You can’t convince minority populations to get this vaccine unless they are represented in these trials. Otherwise, they’re going to feel like they’re guinea pigs, and understandably so.”
RETURN TO TEXTAcknowledgments
Sarah Tanveer helped research the design of studies and identify quotations, and Ulrich Keil provided comments on an early draft of this article.
Footnotes
Competing interests: I co-wrote an op-ed on this topic with Eric Topol, who is quoted in this article, I have been pursuing the public release of vaccine trial protocols, and I co-signed an open letter to the FDA calling for independence and transparency in covid-19 vaccine related decision making.
Provenance and peer review: Commissioned; externally peer reviewed.
This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.
https://bmj.com/coronavirus/usageBelow is one of the responses to the article above. Please follow the link to read the rest of the responses.
21 December 2020
Dear Editor:
Peter Doshi’s nuanced article has opened up a rigorous and valuable discussion about the ongoing COVID-19 mRNA vaccine clinical trials, and I would like to home in on what is perhaps the most salient source of concern about their efficacy and safety, and a key step in building public confidence: the need for extensive data on the vaccines' cellular tropism and MHC Class I vs. MHC Class II-mediated antigen presentation, with attendant questions about potential seeding of autoimmunity. The mRNA vaccines’ nucleic acid payload is ferried into human cells via complex lipid nanoparticles (LNPs) with a lipophilic formulation capable of traversing phospholipid bilayers, through endocytosis and other mechanisms [1]. While some LNP vehicles have been engineered with specific tropisms for target tissues, others have less selective tropisms (or are even potentially omnitropic), capable of entering diverse cell types [2]. From studies thus far, it remains unclear under which category the LNPs used in the COVID vaccine trials appear to fall, and this point is essential for gauging long-term safety and efficacy. If these LNPs have a broad cell tropism, then they would be capable of entering and expressing the SARS-CoV-2 viral spike protein within the parenchyma of vital organs and tissues, well beyond the tropism of wild-type coronavirus. The resulting non-self protein, presented to immune surveillance via MHC-I complexes, would trigger a cytotoxic (CD8-mediated) immune response to the expressing cells, which could with time engender clinically significant tissue damage.
To elaborate briefly on this concern, and the data which could alleviate it, we can roughly subdivide the standard pediatric and adult vaccine repertoires into two broad categories, on the basis of the cells that present the vaccine-mediated antigen to T-lymphocytes. Inactivated virus vaccines (such as Hepatitis A and IPV) and protein subunit vaccines (Hib, Hepatitis B, pertussis) involve primarily MHC-II-based presentation (“Type 1 vaccines” for this discussion). Their antigenic material is taken up principally via phagocytosis by antigen-presenting cells (APCs), particularly dendritic cells in visceral and peripheral tissues, which migrate to lymph nodes and present immunostimulatory epitopes in complex with MHC-II cell-surface molecules to CD4-expressing T-cells (T helper cells), which in turn help to initiate and coordinate the adaptive immune response to pathogens exhibiting the antigen. A cytotoxic response is not induced against the APCs utilizing the MHC-II pathway. Attenuated virus vaccines such as MMR (“Type 2”) enlist not only MHC-II-expressing APCs, but also the MHC-I antigen-presenting pathway present in almost all cell types which, through a standard cascade of events, triggers a cytotoxic response (mediated by CD8-expressing lymphocytes) against presenting cells and tissues, due to the presence of non-self epitopes. Thus attenuated virus (Type 2) vaccines like MMR do entail some cytotoxicity against infected cells, but still confined to the original tropism of the target virus, with an enhanced immunostimulatory effect.
LNP-containing mRNA vaccines represent a novel Type 3 in this classification: enlisting both MHC-II-mediated (through dendritic cells and other APCs) and cytotoxic MHC-I-mediated immunostimulation, but against a far broader array of MHC-I-presenting cells and tissues than the wild-type virus, particularly for LNPs with unselective tissue tropisms. There is potential for enhanced immunostimulatory impact through this process, but also elevated risk of cytotoxic, inflammatory, and autoimmune effects, even more so if the liposomal particles can traverse the blood-brain barrier to enter, for example, motor neurons or oligodendrocytes (the glial cells targeted in multiple sclerosis). These effects, in turn, depend critically on the organ and tissue profile of the cells that receive the LNPs, translate the mRNA payload, and dock the antigenic protein on MHC-I cell-surface molecules. This is doubly true in the case of COVID-19 in light of the still-evolving picture of SARS-CoV-2 immunology. Immune responses appear to be incremental and fleeting, both for natural and vaccine-mediated immunity, suggesting a likely need for multiple boosters after an initial inoculation. Therefore, if cytotoxic responses to integral tissues are transpiring through MHC-I-mediated presentation of SARS-CoV-2 spike protein, the effects may be at first subclinical, manifesting fully only after successive immunizations over months or years.
At present, relatively little has been reported on the tissue localization of the LNPs used to encase the SARS-CoV-2 spike protein-encoding messenger RNA, and it is vital to have more specific information on precisely where the liposomal nanoparticles are going after injection, both in concurrent animal studies and in the two ongoing mRNA vaccine human trials. This process can be commenced in straightforward fashion through cell culture and animal-based investigations, by supplying mRNA expressing a fluorophoric reporter gene (such as green fluorescent protein) delivered via the same LNP formulations as used in the two vaccine trials, and tracking its ingress into varied cells and tissues. The mRNA vaccines represent a remarkable and promising technology, with potential to expedite the development of immunization protocols for future epidemics, but this promise will evaporate if unanticipated safety issues and side effects emerge to weaken public trust in the new modality. Cellular and tissue localization data on the vaccines’ tissue tropisms, obtained and confirmed across multiple independent laboratories, would constitute a valuable step to reinforce public confidence in this regard.
References
[1] doi: 10.1016/j.addr.2016.04.014. Mechanisms of transport of polymeric and lipidic nanoparticles across the intestinal barrier. Ana Beloqui, Anne des Rieux, Véronique Préat. Adv Drug Deliv Rev. 2016 Nov 15;106(Pt B):242-255.
[2] doi: 10.1016/B978-0-12-391860-4.00012-4. Lipid nanoparticles for drug targeting to the brain. Maria Luisa Bondì , Roberto Di Gesù, and Emanuela Fabiola Craparo. Methods Enzymol. 2012;508:229-51.
0 Comments