Assessment of Vaccines
Slides: https://www.andreashandel.com/presentations/
2023-10-30
Vaccines are pretty good
xkcd.com
Evaluation of vaccines
How do we determine if vaccines are good?
- Safety
- Immunogenicity
- Efficacy/Effectiveness
- Cost-effectiveness
Vaccine Development
Knipe et al Science 2020
Outcomes of interest
Vaccines (partially) protect those who receive them (direct/individual effect):
- Reduction in risk of infection/symptoms/hospitalization/death.
- Reduction in strength of symptoms.
Outcomes of interest
Vaccines (partially) protect those who receive them (direct/individual effect):
- Reduction in risk of infection/symptoms/hospitalization/death.
- Reduction in strength of symptoms.
Vaccines can also protect non-vaccinated contacts (indirect effect).
- Reduction of susceptibles in the population leads to overall reduced spread (contagion effect).
- Reduction of infectiousness/transmission potential leads to reduced spread (infectiousness effect).
See Halloran & Hudgens 2016 CER and references therein.
Indirect effect example
- Vaccine 1 reduces risk of clinical infection by 70%, reduces infectiousness by 30%.
- Vaccine 2 reduces risk of clinical infection by 30%, reduces infectiousness by 70%.
Gallagher et al, medRxiv 2020
Ways to evaluate vaccine impact
Measure it:
- Challenge studies
- Clinical trials
- Observational studies
Estimate it:
Measuring vaccine impact
Challenge studies
- One group receives the vaccine, the other placebo.
- Both groups are challenged with the pathogen under consideration.
- Measures vaccine efficacy (VE).
- Well-controlled, can use small(ish) sample size.
- Somewhat unrealistic (e.g., high challenge doses).
- Direct effect only.
- Sometimes not feasible/ethical.
Clinical trials
- One group receives the vaccine, the other placebo.
- Groups are followed and outcome (infection/disease/etc.) recorded.
- Measures vaccine efficacy (VE).
- Good balance between controlled and real-world setting.
- Usually needed for FDA approval.
- Only works if infections are high (not good for emerging pathogens).
- Can measure direct and indirect effects (but usually only direct).
- Expensive.
Observational studies
- Taking vaccine is up to individuals (so must be licensed).
- Cohort and case-control (e.g., test-negative) design.
- Measures vaccine effectiveness (VE).
- Most “real”, least controlled.
- Can lead to biased estimates.
- Can measure direct and indirect effects.
- Can be fairly inexpensive.
Test-negative design
Sullivan et al 2014 Exp Rev Vac
Measuring vaccine impact - summary
- Different study designs are available/useful.
- As you learned from the Halloran & Hudgens paper, because of dependent happenings, designing and analyzing vaccine studies to properly capture all vaccine effects can be tricky.
- Generally, a “classical” convincing phase 3 clinical trial is required to get approval (but see e.g., Ebola vaccine, H5N1 influenza vaccine).
- Measuring actual outcomes is always expensive and time-consuming, sometimes not feasible (e.g., SARS-CoV-3 or H5N1 influenza vaccines).
Estimating vaccine impact
xkcd.com
Correlates of protection (CoP)
- Determining an immunological quantity that correlates with protection can make vaccine assessment easier.
- Finding correlates of protection (for vaccines) is very valuable (but can be tricky).
xkcd.com
Vaccine CoP terminology
- It’s a mess.
- Some individuals (e.g., Plotkin) mean by “correlate” a mechanistic/causal entity.
- Some individuals mean by “correlate” something that correlates and might or might not be mechanistic/causal.
- Other terms are used by some to try to be clearer (e.g., surrogate, mechanistic CoP). See e.g. Plotkin & Gilbert 2012 CID.
- Protection is also often not clearly defined.
Correlates of protection
- An “absolute correlate” (a la Plotkin) does not exist.
- Levels at which a CoP leads to protection depend on pathogen, host, outcome, etc.
CoP - SARS-CoV-2 Example
Khoury et al 2021 Nat Med
CoP - SARS-CoV-2 Example
Khoury et al 2021 Nat Med
CoP - Influenza Example
Coudeville et al 2010 BMC MRM
CoP - Influenza Example
CoP - Influenza Example
CoP - Influenza Example
CoP - Influenza Example
CoP - Influenza Example
CoP - Influenza Example
Age Group 50-64
CoP - Influenza Example
Age Group 65+
CoP - Influenza Example
Age Group 18-49
Estimating vaccine impact - summary
- Using CoP can speed up approval process.
- CoP can depend on details of vaccine (e.g., LAIV vs. IIV) and hosts (e.g., children vs. adults).
- A full mechanistic understanding of
vaccine -> immune response -> protection
is still lacking for any vaccine (afaik).
Keep going?
Image by Aline Dassel/Pixabay
Research project - Assessing influenza vaccine candidates
Current Influenza vaccines
- Need to be reformulated almost every year because of virus evolution
- Need to be taken annually, due to virus evolution and vaccine waning
- Are not very good, even if the vaccine and circulating strains match
Future Influenza vaccines
- Should protect for a long time (lifelong?)
- Should have high efficacy
- Should protect against a wide range of strains
Universal flu vaccine challenges
- Many
- How to assess/compare vaccine candidates
How do we define a vaccine response?
Quantifying vaccine responses
Quantifying vaccine responses
Quantifying vaccine responses
- Magnitude: \(\frac{1}{N}\sum_{n} log(\textrm{TI}_{n,j=1})\)
- Overall strength: \(\frac{1}{N*J}\sum_{n} \sum_{j} log(\textrm{TI}_{n,j})\)
- Breadth: \(\frac{1}{N*J}\sum_{n} \sum_{j} \textrm{SC}_{n,j}\)
SC = Seroconversion, TI = Titer Increase (D28/D0), n = individuals, j = Strains.
Comparing vaccine responses
A new method to quantify/compare vaccine responses
- Organize strains by antigenic distance
- Fit a model to more robustly estimate magnitude/breadth/strength
Strain distance
Strain distance measures
- Time: absolute difference in years of strain isolation.
- Sequence: Some measure based on sequence difference.
- Biophysical: Measures based on computed or measured biophysical properties.
- Phenotypic: Antigenic cartography based on HAI assays.
“Our” data
- Data from UGAFluVac study
- Individuals received vaccine, response to multiple strains was tested
Strain distance measures
Strain distance measures
Quantifying vaccine responses
Comparing vaccine responses
Testing our method
We sampled from the panel of heterologous strains from UGAFluVac to mimic different labs
Testing our method - the results
The table shows the coefficient of variation for each outcome.
Magnitude |
0.088 |
0.103 |
Breadth |
0.059 |
0.431 |
Overall strength |
0.083 |
0.081 |
Our new method is worse (more variable)!
Testing our method with simulations
Create a universe of 50 possible heterologous strains with varying antigenic distances.
Create 10 lab panels by randomly sampling 9 strains and adding the homologous strain (distance of 0).
For each lab, generate 100 random individuals by simulating flu vaccine response titers from a model that shows linearly reduced response with increasing antigenic distance.
Simulation results
Magnitude |
0.025 |
0.008 |
Breadth |
0.199 |
0.020 |
Overall strength |
0.155 |
0.007 |
Now our new method is better. Hm…
Simulation results with 30% censored data
Magnitude |
0.028 |
0.033 |
Breadth |
0.290 |
0.316 |
Overall strength |
0.137 |
0.071 |
With censored data, the current method looks artificially good.
Research Project Summary
- Our proposed new method seems to be generally more robust.
- If a good amount of censored data are present, the current method falsely under-estimates the uncertainty.
- Our method also doesn’t properly handle the censored data (yet).
- We need to update our method to properly deal with the censored values. Then we can do another comparison of our method and the current approach.
Wrap-up
- Phase 3 trials are still the gold standard.
- There is increased recognition that indirect effects can be important and should impact decision making.
- Finding better CoP for any vaccine continues to be an important area of research.
Questions?
https://phdcomics.com/
- Slides: https://www.andreashandel.com/presentations/
- Contact: https://www.andreashandel.com