Sometimes it’s hard to convey complex issues in a short-form article written for a general public audience. Other times, stories start off with an overly-simplified title they never recover from. Occasionally they lack any “so what” factor, or address a problem they never define. Back in early June, an article in the Australian version of the Conversation had a bit of all of this; maybe there was just too much to cover in less than 800 words.  Entitled “The flu vaccine is being oversold – it’s not that effective“, this was a good example of a story that may have simply needed more words.

Others have already commented on some of the things missing from the article. A nice follow-up piece in the Conversation by Prof. Allen Cheng was one example in Most people don’t benefit from vaccination, but we still need it to prevent infections. He also allowed me to report some of his comments in reply to the original Conversation piece in The forgotten benefits of the flu vaccine, as did Dr Sue Ieraci in Flu vaccination and effective communication. 

In a subsequent radio interview, one of the article’s authors, Prof Chris Del Mar went further, stating that the influenza vaccine “only makes a very small difference” and that it “reduces the chance of getting flu by about one to two per cent” and is “largely ineffective”. A Fact Check over at ABC news found the 1 to 2 per hundred (1 to 2%) claims to be misleading. 

The Fact Check noted that the numbers used in the Conversation and the reference reviews were correct but needed to be used together and clearly. Relatively speaking, immunisation led to a 50% drop in influenza cases (from 2.3% down to 0.9%); half as many who were vaccinated came down with confirmed illness. In absolute terms, that meant 0.9% of healthy adults who were vaccinated got influenza instead of what the study had found to be the “normal” 2.3% of unvaccinated healthy adults. 

That 2% figure is an underestimate of influenza cases. It is based on laboratory-confirmed detections which are only ever a fraction of all infections.

The Fact Check noted a range of 5 to 10% [also here] which is 1.2 to 2.4 million people in Australia. That will still be an underestimate. As more is understood of flu, we realise that infection also has an impact on the cardiovascular system, not just the airways. But vaccine studies aren’t always looking at heart disease in their analyses.

We also know too little about how immunisation affects transmission, recovery from milder flu illnesses (that you wouldn’t visit a doctor about), days off work, recovery speed or viral load. Capturing all these effects is complex. But we need to do this if we want to really understand the impact of this viral foe.

A percentage is a fraction of a number. Apply it to the kinds of numbers seen at a national population level and you can see that it might mean a lot fewer direct and indirect illnesses and some big cost savings in healthcare dollars.

But I don’t want to dwell on the 1%/2% thing.

I want to talk more about the statement that the vaccine is largely ineffective and briefly look at the evidence used by the Conversation authors to support that statement. 

Three big review publications were referred to as being inconsistent with the advice “we’re” (the public I presume) being given; get immunised to prevent influenza. Nowhere else in the article do the authors provide an example of what it is that we’re being told about influenza vaccines that is unreasonable. Are we being told they will protect 100% of us? Not specifically. But that could be implied from the language used. Are we being told that the influenza vaccine, is 100% safe? No, and you don’t have to look far beyond any health department’s website or even vaccine manufacturer’s website and information sheet to find out that there is some risk. There are health risks associated with taking any medicine. And with sitting on the couch too much.
So what precisely about the flu vaccine is being oversold? That’s not clear and so the rest of the article is founded on…nothing clear. The problem hasn’t been defined so what record are the authors hoping to set straight?  

The Cochrane reviews.

Nearly half of all words in the Conversation article were devoted to the Cochrane review articles.

The @ABCFactCheck suggesting Prof Del Mar’s 1-2% stat is misleading, is misleading in itself.

Your overall chance of getting the flu after a vaccination does decrease by 1-2% – we explain here: https://t.co/dI5Ls1ILNB

— The Conversation (@ConversationEDU) July 6, 2018

These are systematic reviews that find and weigh all the scientific evidence to answer a scientific question after that evidence has been filtered against pre-specified criteria.  The first of the three reviews was the source of the data that fuelled #percentgate (example above) while the second was considered by the Conversation authors to have found similar results. The third struggled to find good evidence of anything.

Let’s have a look just at the known influenza cases and whether vaccination has an effect that might be in the realm of what the public expects – without actually knowing what that realm looks like.

Review 1: flu vaccines and healthy adults

This review, updated in February 2018, found 25 randomised studies.

Looking only at moderate-certainty evidence, the risk of influenza in immunised healthy adults was reduced by 59% (from 2.3% to 0.9%) for live attenuated vaccines. Seventy-one healthy adults needed to be vaccinated to prevent 1 case of the flu…in healthy adults. I wonder how many flu cases in neonates or pregnant women would be prevented by 71 vaccinated healthy adults? 

The 2.3% figure is worth looking at for a minute.

Healthy adults are not the group where flu is having its biggest impact. Only 2.3% of healthy adults had the flu in this analysis of other’s studies. We also know that healthy adults don’t get as sick as often as the very young and very old; targets for vaccination campaigns.

Among the hospitalised group looked at in this review, 14.7% of people who were unvaccinated had influenza.

I’m not making any comparison’s here – I just think it’s worth noting the differences in influenza rates in different age and illness groups.

Interestingly, Review 1 concluded that vaccination had little effect on hospitalisations. This was based on data from two studies; one from 2000 and one from 1971. No RT-PCR used here (and what impact might testing sensitivity have here?)!

1. Bridges et al (2000) used illness and culture-isolation, virus and serology to confirm influenza in a subset of influenza-like illness (ILI). Only one of the two years yielded data, 1997-1998 (1,184 participants). In this year there was 1 hospitalisation and that was in a vaccinated person. But this was during a year in which there was a vaccine mismatch when flu vaccines cannot perform well.
   Only a subset of ILIs was sampled for laboratory analysis to define the viral strains circulating. A respiratory illness won’t be prevented by an influenza virus vaccine if the cause is a different virus. Surely there are better studies than this?
2. Leibovitz et al (1971) used clinical illness, virus isolation and seroconversion to determine influenza virus infection. Seroconversion happens when there is a steep rise in antibodies specific for the particular virus in a second blood sample, compared to levels in the first.
   The Cochrane authors took their data from patients hospitalized with upper respiratory infection (URI). Among the 1,682 vaccinated, 271 (16.1%) and 1,331 of 7,934 (16.7%) unvaccinated control participants were hospitalised with URI. I’m not sure why, but the Cochrane authors chose not to use the data that showed only 5 of 1,682 (0.3%) vaccinated and 102 of 7,934 (1.3%) unvaccinated participants acquired Hong Kong influenza-positive disease. This was a pretty significant dataset because the vaccine used in this study contained only this flu strain so it is highly relevant to how effective the vaccine was to compare those who acquired the HK influenza disease after vaccination compared to controls. Leibovitz and colleagues thought so because they note that this was a statistically significant difference and calculated that this vaccine was 73% effective in reducing the number of participants hospitalized due to influenza.
   It also turns out that about 35% of influenza positives also had adenovirus coinfections so using URI that is otherwise undefined is weak. Adenoviruses are not prevented by influenza vaccination but they are respiratory pathogens that cause ILI.  

I haven’t gone through the entire review like this but I now wonder about how much we can rely on this review and its 1%/2% conclusion.

Review 2:  flu vaccines and healthy children

This review, updated in February 2018, found 7 randomised controlled studies with laboratory-confirmed influenza.

Looking only at moderate-certainty evidence, the risk of flu in immunised children was reduced by 78% (from 18% to 4%) for live attenuated vaccines. Seven children needed to be vaccinated to prevent 1 case of the flu.

For inactivated vaccines (what we use in Australia), high-certainty evidence confirmed flu was reduced by 64% (from 30% to 11%) in immunised children. Five children need to be vaccinated to prevent 1 case of influenza. 

Between 40-60% is the general and documented knock-down effect we’ve come to expect from influenza vaccine immunisation (those who get vaccinated and develop immunity). Because children contribute a lot to the transmission of respiratory viruses, it’s actually pleasing to see within this group a higher value. This is probably above expectations and not really oversold, depending on what the authors of the Conversation article expect – or think the public expects – influenza vaccines should be delivering. 

Again, the Cochrane review authors make the point that more and better studies are needed. 

Review 3: flu vaccines in the elderly

This review, updated in February 2018, found 3 randomised controlled trials with laboratory-confirmed influenza.

The authors reported that the risk of confirmed influenza among the immunised was reduced by 58% (from 5.7% to 2.4%).  Around 30 elderly people need to be vaccinated to prevent 1 case of influenza in the elderly. That’s an effect that seems to be within specifications, despite how negative the tone of parts of this review seem.

Yep, the Cochrane review authors make the point that more and better studies are needed.

Below expectations or within expectations?

Public health decisions need to be made based on the best data available at the time. The decision-makers aim to deploy the safest measures available to reduce the greater risk represented by naturally acquired disease.

So what are the unreasonable expectations which the public are being oversold about influenza vaccine effectiveness? Where is the vaccine being called unrealistically effective?

I’m going to make an assumption here. I think the overselling claims relate to those occasions when media outlets suggest that injection (vaccination) is the same as immunisation (becoming protected). This is easily implied from “terrible outcomeX after influenza…so get that vaccine”, magic bullet-style messages. But we can see above that even with a good 60% rate of effective protection from influenza, 40% or more of those vaccinated can become infected. It’s misleading to imply that influenza vaccination will prevent everyone injected from becoming ill or from becoming a transmitter of influenza virus to someone else who may become seriously ill. It’s misleading to write in such a way as to imply that we have more evidence than we do. We don’t have enough data to support some of the things said and so we may be misleading. Public health websites use careful language about reducing risk but is that inadvertently given the vaccine more credit than it deserves? We could always be more clear about this when it comes to the public so as to “reduce the risk” of building unrealistic expectations.

The Cochrane reviews – like them or not – and pretty much everyone else working in this space agree that there is a need for more and better data on the burden of influenza disease. Review 1 also found that influenza vaccines are not associated with increased risk of serious adverse events following vaccination, which is worth highlighting.

Those reviews also show us that that influenza vaccines do have an effect and that effect varies with the group being studied. They don’t address the complexities of vaccine mismatch, test method differences, pre-existing influenza immunity, immune priming in response to our earliest experiences with a big bout of influenza, influenza co-infections, persistent circulation of the same influenza strain in the community, or the impact of changes in dominant influenza virus from year to year – to name a few of the complex issues associated with influenza.

Does the influenza vaccine provide the same effect as, say, a 3-dose once-off childhood vaccine that immunises us for most of our lives? No way. Should we talk more realistically about what influenza vaccines can do? Absolutely. But that sentiment cuts both ways. 

Influenza vaccines can and should be much more effective but while we wait, and wait…and wait for those, let’s not undersell the effects of the vaccines we have. 

References

1. The flu vaccine is being oversold – it’s not that effective
   https://theconversation.com/the-flu-vaccine-is-being-oversold-its-not-that-effective-97688
2. Most people don’t benefit from vaccination, but we still need it to prevent infections
   https://theconversation.com/most-people-dont-benefit-from-vaccination-but-we-still-need-it-to-prevent-infections-97928
3. The forgotten benefits of the flu vaccine…
   https://virologydownunder.com/the-forgotten-benefits-of-the-flu-vaccine/
4. Flu vaccination and effective communication…
   https://virologydownunder.com/flu-vaccination-and-effective-communication/
5. Vaccine Effectiveness – How Well Does the Flu Vaccine Work? [last updated 03OCT2018]
   https://www.cdc.gov/flu/about/qa/vaccineeffect.htm
6. How Effective Is the 2018 Flu Shot? Here’s What You Should Know
   http://time.com/5138100/how-effective-is-the-2018-flu-shot/
7. Vaccines for preventing influenza in healthy adults
   http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD001269.pub6/full
8. Vaccines for preventing influenza in healthy children
   http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD004879.pub5/full
9. Vaccines for preventing influenza in the elderly
   http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD004876.pub4/full
10. Fact check: Are your chances of getting the flu after a vaccination only decreased by 1 to 2 per cent?
    http://www.abc.net.au/news/2018-07-05/fact-check-flu-vaccine/9906284
11. Emergency Department demand associated with seasonal influenza, 2010 through 2014, New South Wales, Australia
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5635331/
12. Seasonal influenza, World Health Organization [accessed today]
    http://www.who.int/ith/diseases/influenza_seasonal/en/
13. Progress and Remaining Gaps in Estimating the Global Disease Burden of Influenza
    https://wwwnc.cdc.gov/eid/article/24/7/17-1270_article
14. Effectiveness and cost-benefit of influenza vaccination of healthy working adults: A randomized controlled trial.
    https://www.ncbi.nlm.nih.gov/pubmed/11015795
15. Correlated studies of a recombinant influenza-virus vaccine. IV. Protection against naturally occurring influenza in military trainees.
    https://academic.oup.com/jid/article-abstract/124/5/481/913777