The “false-positive PCR” problem is not a problem

If you haven’t seen all the chatter that states PCRs using more than thirty-something (someone even suggested 25 to me!) cycles will suffer from false positives and so is useless, then read no further. For those who have seen this and wondered, I have two bits of data to show and discuss with you on why the false-positive PCR problem is not a problem at all for SARS-CoV-2 diagnostics. And then there’s a rant.

False positives are really common in practice

Below I’ve included an RT-PCR ‘run’ performed some years ago. It was screening nucleic acids extracted from sick human respiratory swabs and aspirates, looking for human parainfluenza virus type 4 (HPIV-4).

The RT-PCR was run for 55 cycles. Not 30. Not 40. Not 45. 55.

That’s excessive, but I actually liked to do that in my research days to a) make sure the test wasn’t doing anything weird and b) that there weren’t really weak positives I was missing (if a sample came up late (never saw one above 45) I would use a suite of other PCR tests to investigate further.

The point is that after 55 cycles of amplifying nucleic extracts from human respiratory tracts samples, the only thing that crossed the threshold (red line in the figure below) was the positive control (a known extract of HPIV-4 RNA). I have the same sort of data for other viruses.

The results from 72 samples amplified for HPIV-4 RNA. The red line was a known positive control sample positive for HPIV-4a. The pink line was a known positive for HPIV-4b RNA. Pretty specific test huh (HPIV-4b came up in the yellow fluorescence channel FWIW, also no other positives)? Two samples were water-only non-template controls (black lines) and the rest of the 72 lines represent nucleic acids extracted from distinct human respiratory tract swabs and aspirates. No false positives. I chose an arbitrary threshold of 0.05 normalised fluorescence unit (my personal favourite; orange horizontal dashed line). Any sample fluorescence that travelled above that line was considered a positive for HPIV-4 RNA in that sample and the point at which they crossed that threshold was their CT. The RotorGene run green (FAM) fluorescent channel data were exported into Excel for this figure and labelled to make a more colourful and clear image.

To say that any PCR run for more than thirty-something cycle will be a false positive, or increase the likelihood of a false positive, is misleading and wrong. The false-positive PCR problem is not a problem.

Thirty-something and biological variability

Look, I fully agree that the later (higher number) the threshold cycle (CT “cycle number”) of the RT-PCR result, the less target RNA is present at the start of the reaction. That RNA is a surrogate marker for the amount of virus but not an actual measure because PCR methods can’t tell you a virus is infectious.

People are very chuffed to have learned this fact in 2020 by the way. You can tell because they say so. A lot. But despite their newfound out-of-context knowledge they actually know very little about what they’re talking about.

Most of the time, if you pick two different labs to test a panel of the same samples, you’ll get two sets of subtly (sometimes not so subtly) different results. Molecular biology people each have their favoured way of doing things and they get quite dogged in their ways.🙄

A single CT isn’t comparable

I’ve lifted a figure from an article. It highlights this variability, which definitely also extends into the much more volatile realm of cell culture and virus isolation (growing virus from a patient’s sample using immortal cell preparations grown in flasks).

This particular study found that that from among 5/60 (8.3%) patient samples with a CT greater than 35, infectious virus was still present. An arbitrary cut-off at 35 would have missed infectious people. Figure 2 from Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020.[1]

The figure shows that if you were to blindly and ignorantly rely on single CT values to proclaim lack of infectious risk – let’s say 35 or greater – you would miss people who were still shedding infectious virus.

If you surveyed more labs with expertise in cell culture and virus isolation and labs that could get hold of fresh samples and samples from the actual site of virus replication, I’m sure you could find even later CTs. You can also find labs who can’t isolate virus from samples with CTs this late. This biological variability means we need to step back from these stupid single number cut-offs.

Ummmmm…what false positives?

When presented with a conspiracy theory – I like to ask for the evidence. I generally reject YouTube video sources and weird websites. They never have facts just belief and opinion.

So the real question is: where, oh theorists of the conspiracy type, are all these false positives that lots of PCRs all over the world are generating?

(h/t to Klaus Hentrich for this idea)

Let’s look at Australia.

We have no reported local transmission of SARS-CoV-2 or cases of COVID-19 disease to speak of as I write this. I reckon that’s pretty low prevalence in a country of 25 million.

Australia’s COVID-19 cases right now are almost exclusively in quarantine hotels and among Aussies returning, or special-purpose travellers visiting, from a world on fire. We’ve had a few clusters spin out from quarantine but they’ve – so far – been squished dead in weeks to months. Even Victoria, where Wave 2 peaked at over 700 cases per day, is at its 23rd consecutive “doughnut day” (🍩; zero locally transmitted cases). It had 4,293 active cases just three months ago – “today we have none“.[3] Take that in for a second.

This has only been manage in a handful of locations. Specifically, Wuhan (and China in general which peaked at around 4,000 daily cases)[6] and Singapore which contained a dormitory outbreak that peaked at 1,400 cases per day as well as more widespread community cases (outside that crowded foreign worker dorm environment) which never reached higher than 50 cases per day.[5] Both of these locations have now had multiple doughnut days (or months).

Australia hasn’t seen its combined national daily tally of almost exclusively PCR-based tests, fall below 20,000 since June. These aren’t just test results from hotel quarantine by the way – this is mostly from among the community in each of Australia’s eight jurisdictions.

Data on daily testing and the 7-day moving average of testing across Australia.
Source: the excellent ABC News website.[2]

All that testing provides a good number of opportunities for those so-called false-positive PCR results to show up in our daily reported numbers. Surely more testing should mean a steady flow of false positives if these conspiracies are correct? And others love to add that it should be even more of a problem in a low prevalence setting. Australia certainly fits that bill. Plus, we’re testing mostly sick people but also asymptomatic and presymptomatic people.

But lo and behold there is no steady stream of false positives. We don’t have an issue because once again, this is an over-oxygenated amateur conspiracy theory and not an actual issue.

If we have a look at the next figure, we can see no constant stream of positives among the local testing. Yes, cases are occurring (as far as I know, mostly the same RT-PCR-based testing) among those coming from outside Australia but these are – as far as I know – linked to ill travellers diagnosed as having COVID-19. Actual clinical cases. The false-positive PCR problem is not a problem.

That’s not to say there are never false positives. There are. But they are very, very rare events that are almost always caught by the process involved in reporting test results. This process considers the lab results alongside clinical and epidemiological context and checks itself before reporting.

It may be that some parts of the world are too overwhelmed to use that sort that process. And in that instance, other testing might prove helpful. For example the less sensitive overall, but contagiousness-detecting rapid antigen tests may be helpful.[4] Or they might not; there isn’t a lot of real-world evidence for or against their helpfulness. But in the US – what is there to lose?

Nonetheless, an overwhelmed testing system doesn’t make PCR results useless it just makes them less obtainable.

The false-positive PCR problem is not a problem

There clearly isn’t a CT crisis or a false positive plague or a “casedemic”. The false-positive PCR problem is not a problem. There is, however, a crisis in the lack of understanding about a heretofore obscure branch of science that toils in relative obscurity and a tool it relies on; pathology testing and PCR.

But hey, is anyone actually saying there is a problem? Sadly, yes. These are is a couple of the many I’ve read. One is a comment currently in holding, submitted to this blog. The other a tweet at me last night. And yet despite sounding so sure of itself, it isn’t supported by facts and is simply absolute rubbish.

Don’t blame PCR for lots of positive people – blame your government

Just about every aspect of the COVID-19 pandemic has at one time or another during this pandemic, been hauled up as a reason to deflect from the reality of 2020. The reality is that ineffective, unprepared, slow-to-act and cloth-eared leadership – supported by the cult of wish-it-away – have slowed the response to this pandemic, making things worse. Not just in 2020 but across decades of profit-over-people decision making.

Leadership needs to lead

Governments that won’t provide universal healthcare can’t look after the mental and physical wellbeing of their populations so they haven’t been able to safely take the harsh measures which most effectively break widespread transmission while also maintaining suitable care for the multitude of health needs of those under their care.

Governments who won’t get serious about financial protections for the lives and livelihoods of frontline workers have failed to protect their communities. Instead, they’ve doubled down against the most effective restrictions to prevent bad economic numbers.

Governments who had in place convoluted lab test development regulations and who hadn’t built up testing capacity have instead had to settle for less ideal ways to test people because the gold standard is unachievable. Or it’s just too late.

Governments who failed to invest and listen to science and public health have not been able to use the power of contact tracing to help contain more transmission and have generally made poor decision putting their communities at greater risk.

Governments who have failed to learn from successful countries around subjects like safer return to school, how to reduce restrictions carefully, the importance of multi-lingual and multilayered communication and eductaion on all facets of the pandemic and changing the dogma of “droplet” verse “airborne” in response to the screaming pleas of expert aerosol scientists.

There are also the loud angry supporters of those governments who consciously hold fast to and amplify these failings for their own benefit and fame, or because they don’t know any better, or they simply follow other malicious agents of chaos down a dark path of often angry and usually selfish fantasy.

Individual rights over societal good is a doctrine that yields disastrous results it seems, for the health of an interconnected world. Who could have guessed this?

But I’ve digressed.

The PCR test works to find infections. It can find them before you become sick. That helps in containing spread if you have a process for isolation.

PCR testing can find infection if you were sick recently but missed a timely test. Whether you feel sick or not, that means contact tracing can be used to contain transmission chains that you may have started.

PCR can actually be used to slow SARS-CoV-2 spread and contribute to bringing an epidemic under control. Of course only if you actually do something to respond to those results.

If you miss transmission chains and cases or find them but don’t do anything to interrupt them, then you simply have to rely on hope and faith that things will work out in the end. So far, hope and faith haven’t helped much.

References

  1. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020
    https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2020.25.32.2001483
  2. Charting the COVID-19 spread in Australia
    https://mobile.abc.net.au/news/2020-03-17/coronavirus-cases-data-reveals-how-covid-19-spreads-in-australia/12060704?nw=0&pfmredir=sm
  3. Statement From The Premier, Victoria, 22NOV2020
    https://www.premier.vic.gov.au/statement-premier-82
  4. Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture
    https://www.medrxiv.org/content/10.1101/2020.10.02.20205708v1
  5. Singapore’s COVID-19 Interactive Situation Report
    https://www.moh.gov.sg/covid-19/situation-report
  6. China COVID-19 cases via the World Hleath Organization dashboard
    https://covid19.who.int/region/wpro/country/cn

Hits: 2858

24 thoughts on “The “false-positive PCR” problem is not a problem”

  1. Early in the pandemic my lab, as was the case with so many other labs, received our PCR chemistry from a company that had inadvertently contaminated most of the covid N gene test with the template they were producing as a positive control for the test. we ended up with 70,000 reactions worth of this test that had a false positive rate of about 1in 500. However because my lab is competent when we saw a single positive reaction at CT 37+ from a chemistry that we knew was compromised we would rerun that sample three times with that test and three times with a secondary test targeting a different Gene. It sounds excessive but actually the cost of these test is very inexpensive compared to the cost of the manpower required to run them and since we were running several hundred per day rerunning one sample six times did not increase our burden. All of these results were available same day the swab was taken, typically within five or six hours. We found a pretty even mix of those who were shown to be negative and those with late CT values who were positive in all six tests. The ones we were confident were positive were re swabbed the following day and would have CT values in the low twenties. Because we were testing asymptomatic people sometimes we just caught it so early that there wasn’t much virus available to detect and the following day they would be symptomatic with a much higher level of infection.

  2. I feel like you left out something about false positives: for this type of testing a false positive isn’t harmful. A false negative would allow an infected person to continue to be infecting others. A false positive would mean a person is quarantining when they don’t have to. Just to further point out that false positives aren’t as much of a problem as people seem to think.

    1. There is harm. Lockdowns can result from false positives which weren’t necessary. There is mental harm from them but also from the stress of thinking there are cases in your neighbourhood and that you need to get tested, rush to eth shops to ‘stock up’ and so on.

  3. Thanks Ian. It boils down to the use of positive and negative controls in every assay, and the use of sufficient replicates. That way you don’t have to pick some arbitrary Ct cut-off.

    1. Not quite. You accept the Ct based on your expertise in the lab, knowledge of that test, the setting of a useful threshold and your skill in using PCR. And that doesn’t = diagnosis. A PCR result is just one step in the path to diagnosis. A diagnosis is a clinical endpoint, not a lab endpoint.

  4. I don’t see that you have proven that false positives dont occur at some level here.
    1. You start by describing an example where a control positive clearly stands out from another 72 samples. So no false positives in 72 tests. That might happen randomly if the false positive rate were 1% for example.
    2. You explain that 5 of 60 positive tests at 35 amplifications are indeed positive with live virus. So you need that amplification to detect almost all real positives. But by that same statement 55 out of 60 positive tests at 35 amplifications did not contain live culturable virus so were presumably false positives.
    3. You discuss the example of Australia and generalise the result to the rest of the world. As Australia has been successful in test and trace containment the virus is not in epidemic or endemic stages there. The prevalance is all but zero and therefore the likelihood of any operational cross-contamination between positive and negative swabs is also very small. But this isnt true of the ROW. In the UK swabs are collected by the person themselves and bagged by them. Someone then grabs the bag with a grabber and chucks it into a cardboard box with 100s of others. All bags contact an infected/non-infected person, the same grabber, and numerous other bags in the same box. Therefore immediate cross contamination possible – especially if there are say 1 or 2% true positives in the population. Testing is also in very high volume – 400,000 a day in rapidly created labs using people with very limited training. Some swab test tubes are found to have leaked from the bag in the box on arrival. These are handled alongside the debagging of the other swab tubes by the same people. Labs are under extreme pressure to process tests quickly in vast numbers. Labs have been allowed to waive quality controls in favour of maximising throughput. No monitoring of operational false positives/negatives appears to take place using known control positives/negatives inserted at the testing stations.
    4. Here is the UK statement on false positives for PCR…https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/895843/S0519_Impact_of_false_positives_and_negatives.pdf
    No monitoring appears to be in place and the official answer in parliament is that the operational false positive rate for the PCR testing is unknown.

    Now tell me that false positives can never be a problem and are always vanishingly small.

    1. 1. It might. This is meant to address those who say “any PCR run for more than X cycles” will come up positive.
      2. Not false. True positive just not with detectable virus in that sample, using that method at that time. Doesn’t change the fact that those people are all infected.
      3. Yes, because Australia is one of a handful of countries not on fire
      4. Sorry, but that doesn’t change what this post set out to rebut. False positives don’t just happen with late cycle numbers or even very often at all (<1 in tens or hundreds of thousands of tests when performed by good labs). Yes, when things are completely out of control, this may change, but then so do many aspects of the pandemic under those conditions. Like the percentage of excess deaths.

      Your complaints all suggest you didn't read the opening paragraph of the post in which I lay out the scope of what will be addressed.

  5. I’d like to hear your comments on this study: https://www.cebm.net/study/covid-19-testing-and-correlation-with-infectious-virus-cycle-thresholds-and-analytical-sensitivity/

    This seems to be specifically looking at culturability of the virus. If it’s not culturable it’s almost certainly not going to be infectious, right? So we see one specimen where a 30 CT cut off would have been a false negative. But with a 40 CT cut off a ton of false positives.

    Am I reading this wrong?

    1. What you see really nicely by comparing the single example here and the single example in my blog – is a range of results. As I wrote, stating a single Ct value like 30 or 35 as the cut-off is not even attempting to accommodate that variability. Bad science. People may not be aware that culturing virus is an artform. And it’s even more subject to biological variab ility and the effects of poor specimen transport, collection, storage freezing and so on. The expertise to perform viral isolation by cell culture also sits across a spectrum, just like the expertise of labs conducting PCR.

  6. “This particular study found that that from among 5/60 (8.3%) patient samples with a CT greater than 35, infectious virus was still present.”

    I’m just trying to understand if this means that only 8.3% of the people that tested positive actually have an infectious level of virus? If so wouldn’t the false positive rate be sky high?

    If you’re only worried about missing those five people then I guess you’re not worried about that. But if you’re worried that the other 55/60 false positives will trigger some threshold that closes your kids schools (like here in NYC) then it matters a lot.

    1. The point here is that saying “anything over 30 or 35 cycles is not infection virus – as people do – is wrong. The % isn’t the point.

  7. Good article, but I think a lot of the public will have the same questions as me:

    1) You performed this test with the human parainfluenza virus type 4. How do you know the same results will occur with Covid?
    2) Is it true, even a tiny fragment of a dead virus will be picked up in this test? In which case, there will be many false positives.
    3) Did I imagine it, or did the inventor of the test, Kary Mullis, say it wasn’t to be used for detecting viruses? That was of course, before he mysteriiously died in October 2019, just before the Pandemic.

    1. 1) It won’t. But from my epxerience it works the same way with SARS-CoV-2.
      2) A certain amount is needed even for this test to produce a signal. Thousands and thousands of negative tests and no positives show that this type of false positive isn’t happening.
      3) If he did, he was completely wrong. I tell you this as someone who uses PCR-based tests to detect viruses every week and has done so for over 20 years. You can, of course, believe what you like but I’m not here to lie to you or to sell a book or a vitamin or go on a speaking tour.

      1. “Thousands and thousands of negative tests and no positives show that this type of false positive isn’t happening.”

        That’s in Australia right? There are plenty of positives in other countries. How do they make sure they’re detecting current infection and not an already past infection?

        As for the economics that sort of response is all fine and well if you’re rich. Much poorer countries whose economy was already on life support have been devastated by the steps taken by their governments.

  8. So i guess Fauci is wrong again by claiming anybody with a cycle above 35 would almost certainly be inactive virus, can’t be cultured etc?

  9. Ian, Thank you for your initial response. I wasn’t complaining, I was just pointing out limitations in your argument. You may say your scope is limited to PCR multiplications, but the title of your article is broader and is being cited by some as showing that operational false positives cannot occur in significant numbers. Clearly as you agree this is not correct and there are known cases of PCR testing problems with false positives. Locations under taking large scale testing need to be aware of the danger of PCR operational false positives and apply very tight quality control checks to combat the real risk of pseudo-epidemics.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.