Diagnostic Accuracy of STANDARD Q COVID-19 Antigen Detection Kit in Comparison with RT-PCR Assay using Nasopharyngeal Samples in India

1. Senior Resident, Department of Microbiology, Government Medical College, Srinagar, Jammu and Kashmir, India. 2. Senior Resident, Department of Community Medicine, Government Medical College, Srinagar, Jammu and Kashmir, India. 3. Senior Resident, Department of Community Medicine, Government Medical College, Srinagar, Jammu and Kashmir, India. 4. Professor and Head, Department of Community Medicine, Government Medical College, Srinagar, Jammu and Kashmir, India.

Abstract

Introduction: Real-time Reverse Transcription-Polymerase Chain Reaction (RT-PCR) can be considered to be the gold standard for diagnosis of Coronavirus Disease-2019 (COVID-19). Though it is highly accurate but has some limitations in terms of its use, which means that Rapid Antigen Tests (RAT) can support COVID-19 mitigation efforts.

Aim: To estimate sensitivity, specificity and degree of agreement of STANDARD Q COVID-19 Antigen Detection Kit in comparison to real-time quantitative RT-PCR (qRT-PCR).

Materials and Methods: This cross-sectional study was conducted at Government Medical College, Srinagar, Jammu and Kashmir, India, in April 2021. Socio-demographic and clinical information was collected on a pretested schedule after which two consecutive nasopharyngeal swabs were collected from each subject. One sample was tested using the STANDARD Q COVID-19 antigen test and the other was tested using qRT-PCR. Sensitivity and specificity were calculated using standard formulas. Cohen’s Kappa was calculated and Mann-Whitney U test was used for comparison.

Results: The study included 473 subjects with a mean age of 38.4±12.2 years. Around 1/4^th (124 subjects) of subjects were symptomatic at testing with the most common symptoms being fever (57.2%), cough (50%), sore throat (43%), myalgia (25%) and diarrhoea (13%). The sensitivity, specificity, positive likelihood ratio and negative likelihood ratio were estimated to be 54.4%, 99.2%, 71.49 and 0.46, respectively. The Cohen’s Kappa between the two tests was 0.644. Cycle threshold value was significantly lower in subjects with symptoms and those with a positive rapid test among those positive on qRT-PCR.

Conclusion: The STANDARD Q COVID-19 antigen test has a reasonable sensitivity, high specificity with a substantial inter-test agreement in comparison to qRT-PCR.

Coronavirus disease-2019, Rapid antigen, Reverse transcription polymerase chain reaction, Sensitivity, Specificity

The Coronavirus Disease-2019 (COVID-19) pandemic led to significant morbidity, mortality in addition to unprecedented disruption of economic activities globally (1),(2). A total of 183 million cases and more than 3.9 million deaths have been reported globally till June 2021. India accounted for around 30 million cases and 0.4 million deaths of these numbers till June 2021 (3). It has undone decades of improvement made in health of the communities and it is imperative to bring this pandemic under control so that a concerted global effort is made towards achieving Sustainable Development Goals (4),(5). Mitigating the COVID-19 pandemic will require multipronged interventions but augmentation of testing capacity is one of the core strategies that were advocated by the World Health Organisation (WHO) (6). Testing formed one of the main components of what came to be known as Test-Trace-Isolate-Treat strategy (7).

Timely and accurate diagnosis of COVID-19 is essential for limiting the spread and early clinical management of COVID-19 (8). Real-Time quantitative Reverse Transcription-Polymerase Chain Reaction (qRT-PCR) is considered as the gold standard test for detection of Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) owing to its high sensitivity and specificity but the requirement of special equipment, long turnover time, high cost and need for skilled staff limit its use in the field settings (9),(10). A need for a rapid and less resource-intensive antigen detection assay was felt early in the course of this pandemic and multiple RAT were developed (6). Despite having lower sensitivity and specificity than the conventional qRT-PCR, these tests still are an important tool for mitigation of COVID-19 pandemic particularly in field/community settings (11),(12).

These rapid diagnostic tests are easy to perform, did not require specialised laboratory support and can easily be done at point of care. These benefits need to be balanced with the decrease in diagnostic accuracy and that needs data regarding the diagnostic accuracy of these Rapid Antigen Tests (RAT). The present study was conducted with the purpose of estimating the diagnostic accuracy of one rapid antigen diagnostic kit in comparison to qRT-PCR tests.

The objectives of the study were:

• To estimate the sensitivity and specificity of STANDARD Q RAT in comparison to qRT-PCR for COVID-19.
• To estimate the degree of agreement between the two diagnostic tests.

Material and Methods

A cross-sectional study was conducted by Government Medical College, Srinagar, Jammu and Kashmir, India, at one urban health center affiliated with the Department of Community Medicine. Samples for the two tests were collected between 1^st to 30^th April 2021. The health facility was one of the designated COVID-19 testing centres. Testing services were provided on all days and subjects included those referred from Outpatient Department (OPD), contacts of positive patients and those requiring testing for any other criteria like before undertaking travel or any elective procedure. The centre conducted an average of 150 tests daily during April 2021. The study was approved by Institutional Review Board of the Institution (No. GMCS/EC/2021/109) and informed consent was taken from all subjects and only subjects who provided informed consent were included in the study. The study period corresponded to time just after the peak of the deadly second wave in India. (Table/Fig 1) has depicting the time period when the study was conducted.

Sample size calculation: Sample size was calculated using Buderer’s formula at 95% confidence level with an expected sensitivity of 70%, expected specificity of 95%, a precision of 10%. The sample size was estimated to be 359. A total of 473 were included in the study (12),(13).

Inclusion criteria: All subjects aged 18 years and above who reported to the health facility for testing were included in the study.

Exclusion criteria: Subjects coming for repeat testing, subjects with a history of recent facial trauma/fracture/surgery, subjects with bleeding disorders, mucositis (14),(15).

A total of 512 subjects met the inclusion criteria of which 475 subjects provided consent. Of these 2 subjects were excluded as there qRT-PCR was inconclusive.

Interview schedule and sample collection: Data was collected on a pretested and structured schedule that collected information regarding the subject’s age, gender, clinical features and the primary reason for testing. The data elements to be included were developed from the sample referral form developed by Indian Council of Medical Research (ICMR) for collection of qRT-PCR samples (16),(17),(18). The schedule was pretested on 20 subjects. Data from these 20 subjects was excluded from the present study.

Case Definitions Used

Standard case definitions as provided by National Center for Disease Control were used for defining a contact who were then categorised as high risk and low risk based on same case definitions (19),(20).

COVID-19 contact: A contact was defined as any individual who met any one of the three criteria: (1) Stayed in same close environment of a laboratory confirmed COVID-19 patient; (2) Travelled together in close proximity (1m) with a symptomatic person who later tested positive for COVID-19; (3) Provided direct care without proper Personal Protective Equipment (PPE) for COVID-19 patients (20).

High risk contact: A contact who touched body fluids of the patient without PPE, had direct physical contact with the body of patient including physical examination without PPE, touched or cleaned the linens, clothes or dishes of patient, lives in same household as the patient, within 1 metre of confirmed case without precautions (20).

Low risk contact: A contact who did not meet the above criteria was labelled as low risk contact (20).

Procedure

Testing procedure: The subject was made to sit comfortably and a repeat consent taken. A nasopharyngeal swab for qRT-PCR was collected first under proper aseptic procedures and as per the recommended procedure by a trained laboratory technician (15),(21). The swab was sealed in viral transport medium, labelled and stored in a cold chain (2-8^oC) for transportation to the laboratory. A second nasopharyngeal sample was collected by the same technician and the sample was processed for RAT. The test was done as per the manufacturer’s guidelines mentioned in the product inlet. Briefly, the nasopharyngeal swab was put in the buffer medium provided with the kit. The swab was kept in the buffer for 15 seconds. After this the swab was withdrawn while squeezing the sides of buffer tube. The rapid card was then kept on level surface and three drops from the buffer mixture were put in sample well. The results were read after 15 minutes and the same was communicated to the subject. The RAT was done using STANDARD Q COVID-19 (SD Biosensor, Inc. Republic of Korea). STANDARD Q COVID-19 Antigen Test is a rapid chromatographic immunoassay for the qualitative detection of SARS-CoV-2 (19). The result was read as positive, negative, invalid (if no control line was shown). Repeat sampling was done for subjects with invalid tests and then categorised as positive and negative (22). Subjects with a positive RAT were advised to contact the concerned health facilities.

Processing swabs collected for qRT-PCR: All the qRT-PCR samples collected in a single day were sent to the qRT-PCR laboratory at the end of each day. The samples were transported under proper precautions and were processed on same day of collection. A volume of 200 µL was collected from each Viral Transport Medium (VTM) and processed further for Ribonucleic Acid (RNA) extraction (23). Single step RT-PCR for SARS-CoV-2 targeting the E gene was conducted on the sample. A Cycle threshold (Ct) value of less than 35 was reported as positive (24).

Bias: Sample for qRT-PCR was taken before RAT to avoid any bias owing to prior knowledge of COVID-19 status of subjects by technicians. The qRT-PCR Laboratory was not aware of the result on RATs.

Variables: The primary variables collected were basic clinical information and test results for RAT/RT-PCR. Sensitivity, specificity, Positive Predictive Values (PPV) and Negative Predictive Values (NPV) and Cohen’s Kappa were calculated (25)

Main outcome measures:

• Sensitivity and specificity for the RAT kit.
• Cohen’s Kappa score for agreement between the two tests.
• Cycle threshold value of positive samples.

Statistical Analysis

Socio-demographic and clinical profile was described using percentages and mean. Sensitivity, Specificity, PPV and NPV of RAT was calculated using relevant formulas by keeping qRT-PCR as a gold standard. Cohen’s Kappa was calculated. Ct values between two groups were compared by Mann-Whitney U test. The p-value of less than 0.05 was considered to be statistically significant (26).

Results

A total of 475 subjects provided simultaneous samples for both tests of which two samples were reported as rejected in qRT-PCR. Both these subjects had a negative rapid test and were excluded from the final analysis. A total of 473 subjects were included in the final analysis. The selection of study subjects in depicted in (Table/Fig 2). The subjects comprised of 277 (58.6%) males and 196 (41.4) females. The mean age of subjects was 38.4±12.2 years and 57.29% of subjects belonged to urban areas. A total of 124 subjects (26.2%) had symptom at the time of testing. The most common presenting symptom was fever reported by 71 subjects (15.01%). Loss of smell was reported by seven (1.48%) subjects. Total 13.1% subjects had a previous history of COVID-19. The primary reason for testing included a positive contact history 221(46.7%) subjects, symptoms 124(26.2%) and voluntary testing 116 (24.5%). A total of 1/5^th of subjects had any concomitant co-morbidity. The socio-clinical profile of subjects is described in (Table/Fig 3).

Of the 473 subjects, 79 (16.7%) were positive on qRT-PCR of which 43 (54.4%) subjects were positive on RAT as well. Overall positivity rate was 16.7% and 9.7% on qRT-PCR and RAT respectively. These results have been depicted in (Table/Fig 4).

Considering qRT-PCR as the gold standard, the sensitivity and specificity were estimated at 54.43% (42.83%-65.69%) and 99.24% (97.79% to 99.84%), respectively. Positive Likelihood Ratio (PLR), Negative Likelihood Ratio (NLR), PPV and NPV are depicted in (Table/Fig 5). The Cohen’s Kappa between the two was 0.644 (95% CI:0.543-0.745) which depicted a fair level of agreement between the two tests. Authors compared Ct values of subjects positive on qRT-PCR on the basis of their symptoms and their results on Rapid tests. Symptomatic subjects had a significantly lower Ct values than asymptomatic subjects. Similarly subjects positive of RAT had a lower Ct value than those negative on RAT. The median cycle threshold value of rapid antigen-positive subjects was 19 (range 16-30) and was 33 (range 24-35) for RAT negative cases. The median Ct value among symptomatic subjects was significantly lower (20) than asymptomatic subjects (32). The detailed values are depicted in (Table/Fig 6). The mean cycle threshold value of positive subjects was 25.6±6.7 with a range of 16-35. The Ct values were associated with the presence or absence of symptoms but not with the duration of illness (Table/Fig 7).

Discussion

The qRT-PCR is the gold standard test for detection of SARS-CoV-2 in respiratory specimens but its long turnover time and need for sophisticated equipment limit its use. RAT can complement qRT-PCR in the diagnosis of COVID-19 in specific settings. This study evaluated the performance of RAT in comparison with qRT-PCR.

The present study estimated the sensitivity and specificity of RAT to be 54.43% (42.83% to 65.69%) and 99.24 (97.79% to 99.84%), respectively. The overall accuracy was estimated at 91.75%. The sensitivity may appear to be very low but its high specificity coupled with almost instantaneous test report mean that it can augment testing capacity in specific settings. The sensitivity is comparable to a previous study done in Belgium by Lambert-Niclot S et al., and another study by Ristic M et al., in Serbia (27),(28). The sensitivity is also comparable to that estimated for nasopharyngeal swabs by Yamayoshi S et al., (29). Igloi Z et al., conducted a similar study in Netherlands using the same rapid antigen kits and reported a much higher sensitivity of 84% (30). The overall qRT-PCR positivity rate of 16.7% in our study is comparable to the positivity rate of 19% in that study. The higher sensitivity of 84% in their study may have been on account of higher proportion of symptomatic subjects than in their study. Of those positive on qRT-PCR in our study, 56.9% were symptomatic at the time of testing whereas 74% of subjects in the study by Igloi Z et al., were symptomatic (30). Another study conducted by Cerutti F et al., also reported higher sensitivity of 70.6%, the study had included only symptomatic subjects at the time of testing (8). Another previous similar studies conducted in Spain and another one in Uganda have reported higher sensitivity in the range of 70% but those studies had a higher proportion of symptomatic subjects in the sample (31),(32). The sensitivity is higher than one previous study conducted in Brussels and one more study by Lee J et al., (33).

Authors estimated that the STANDARD Q rapid test had a very high specificity. This is comparable to multiple previous studies that also found the specificity to be more than 98% (8),(33),(34). It infers that rapid antigen kits have very less likelihood to give false positive results and a subject with a positive test should be considered positive for SARS-CoV-2. All test kits have to apply for validation before actual use and the regulatory authorities in India have kept minimum acceptance criteria of 50% sensitivity and 95% specificity for point of care tests which are used in a field setting without laboratory support (21). The test kit used in our study met both these criteria.

Cycle threshold (Ct) value for qRT-PCR has been a subject of great debate in recent times. As Ct value refers to the number of replication cycles required for detection of viral RNA, many studies have tried to estimate the clinical implications of Ct values particularly its implications in determining viral loads and clinical severity (35),(36). Of the subjects positive on qRT-PCR in our study, Ct values ranged from 16 to 35. The values were significantly lower for symptomatic subjects in comparison to asymptomatic subjects for SARS-CoV-2 positive subjects. Symptomatic subjects had a median Ct Value of 20 (IQR 18-25) which was significantly lower than the Ct Value of asymptomatic subjects 32 (IQR 30-34). This supports the results from multiple other studies and can be explained that symptomatic patients have a higher viral load (35),(36). This has implications in deciding the level of protection. Studies by Cerutti F et al., and Igloi Z et al., have also found lower Ct values/ higher viral load among symptomatic subjects in comparison to asymptomatic subjects (8),(30). It may also translate to higher infectiousness among symptomatic subjects and therefore enhanced personal protection may be required when dealing with symptomatic patients as compared to asymptomatic patients. Most of the symptomatic subjects were on day 2^nd or 3^rd of their illness with only five symptomatic subjects having an illness duration of six or more days. The duration of illness had no significant relation with Ct values which could be due to a smaller number of subjects with a duration of illness of more than six days.

The present study also found a significantly lower Ct values for subjects positive on RATs as well. The median Ct value of subjects positive on rapid tests was 19 in comparison to 33 for those negative of RAT. Studies by Cerutti F et al., and a study by Igloi Z et al., also found that the sensitivity of rapid tests increases with lower Ct values (8),(30). In other words, subjects who are positive on RAT are more likely to have lower Ct values. These supports the growing body of evidence that lower Ct values means higher viral loads which inturn increase the probability of a positive RAT (27),(28). The strengths of the study included that only a single trained laboratory technician collected the samples and qRT-PCR samples were processed on the same day. Rapid tests and qRT-PCR results were read by different persons to avoid any bias.

Limitation(s)

The major limitation of this study is the lack of a true gold standard as multiple studies have estimated the sensitivity and specificity of qRT-PCR to be between 70-80%. This results in uncertainty in labelling samples as true positives and negatives. One more limitation was the low number of subjects with an illness duration of more than six days which decreased the power of this study to estimate the trend of Cycle threshold value with days of illness.

Conclusion

The STANDARD Q RAT has reasonable sensitivity and high specificity. The two tests have a substantial inter-test agreement. Sensitivity was specifically high in those symptomatic at the time of testing. This can particularly be helpful in early identification followed by isolation/treatment of symptomatic subjects which otherwise can get delayed if only qRT-PCR is available. Using both of these tests together and following up a RAT negative person with qRT-PCR will enhance the overall sensitivity.

Acknowledgement

The authors would like to thank Ms. Gousia (laboratory technician) and other support staff who were involved in sample collection, processing and interpretation of results.

b#bAuthor’s contribution:b?b All the authors contributed to the conceptualization of study design, interpretation of data and the drafting of the article. NK, SA and AJ were also involved in data collection, acquisition and in finalizing the manuscript.

DOI and Others

DOI: 10.7860/JCDR/2022/52286.15825

Date of Submission: Sep 06, 2021
Date of Peer Review: Nov 03, 2021
Date of Acceptance: Dec 01, 2021
Date of Publishing: Jan 01, 2022

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. No

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Sep 07, 2021
• Manual Googling: Oct 29, 2021
• iThenticate Software: Nov 26, 2021 (10%)

ETYMOLOGY: Author Origin