Next Article in Journal
Diagnostic Management of Acute Pulmonary Embolism in COVID-19 and Other Special Patient Populations
Next Article in Special Issue
Association of Prognostic Nutritional Index with Severity and Mortality of Hospitalized Patients with COVID-19: A Systematic Review and Meta-Analysis
Previous Article in Journal
Ultrasound-Guided Trans-Uterine Cavity Core Needle Biopsy of Uterine Myometrial Tumors to Differentiate Sarcoma from a Benign Lesion—Description of the Method and Review of the Literature
Previous Article in Special Issue
SARS-CoV-2 Antigen Test Results to Infer Active or Non-Active Virus Replication Status in COVID-19 Patients
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 12
Issue 6
10.3390/diagnostics12061349
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Comparison of Four Systems for SARS-CoV-2 Antibody at Three Time Points after SARS-CoV-2 Vaccination

by
Jong Do Seo
1,
Minjeong Nam
2,
Tae Hwan Lee
1,
Yeon-Sun Ahn
1,
Seon-Hyeon Shin
1,
Hye Young Han
1 and
Hee-Won Moon
1,*
1
Department of Laboratory Medicine, Konkuk University School of Medicine, Seoul 05030, Korea
2
Department of Laboratory Medicine, Korea University Anam Hospital, Seoul 02841, Korea
*
Author to whom correspondence should be addressed.
Diagnostics 2022, 12(6), 1349; https://doi.org/10.3390/diagnostics12061349
Submission received: 18 May 2022 / Revised: 27 May 2022 / Accepted: 27 May 2022 / Published: 29 May 2022
(This article belongs to the Special Issue Multidisciplinary Approaches to Manage COVID-19: From Surveillance to Diagnosis)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: Immunity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) wanes over time after vaccination. Methods: We compared SARS-CoV-2 antibody levels in serial samples from 350 vaccinated individuals at 3 time points (3 weeks after the first or second dose and before the third dose) with 4 assays: GenScript cPASS SARS-CoV-2 neutralization antibody detection kits (cPASS), Siemens SARS-CoV-2 IgG (sCOVG), Abbott SARS-CoV-2 IgG II Quant (CoV-2 IgG II), and an Immuno-On™ COVID-19 IgG test (Immuno-On IgG). Antibody levels by time, concordance between assays, and values from other tests corresponding to the percent inhibition results in cPASS were assessed. Results: The median values at three time points were 49.31%, 90.87%, and 53.38% inhibition for cPASS, 5.39, 13.65, and 2.24 U/mL for sCOVG, 570.25, 1279.65, and 315.80 AU/mL for CoV-2 IgG II, and 223.22, 362.20, and 62.20 relative units (RU) for Immuno-On IgG. The concordance with cPASS at each time point ranged from 0.735 to 0.984, showing the highest concordance in the second sample and lowest concordance in the third in all comparative tests. The values corresponded to 30% inhibition, and the cutoffs of cPASS, were 2.02 U/mL, 258.6 AU/mL, and 74.2 RU for each test. Those for 50%, 70%, and 90% inhibition were 3.16, 5.66, and 8.26 U/mL for sCOVG, while they were 412.5, 596.9, and 1121.6 AU/mL for CoV-2 IgG II and 141.8, 248.92, and 327.14 RU for Immuno-On IgG. Conclusions: This study demonstrated the dynamic changes in antibody values at different time points using four test systems and is expected to provide useful baseline data for comparative studies and standardization efforts in the future.

1. Introduction

Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was first reported in 2019 and has spread worldwide [1]. Despite the development and introduction of diverse vaccines such as BNT162b2 developed by BioNTech/Pfizer, mRNA-1273 developed by Moderna (Cambridge, MA, USA), and ChAdOx1 nCoV-19 developed by Oxford/AstraZeneca [2], as well as their confirmed efficacy for protection from the disease [3,4,5,6,7,8], the COVID-19 pandemic is still ongoing [1]. The waning of humoral immunity and the decrease in antibodies against SARS-CoV-2 induced by vaccination over time have been revealed in previous studies [9,10,11]. As defense against infection is correlated with the antibody titer [12,13], assays for anti-SARS-CoV-2 antibody detection and quantification can be helpful tools for the evaluation of disease protection. Several laboratory assays using diverse test principles and targeting various antibodies such as RBD-binding or neutralizing antibodies against SARS-CoV-2 have been developed and launched [14]. Tests evaluating virus neutralization using a pseudovirus-based virus neutralization test [15,16] or surrogate virus neutralization test (sVNT) [17,18] directly assess humoral immunity. However, there are restrictions to automation and increasing test throughput due to manual processing and incubation time. Automated immunoassays targeting RBD-binding antibodies have also been developed as alternative indicators of humoral immunity [14,19,20,21,22]. The titer of the binding antibody correlates with the neutralization of the neutralizing antibody [23,24]. Therefore, it is expected to be an alternative indicator for semi-automated and semiquantitative neutralization tests.
In this study, antibody tests were performed in four test systems for serum samples obtained from vaccinated individuals at three time points over a period of 6 months after completion of the second dose. The change in antibody levels over time, the qualitative decision, and the measured values were compared between samples from different time points using each test system. In addition, the quantitative or semiquantitative values from the binding antibody assays that correspond to each point of percent signal inhibition on neutralization test, including 30% signal inhibition of the test cutoff, are estimated in this study.

2. Materials and Method

2.1. Samples

This study recruited 380 subjects who were vaccinated with BNT162b2 or ChAdOx1 nCoV-19 at our institution. All subjects were ≥18 years of age at enrollment. Vaccination was performed according to the manufacturer’s instructions, and blood samples were drawn from each subject at three time points: 3 weeks after the first dose, 3 weeks after the second dose, and before the third dose from March to November 2021. In accordance with the government’s guidelines for vaccination, the dosing interval between the first and second booster doses of BNT162b2 was 3 weeks, and that of ChAdOx1 nCoV-19 ranged from 8 to 12 weeks. The interval between the second and third booster doses of BNT162b2 was 6 months, and that of ChAdOx1 nCoV-19 was 5 months. Among the 380 participants initially included in the study, those whose blood samples were not taken, patients who were diagnosed with COVID-19, and individuals who had received booster vaccination before the third blood collection were excluded from the analysis. A total of 350 subjects remained, and 348, 318, and 264 samples were collected for each respective time point. For cPASS, the indicators of neutralization were confirmed for 183, 314, and 253 samples from each point, which were included for comparison analysis, whereas some samples with sCOVG and CoV-2 IgG II results not paired with cPASS were included in the result distribution analysis only. As described in a previous study [24], the blood samples were collected into Vacuette CAT serum clot activator (Greiner Bio-One, Kremsmunster, Austria) and centrifuged at 1977× g for 10 min, and the serum was aliquoted into two microcentrifuge tubes and stored at −80 °C until measured in four test systems as described below.

2.2. Assays

The samples were evaluated using four test systems. The results of the sVNT—a GenScript cPASS SARS-CoV-2 neutralization antibody detection kit (cPASS; GenScript, USA Inc., Piscataway, NJ, USA) adopting an enzyme-linked immunosorbent assay (ELISA) as a test principle—were used as the standard for virus neutralization in the present study. Siemens SARS-CoV-2 IgG (sCOVG; Siemens Healthcare Diagnostics Inc., Tarrytown, NY, USA) and Abbott SARS-CoV-2 IgG II Quant (CoV-2 IgG II; Abbott Laboratories, Sligo, Ireland) were selected as representative automated quantitative binding antibody immunoassays, with an Immuno-On™ COVID-19 IgG test (Immuno-On IgG; Osang Healthcare Inc., Anyang, Korea) for semiquantitative lateral flow immunoassays. All the tests were performed in accordance with the manufacturer’s instructions.

2.3. Data Analysis

The distribution and change in test results for each system over time were analyzed, and the decisions and values of each system were compared with those of the neutralization test. The concordance of decision with the neutralization test and its 95% confidence interval (CI) for each time point and the entire period was analyzed by diagnostic test evaluation, using the neutralization test as the gold standard. The optimal cutoff to obtain the maximum diagnostic accuracy corresponding to 30%, 50%, 70%, and 90% signal inhibition in cPASS was determined using receiver operating characteristic (ROC) curve analysis by calculating the area under the ROC curve (AUC) and a 95% CI. All statistical analyses were performed using IBM SPSS Statistics for Windows (version 26.0; IBM Corp., Armonk, NY, USA) and MedCalc version 14.8.1 (MedCalc Software, Ostend, Belgium).

3. Results

The seropositive or negative test rates were determined by the manufacturer-claimed cutoff of each test system: ≥30% signal inhibition for cPASS, ≥1.0 U/mL for sCOVG, ≥50.0 AU/mL for CoV-2 IgG II, and ≥14.9 RU for Immuno-On IgG. As shown in Table 1 and Figure 1, the results from the four test systems showed a maximal increase in antibody levels after the second dose of vaccination, which decreased over time. For the cPASS test, the median and interquartile range (IQR) of the percent inhibition was 49.31% (31.79–65.99%) at 3 weeks after the first dose, which then increased to 90.87% (75.66–96.53%) at 3 weeks after the second dose and decreased to 53.38% (31.82–75.18%) before the third dose. The seropositivity rates were 79.8%, 97.8%, and 77.9% at each timepoint. For sCOVG, the median and IQR were 5.39 U/mL (2.28–12.90 U/mL), 13.65 U/mL (6.97–53.17 U/mL), and 2.24 U/mL (1.25–4.94 U/mL), respectively, showing a similar change to the neutralization test, an increase after the second dose, and a decrease over time. The seropositivity rates according to the manufacturer-claimed cutoffs were 90.4%, 99.7%, and 81.3%, respectively. For the CoV-2 IgG II test, the median and IQR were 570.25 AU/mL (252.45–1308.20 AU/mL), 1279.65 AU/mL (714.00–3764.60 AU/mL), and 315.80 AU/mL (181.40–682.50 AU/mL), and the seropositive rates were 96.2%, 99.7%, and 96.6%, respectively. For the Immuno-On IgG test, the median and IQR were 223.22 relative units (RU) (56.71–347.30 RU), 362.20 RU (232.86–456.80 RU), and 62.20 RU (17.39–213.40 RU), and the seropositive rates were 88.0%, 99.4%, and 79.1% at each point, respectively.
The concordance of qualitative decisions between cPASS and other assays at three time points ranged from 0.897 to 0.981 for sCOVG, from 0.814 to 0.981 for CoV-2 IgG II, and from 0.735 to 0.984 for Immuno-On IgG, as shown in Figure 2. The trend in the highest concordance in the second sample and lowest concordance in the third sample was observed in all comparative tests. The concordance with cPASS for all samples from the entire study period was 0.935 for sCOVG, 0.891 for CoV-2 IgG II, and 0.879 for Immuno-On IgG.
In the ROC curve analysis based on the decision of cPASS with a claimed cutoff of 30% inhibition for all samples, the area under the curve (AUC) and its 95% CI were 0.962 (0.945–0.974) for sCOVG, 0.963 (0.947–0.976) for CoV-2 IgG II, and 0.845 (0.817–0.870) for Immuno-On IgG. In contrast to the high sensitivity of 94.8–100% at the claimed cutoff, the specificity showed relatively low values of 19.0–59.0%, as shown in Table 2. The optimal cutoff to obtain the highest diagnostic accuracy was ≥2.02 U/mL for sCOVG, ≥258.6 AU/mL for CoV-2 IgG II, and ≥74.2 RU for Immuno-On IgG, which were much higher than the manufacturer’s suggestion. When the optimal cutoff was applied to the analysis, the sensitivity of the three systems ranged from 80.0% to 88.2%, and the specificity ranged from 80.0% to 95.0%. The increase in the specificity surpassed the decrease in sensitivity, and thus the total diagnostic accuracy improved.
Similar to the values from the binding antibody assays that corresponded to the estimated 30% inhibition of cPASS, the values that corresponded to 50%, 70%, and 90% inhibition were calculated, and the qualitative decision concordance with the AUC in the ROC curve analysis for 30% inhibition and other alternate cutoff values were estimated. The values that corresponded to 50%, 70%, and 90% inhibition were 3.16, 5.66, and 8.26 U/mL for sCOVG, 412.5, 596.9, and 1121.6 AU/mL for CoV-2 IgG II, and 141.8, 248.92, and 327.14 RU for Immuno-On IgG, respectively. The decision concordance following the change in the percent inhibition cutoff is shown in Figure S1, and the AUC and 95% CI of each system, which corresponded to 50%, 70%, and 90% inhibition, were 0.941 (0.922–0.957), 0.911 (0.888–0.930), and 0.936 (0.916–0.953) for sCOVG, 0.944 (0.925–0.959), 0.917 (0.895–0.936), and 0.940 (0.920–0.956) for CoV-2 IgG II, and 0.867 (0.840–0.890), 0.863 (0.836–0.887), and 0.899 (0.875–0.920) for Immuno-On, respectively. The sensitivity and specificity of the binding antibody assays at the claimed or newly estimated cutoff values compared with the diverse cPASS cutoffs are shown in Table S1.

4. Discussion

sVNT is an available option for evaluation of the humoral immune response against SARS-CoV-2 induced by COVID-19 infection or vaccination [17,18]. This test, using ELISA as the test principle, is useful to evaluate virus neutralization and infection defense. However, there are obstacles to automated, massive, and rapid testing due to a long turn-around time and the manual process involved in the test. Therefore, if automated antibody tests that are appropriate for massive and rapid tests can be an alternative indicator of virus neutralization, it will become easier to meet the increasing demand for immunity evaluation. According to previous studies, protection against infection is known to be related to the titer of neutralizing antibodies [9], and the humoral immune response acquired by a vaccine is enhanced by the second dose [10,25]. Then, the titers of the neutralizing and binding antibodies wane over time [9,10,11], just as with immunity acquired through COVID-19 infection or immunity by vaccine for other infectious diseases [26,27]. In the present study, the effect of vaccination on humoral immunity and the change in immune response over time, which were identified by previous studies, were reconfirmed by four test systems adopting different test principles and detecting different targets. The distribution of the measured values showed statistically significant differences between the first, second, and third blood samples in the four test systems. The concordance of decision between the test systems showed the highest concordance in the decision for the second blood samples, which were expected to have the highest antibody titer, and showed the lowest concordance for the third blood samples, which were expected to have the lowest titer. The difference in concordance by the time of sample collection was considered to be due to the difference in antibody titer, and it is believed that the discrepancy largely originated from the difference between the performance of the systems for the decision of the low-concentration samples. The value of this study is that there were few studies showing the assessment of concordance by the time point of sample collection taken from the same subjects. From the early period of test development, there were attempts to evaluate the humoral immunity at a specific time point [28] or serial change over time by comparing multiple tests including our previous study [24]. However, few studies had evaluated the change up to just before the third dose of a vaccine. According to the results of this study, the timing of blood sampling after infection or vaccination and the expected antibody titer should be considered important factors in future studies to evaluate the concordance between antibody assays.
In the ROC curve analysis, each test showed high sensitivity and relatively low specificity when the claimed cutoff was applied, and the claimed cutoff was lower than the optimal cutoff to obtain maximum diagnostic accuracy. Therefore, the claimed cutoff is considered strict to secure high sensitivity and is appropriate for detecting seropositive subjects in a population with a high seropositive rate. Meanwhile, because of the high false-positive rate predicted in populations with low seropositive rates due to relatively low specificity, there are limitations on the use of seronegative subjects [14,22,29]. Since there is controversy about the standard for evaluating virus neutralization [13,30], the cutoff is not an absolute value but is changeable, similar to the change in the claimed cutoff for cPASS from 20–30% [24]. The values from other tests have been estimated in this study, corresponding to each point of signal inhibition in sVNT. The predicted values corresponding to 30% inhibition—2.02 U/mL for sCOVG and 258.6 AU/mL for CoV-2 IgG II—were slightly lower than those of our previous study—2.42 U/mL and 284.0 AU/mL [24]—estimated by the samples from only the first and second time points. The inclusion of samples from the third time point with low antibody titer and a relatively large decrease in binding antibodies compared with that of neutralizing antibodies are thought to be the cause of these results. As the cutoff of sVNT increased from 30% to 90% inhibition, more “positive” results have been reclassified as “negative,” and the concordance of decision with other tests with fixed cutoffs has decreased.
Through this study, high qualitative concordance with the neutralization test was confirmed in all the test systems included in present study, and the availability of an anti-SARS-CoV-2 antibody automation test and lateral flow immunochromatography as alternative indicators of the neutralization test has been evaluated by calculating the measured values in these platforms that correspond to 30%, 50%, 70%, and 90% signal inhibition in sVNT. It is considered that the relatively low concordance and AUC value in flow immunochromatography, compared with that of the quantitative binding antibody assays, are due to the characteristics of the test design, which was developed as a semiquantitative assay.
The present study has some limitations. First, it was impossible to include seronegative subjects as counterparts to the vaccinated subjects in the study design because of the continuous increase in the vaccination rate. As a result, a biased distribution of the decision occurred, and the Cohen–Kappa coefficient, assuming an even distribution between decisions, was affected [31]. Therefore, the coefficient for the comparison between test methods became meaningless and unavailable, and it was then excluded from the analysis. Second, the medical information of individuals could not be reviewed, because informed consent does not include the right to access the medical records of the subjects. Thus, data that were inappropriate for analysis due to COVID-19 infection after vaccination or an inaccurate booster vaccination schedule could not be completely excluded. Lastly, since this study was designed to include only vaccinated individuals, the possibility of limited distribution in humoral immunity or antibody titer cannot be excluded, despite the fact that individuals who received different vaccines such as BNT162b2 or ChAdOx1 nCoV-19, which are known for different antibody titer distributions, were enrolled for analysis. Therefore, it is necessary to note that there may be limitations in applying the results of the present study to the interpretation of test results in the clinical setting.
Despite these limitations, to the best of our knowledge, this is the first study to evaluate the dynamic changes in the humoral immune response against COVID-19 at three time points using four test systems. Therefore, this study is expected to provide useful baseline data for comparative study in the future. Moreover, these data could be helpful for the interpretation of humoral immunity using each binding antibody assay by showing the values of each test corresponding to various levels of percent inhibition in the neutralization test. Efforts to standardize the values of the diverse antibody tests are required in future studies.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/diagnostics12061349/s1. Figure S1: Concordance and 95% CI of qualitative decision of each test system with cPASS following the change in % inhibition cutoff for neutralization, (A) when the manufacturer-claimed cutoff for each test was applied or (B) the estimated cutoff for each test corresponding to change of neutralization cutoff was applied. If there is no overlap of 95% CIs, it can be interpreted as having a statistically significant difference at the significance level of 0.05.; Table S1: The sensitivity and specificity of each test system at manufacturer-claimed cutoff and estimated optimal cutoff by the alteration of cPASS cutoff.

Author Contributions

J.D.S. collected and analyzed the data and wrote the initial draft. H.-W.M. designed and supervised the study and finalized the draft. M.N. collected and analyzed the data. T.H.L., Y.-S.A., S.-H.S. and H.Y.H. collected the samples and performed the tests. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted after obtaining approval from the Institutional Review Board of our institution (IRB No. KUMC 2021-05-033; 17 June 2021).

Informed Consent Statement

All subjects were ≥18 years of age at enrollment, and written informed consent was obtained.

Data Availability Statement

A data set of serological responses of 930 samples was deposited at https://dataverse.harvard.edu/ (accessed on 25 February 2022) (https://doi.org/10.7910/DVN/CMRD77 accessed on 25 February 2022).

Acknowledgments

Siemens Healthcare Diagnostics Inc., Abbott Laboratories, and Osang Healthcare Inc. showed their support through the provision of anti-SARS-CoV-2 IgG test kits. Siemens, Abbott, and Osang did not play any role in the study design or data analysis.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. WHO. COVID-19 Weekly Epidemiological Update. Available online: https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---6-january-2022 (accessed on 10 January 2022).
  2. WHO. Status of COVID-19 Vaccines within WHO EUL/PQ Evaluation Process. Available online: https://extranet.who.int/pqweb/news/status-covid-19-vaccines-within-who-eulpq-evaluation-process-updated-1 (accessed on 10 January 2022).
  3. Anderson, E.J.; Rouphael, N.G.; Widge, A.T.; Jackson, L.A.; Roberts, P.C.; Makhene, M.; Chappell, J.D.; Denison, M.R.; Stevens, L.J.; Pruijssers, A.J.; et al. Safety and Immunogenicity of SARS-CoV-2 mRNA-1273 Vaccine in Older Adults. N. Engl. J. Med. 2020, 383, 2427–2438. [Google Scholar] [CrossRef] [PubMed]
  4. Walsh, E.E.; Frenck, R.W., Jr.; Falsey, A.R.; Kitchin, N.; Absalon, J.; Gurtman, A.; Lockhart, S.; Neuzil, K.; Mulligan, M.J.; Bailey, R.; et al. Safety and Immunogenicity of Two RNA-Based Covid-19 Vaccine Candidates. N. Engl. J. Med. 2020, 383, 2439–2450. [Google Scholar] [CrossRef] [PubMed]
  5. Polack, F.P.; Thomas, S.J.; Kitchin, N.; Absalon, J.; Gurtman, A.; Lockhart, S.; Perez, J.L.; Perez Marc, G.; Moreira, E.D.; Zerbini, C.; et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N. Engl. J. Med. 2020, 383, 2603–2615. [Google Scholar] [CrossRef] [PubMed]
  6. Regev-Yochay, G.; Amit, S.; Bergwerk, M.; Lipsitch, M.; Leshem, E.; Kahn, R.; Lustig, Y.; Cohen, C.; Doolman, R.; Ziv, A.; et al. Decreased infectivity following BNT162b2 vaccination: A prospective cohort study in Israel. Lancet Reg. Health Eur. 2021, 7, 100150. [Google Scholar] [CrossRef]
  7. Voysey, M.; Clemens, S.A.C.; Madhi, S.A.; Weckx, L.Y.; Folegatti, P.M.; Aley, P.K.; Angus, B.; Baillie, V.L.; Barnabas, S.L.; Bhorat, Q.E.; et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: An interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2021, 397, 99–111. [Google Scholar] [CrossRef]
  8. Haas, E.J.; Angulo, F.J.; McLaughlin, J.M.; Anis, E.; Singer, S.R.; Khan, F.; Brooks, N.; Smaja, M.; Mircus, G.; Pan, K.; et al. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: An observational study using national surveillance data. Lancet 2021, 397, 1819–1829. [Google Scholar] [CrossRef]
  9. Bergwerk, M.; Gonen, T.; Lustig, Y.; Amit, S.; Lipsitch, M.; Cohen, C.; Mandelboim, M.; Levin, E.G.; Rubin, C.; Indenbaum, V.; et al. Covid-19 Breakthrough Infections in Vaccinated Health Care Workers. N. Engl. J. Med. 2021, 385, 1474–1484. [Google Scholar] [CrossRef]
  10. Shrotri, M.; Navaratnam, A.M.D.; Nguyen, V.; Byrne, T.; Geismar, C.; Fragaszy, E.; Beale, S.; Fong, W.L.E.; Patel, P.; Kovar, J.; et al. Spike-antibody waning after second dose of BNT162b2 or ChAdOx1. Lancet 2021, 398, 385–387. [Google Scholar] [CrossRef]
  11. Dan, J.M.; Mateus, J.; Kato, Y.; Hastie, K.M.; Yu, E.D.; Faliti, C.E.; Grifoni, A.; Ramirez, S.I.; Haupt, S.; Frazier, A.; et al. Immunological memory to SARS-CoV-2 assessed for up to 8 months after infection. Science 2021, 371, eabf4063. [Google Scholar] [CrossRef]
  12. Vanshylla, K.; Di Cristanziano, V.; Kleipass, F.; Dewald, F.; Schommers, P.; Gieselmann, L.; Gruell, H.; Schlotz, M.; Ercanoglu, M.S.; Stumpf, R.; et al. Kinetics and correlates of the neutralizing antibody response to SARS-CoV-2 infection in humans. Cell Host Microbe 2021, 29, 917–929.e914. [Google Scholar] [CrossRef]
  13. Khoury, D.S.; Cromer, D.; Reynaldi, A.; Schlub, T.E.; Wheatley, A.K.; Juno, J.A.; Subbarao, K.; Kent, S.J.; Triccas, J.A.; Davenport, M.P. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection. Nat. Med. 2021, 27, 1205–1211. [Google Scholar] [CrossRef] [PubMed]
  14. Indenbaum, V.; Koren, R.; Katz-Likvornik, S.; Yitzchaki, M.; Halpern, O.; Regev-Yochay, G.; Cohen, C.; Biber, A.; Feferman, T.; Cohen Saban, N.; et al. Testing IgG antibodies against the RBD of SARS-CoV-2 is sufficient and necessary for COVID-19 diagnosis. PLoS ONE 2020, 15, e0241164. [Google Scholar] [CrossRef] [PubMed]
  15. Legros, V.; Denolly, S.; Vogrig, M.; Boson, B.; Siret, E.; Rigaill, J.; Pillet, S.; Grattard, F.; Gonzalo, S.; Verhoeven, P.; et al. A longitudinal study of SARS-CoV-2-infected patients reveals a high correlation between neutralizing antibodies and COVID-19 severity. Cell Mol. Immunol. 2021, 18, 318–327. [Google Scholar] [CrossRef] [PubMed]
  16. Bewley, K.R.; Coombes, N.S.; Gagnon, L.; McInroy, L.; Baker, N.; Shaik, I.; St-Jean, J.R.; St-Amant, N.; Buttigieg, K.R.; Humphries, H.E.; et al. Quantification of SARS-CoV-2 neutralizing antibody by wild-type plaque reduction neutralization, microneutralization and pseudotyped virus neutralization assays. Nat. Protoc. 2021, 16, 3114–3140. [Google Scholar] [CrossRef] [PubMed]
  17. Tan, C.W.; Chia, W.N.; Qin, X.; Liu, P.; Chen, M.I.; Tiu, C.; Hu, Z.; Chen, V.C.; Young, B.E.; Sia, W.R.; et al. A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction. Nat. Biotechnol. 2020, 38, 1073–1078. [Google Scholar] [CrossRef] [PubMed]
  18. Valcourt, E.J.; Manguiat, K.; Robinson, A.; Chen, J.C.; Dimitrova, K.; Philipson, C.; Lamoureux, L.; McLachlan, E.; Schiffman, Z.; Drebot, M.A.; et al. Evaluation of a commercially-available surrogate virus neutralization test for severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). Diagn. Microbiol. Infect. Dis. 2021, 99, 115294. [Google Scholar] [CrossRef]
  19. Kim, Y.; Lee, J.H.; Ko, G.Y.; Ryu, J.H.; Jang, J.H.; Bae, H.; Yoo, S.H.; Choi, A.R.; Jung, J.; Lee, J.; et al. Quantitative SARS-CoV-2 Spike Antibody Response in COVID-19 Patients Using Three Fully Automated Immunoassays and a Surrogate Virus Neutralization Test. Diagnostics 2021, 11, 1496. [Google Scholar] [CrossRef]
  20. Meyer, B.; Torriani, G.; Yerly, S.; Mazza, L.; Calame, A.; Arm-Vernez, I.; Zimmer, G.; Agoritsas, T.; Stirnemann, J.; Spechbach, H.; et al. Validation of a commercially available SARS-CoV-2 serological immunoassay. Clin. Microbiol. Infect. 2020, 26, 1386–1394. [Google Scholar] [CrossRef]
  21. Espejo, A.P.; Akgun, Y.; Al Mana, A.F.; Tjendra, Y.; Millan, N.C.; Gomez-Fernandez, C.; Cray, C. Review of Current Advances in Serologic Testing for COVID-19. Am. J. Clin. Pathol. 2020, 154, 293–304. [Google Scholar] [CrossRef]
  22. Gudbjartsson, D.F.; Norddahl, G.L.; Melsted, P.; Gunnarsdottir, K.; Holm, H.; Eythorsson, E.; Arnthorsson, A.O.; Helgason, D.; Bjarnadottir, K.; Ingvarsson, R.F.; et al. Humoral Immune Response to SARS-CoV-2 in Iceland. N. Engl. J. Med. 2020, 383, 1724–1734. [Google Scholar] [CrossRef]
  23. Mendrone-Junior, A.; Dinardo, C.L.; Ferreira, S.C.; Nishya, A.; Salles, N.A.; de Almeida Neto, C.; Hamasaki, D.T.; Facincani, T.; de Oliveira Alves, L.B.; Machado, R.R.G.; et al. Correlation between SARS-COV-2 antibody screening by immunoassay and neutralizing antibody testing. Transfusion 2021, 61, 1181–1190. [Google Scholar] [CrossRef] [PubMed]
  24. Nam, M.; Seo, J.D.; Moon, H.W.; Kim, H.; Hur, M.; Yun, Y.M. Evaluation of Humoral Immune Response after SARS-CoV-2 Vaccination Using Two Binding Antibody Assays and a Neutralizing Antibody Assay. Microbiol. Spectr. 2021, 9, e0120221. [Google Scholar] [CrossRef] [PubMed]
  25. Chodick, G.; Tene, L.; Rotem, R.S.; Patalon, T.; Gazit, S.; Ben-Tov, A.; Weil, C.; Goldshtein, I.; Twig, G.; Cohen, D.; et al. The effectiveness of the TWO-DOSE BNT162b2 vaccine: Analysis of real-world data. Clin. Infect. Dis. 2021, 74, 472–478. [Google Scholar] [CrossRef] [PubMed]
  26. De Carlo, A.; Lo Caputo, S.; Paolillo, C.; Rosa, A.M.; D’Orsi, U.; De Palma, M.; Reveglia, P.; Lacedonia, D.; Cinnella, G.; Foschino, M.P.; et al. SARS-COV-2 Serological Profile in Healthcare Professionals of a Southern Italy Hospital. Int. J. Environ. Res. Public Health 2020, 17, 9324. [Google Scholar] [CrossRef]
  27. Saffar, H.; Mousavi, S.J.; Saffar, H.; Parsaei, M.R.; Ghorbani, G.R.; Saffar, M.J. Seroconversion rates following 2 doses of measles- mumps- rubella vaccination given at the ages 12 and 18 months: Data for possible additional dose at older age. BMC Immunol. 2022, 23, 2. [Google Scholar] [CrossRef]
  28. The National SARS-CoV-2 Serology Assay Evaluation Group. Performance characteristics of five immunoassays for SARS-CoV-2: A head-to-head benchmark comparison. Lancet Infect. Dis. 2020, 20, 1390–1400. [Google Scholar] [CrossRef]
  29. Alter, G.; Seder, R. The Power of Antibody-Based Surveillance. N. Engl. J. Med. 2020, 383, 1782–1784. [Google Scholar] [CrossRef]
  30. von Rhein, C.; Scholz, T.; Henss, L.; Kronstein-Wiedemann, R.; Schwarz, T.; Rodionov, R.N.; Corman, V.M.; Tonn, T.; Schnierle, B.S. Comparison of potency assays to assess SARS-CoV-2 neutralizing antibody capacity in COVID-19 convalescent plasma. J. Virol. Methods 2021, 288, 114031. [Google Scholar] [CrossRef]
  31. Feinstein, A.R.; Cicchetti, D.V. High agreement but low kappa: I. The problems of two paradoxes. J. Clin. Epidemiol. 1990, 43, 543–549. [Google Scholar] [CrossRef]
Diagnostics 12 01349 g001 550
Figure 1. The measured results of the samples for each test system: (A) percent inhibition for cPASS, (B) U/mL for sCOVG, (C) AU/mL for CoV-2 IgG II, and (D) RU for Immuno-On IgG, presented as box and whisker plots. The horizontal line and the box represent the median and interquartile range, respectively, whereas the circle and inverted triangles represent near and far outliers confirmed by Tukey’s method. The results for sCOVG and CoV-2 IgG II are presented as log scales.
Figure 1. The measured results of the samples for each test system: (A) percent inhibition for cPASS, (B) U/mL for sCOVG, (C) AU/mL for CoV-2 IgG II, and (D) RU for Immuno-On IgG, presented as box and whisker plots. The horizontal line and the box represent the median and interquartile range, respectively, whereas the circle and inverted triangles represent near and far outliers confirmed by Tukey’s method. The results for sCOVG and CoV-2 IgG II are presented as log scales.
Diagnostics 12 01349 g001
Diagnostics 12 01349 g002 550
Figure 2. Concordance and 95% CI of qualitative decision between test systems for serum samples collected at (A) 3 weeks after the first dose, (B) 3 weeks after the second dose, (C) 6 months after the second dose, and (D) the whole samples. If there is no overlap of 95% CIs, it can be interpreted as having a statistically significant difference at a significant level of 0.05 (5%).
Figure 2. Concordance and 95% CI of qualitative decision between test systems for serum samples collected at (A) 3 weeks after the first dose, (B) 3 weeks after the second dose, (C) 6 months after the second dose, and (D) the whole samples. If there is no overlap of 95% CIs, it can be interpreted as having a statistically significant difference at a significant level of 0.05 (5%).
Diagnostics 12 01349 g002
Table
Table 1. The median and interquartile range of measured value and decision on humoral immunity over time measured by multiple test systems.
Table 1. The median and interquartile range of measured value and decision on humoral immunity over time measured by multiple test systems.
SystemSamplenMedian and IQRDecision *
PositiveNegative
cPASSFirst18349.31 (31.79–65.99)79.8% (146/183)20.2% (37/183)
(% inhibition)Second31590.87 (75.66–96.53)97.8% (308/315)2.2% (7/315)
Third25353.38 (31.82–75.18)77.9% (197/253)22.1% (56/253)
sCOVGFirst3445.39 (2.28–12.90)90.4% (311/344)9.6% (33/344)
(U/mL)Second31513.65 (6.97–53.17)99.7% (314/315)0.3% (1/315)
Third2622.24 (1.25–4.94)81.3% (213/262)18.7% (49/262)
CoV-2 IgG IIFirst344570.25 (252.45–1308.20)96.2% (331/344)3.8% (13/344)
(AU/mL)Second3161279.65 (714.00–3764.60)99.7% (315/316)0.3% (1/316)
Third262315.80 (181.40–682.50)96.6% (253/262)3.4% (9/262)
Immuno-On IgGFirst183223.22 (56.71–347.30)88.0% (161/183)12.0% (22/183)
(RU)Second315362.20 (232.86–456.80)99.4% (313/315)0.6% (2/315)
Third25362.20 (17.38–213.40)79.1% (200/253)20.9% (53/253)
The decisions were determined by the manufacturer-claimed cutoff of each test system: ≥30% signal inhibition for cPASS, ≥1.0 U/mL for sCOVG, ≥50.0 AU/mL for CoV-2 IgG II, and ≥14.9 RU for Immuno-On IgG. * The seropositive or negative rates are presented as percentages with the number of subjects in parentheses. Abbreviations: IQR = interquartile range; cPASS = GenScript cPASS SARS-CoV-2 neutralization antibody detection kit; sCOVG = Siemens SARS-CoV-2 IgG; CoV-2 IgG II = Abbott SARS-CoV-2 IgG II Quant; Immuno-On IgG = Osang Immuno-On™ COVID-19 IgG test; RU = relative units.
Table
Table 2. The sensitivity and specificity of each test system with trade-offs in the cutoff point, compared with the cPASS decision with a cutoff of 30% inhibition.
Table 2. The sensitivity and specificity of each test system with trade-offs in the cutoff point, compared with the cPASS decision with a cutoff of 30% inhibition.
AUC (95% CI *)CutoffTrade-offSensitivity (95% CI)Specificity (95% CI)
sCOVG0.962 (0.945–0.974)Claimed≥1.0 U/mL98.6 (97.4–99.4)59.0 (48.7–68.7)
Optimal≥2.02 U/mL87.2 (84.4–89.7)94.0 (87.4–97.8)
CoV-2 IgG II0.963 (0.947–0.976)Claimed≥50.0 AU/mL100.0 (99.4–100.0)19.0 (11.8–28.1)
Optimal≥258.6 AU/mL88.2 (85.4–90.6)95.0 (88.7–98.4)
Immuno-On IgG0.845 (0.817–0.870)Claimed≥14.9 RU94.8 (92.8–96.4)44.0 (34.1–54.3)
Optimal≥74.2 RU80.0 (76.7–83.0)80.0 (70.8–87.3)
The sensitivity and specificity of the test systems at each cutoff point were determined based on the decision of cPASS with a cutoff of 30% inhibition. * If there is no overlap in the 95% CIs between parameters, it can be interpreted as having a statistically significant difference at the significance level of 0.05 (5%). Abbreviations: AUC = area under the curve; CI = confidence interval; cPASS = GenScript cPASS SARS-CoV-2 neutralization antibody detection kit; sCOVG = Siemens SARS-CoV-2 IgG; CoV-2 IgG II = Abbott SARS-CoV-2 IgG II Quant; Immuno-On IgG = Osang Immuno-On™ COVID-19 IgG test.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Seo, J.D.; Nam, M.; Lee, T.H.; Ahn, Y.-S.; Shin, S.-H.; Han, H.Y.; Moon, H.-W. Comparison of Four Systems for SARS-CoV-2 Antibody at Three Time Points after SARS-CoV-2 Vaccination. Diagnostics 2022, 12, 1349. https://doi.org/10.3390/diagnostics12061349

AMA Style

Seo JD, Nam M, Lee TH, Ahn Y-S, Shin S-H, Han HY, Moon H-W. Comparison of Four Systems for SARS-CoV-2 Antibody at Three Time Points after SARS-CoV-2 Vaccination. Diagnostics. 2022; 12(6):1349. https://doi.org/10.3390/diagnostics12061349

Chicago/Turabian Style

Seo, Jong Do, Minjeong Nam, Tae Hwan Lee, Yeon-Sun Ahn, Seon-Hyeon Shin, Hye Young Han, and Hee-Won Moon. 2022. "Comparison of Four Systems for SARS-CoV-2 Antibody at Three Time Points after SARS-CoV-2 Vaccination" Diagnostics 12, no. 6: 1349. https://doi.org/10.3390/diagnostics12061349

APA Style

Seo, J. D., Nam, M., Lee, T. H., Ahn, Y. -S., Shin, S. -H., Han, H. Y., & Moon, H. -W. (2022). Comparison of Four Systems for SARS-CoV-2 Antibody at Three Time Points after SARS-CoV-2 Vaccination. Diagnostics, 12(6), 1349. https://doi.org/10.3390/diagnostics12061349

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop