As expected, there was considerable discordance between the eight ELISAs. We tested 89 samples from patients and controls on all eight ELISAs. Of the complete set of serum samples, 35/89 (39%) were negative in all assays, while 16/89 (18%) were positive in all assays. The remaining 38/89 (43%) samples were positive in one to seven ELISAs.
In the 59 patients that were suspected of Borrelia infection, we observed a wide range of positive results, with percentages of positive ELISAs varying between 34 and 61% (Table 1). We did not observe a relation between the fraction of positive results and the nature of antigen used for the ELISA. The specificity of the ELISAs also varied widely. Although we had only small numbers of positive tests in healthy controls, some ELISAs produced up to 38% of positive tests in the cross-reactivity group (syphilis and M. pneumonia-infected patients).
We aggregated results from the IgM and IgG tests and assessed them using a kappa statistic to determine agreement between the ELISAs. The kappa values ranged from 0.41 (moderate agreement) to 0.79 (substantial to good agreement), emphasising the differences between the ELISAs (Table 2). The choice of antigen does not seem to influence the level of agreement. Even the lowest kappa values were observed between two ‘whole-cell + VlsE’ ELISAs (0.43).
We tested a subset of 31 serum samples from patients suspected of Borrelia infection in all five immunoblots. Samples were from patients with positive and negative ELISA results, allowing us to investigate the specificity of the immunoblots. In general, we observed a much lower agreement for the immunoblots than for the ELISAs. Kappa values ranged from 0 (poor agreement) to 0.84 (good agreement), indicating that, for many samples, the outcome of the immunoblot is highly dependent on the choice of manufacturer (Table 3). Inter-blot agreement was disappointingly low for IgM and much higher for IgG (Table 3). Interestingly, recombinant blots did not have a higher agreement than whole-cell blots, and there was limited agreement even between recombinant blots. The highest agreement was for the home-made whole-cell blot with the Mikrogen recombinant blot. Additional analysis on the individual band level revealed similarly poor agreement, even in immunoblots containing recombinant antigens.
When performing eight different ELISAs and five different blots, there are 40 possible ELISA–blot combinations. Thirty-one samples were tested in all 40 combinations. A score of 0 indicates a negative result in all ELISAs and all blots, while a score of 40 indicates a positive result in all ELISAs and all blots. A score between 0 and 40 indicates that not all possible combinations yielded a positive result (i.e. disagreement between various ELISA–blot combinations). Of this small sample cohort, 20/31 (65%) had either a score of 0 or 40, indicating perfect agreement, irrespective of the ELISA–blot combination used. Discordant interpretations were generated in the other 35% of samples.
The influence of assay choice is further illustrated by investigation of the relationship between each ELISA and the fraction of positive blots. Surprisingly, we found anti-Borrelia immunoblot reactivity in samples that were negative in all eight ELISAs. These are samples that normally would not have been tested in immunoblots. Again, this was not dependent on the nature of the antigen used for the immunoblot. For the Euroimmun immunoblot, 4/11 (36%) of the ELISA-negative samples were blot-positive. Some immunoblots also seem to lack sensitivity, since samples that were positive in six to all eight of the tested ELISAs remained negative in all immunoblots. Some of these samples were from Lyme disease patients with a short duration of symptoms, confirming that ELISAs may have a higher sensitivity than immunoblots during the early phase of a Borrelia infection.
For some ELISA–blot combinations, only about half of the ELISA-positive samples could be confirmed by immunoblot (e.g. VIDAS ELISA–Virotech immunoblot, Table 4). The quality of the other ELISAs was so high that the majority of ELISA-positive samples were confirmed with immunoblots (e.g. Diacheck/Moran and Enzygnost ELISAs). When taking into account the lack of specificity of a number of the immunoblots, it is clear that the combination of a non-specific ELISA with a non-specific blot will lead to a high fraction of presumably false-positive test results.
The ELISA test value is the final factor influencing the fraction of positive confirmatory blots. Figure 1 depicts an example—values for the VIDAS and Immunetics C6 Lyme ELISA according to the immunoblot results of a whole-cell blot (home-made) and a recombinant blot (Mikrogen). For the VIDAS–home-made blot combination, it is difficult to indicate a cut-off value for the VIDAS ELISA with a good separation between blot-positives and blot-negatives. When using the Immunetics ELISA as a screening tool, it becomes clear that, irrespective of the blot method used, samples with an index >4 are almost always blot-positive. These characteristics make it possible to define groups of ELISA-positive serum samples that do not need immunoblot confirmation.