The data that support the findings of this study are available from the corresponding author upon reasonable request and New York University Langone Health Institutional Review Board committee approval.The development and testing of AlphaMissesne have been reported elsewhere.9 Briefly, AlphaMissense predicts the pathogenicity of all possible amino acid (AA) changes in all known protein sequences. The model is trained by using an unsupervised learning approach, implementing a language model architecture. In an initial pretraining step, deep complex representations of protein sequence data are built by masking and unmasking AAs at random along a given sequence. By training the model to predict the masked AAs, the model learns the fundamental properties of the human proteome and the compatibility of any AA in any position. In the second step, the model is fine‐tuned on a set of variants that are labeled on the basis of either being highly frequent or completely absent from human and primate populations. To allow easy interaction with AlphaMissense predictions, our team, led by the senior author (L.J.), built VarCard.io, a free nonprofit web engine that allows query of all missense variants by gene name and cDNA or AA change and extracts the AlphaMissense annotation of each variant as either likely benign (LB), likely pathogenic (LP), or ambiguous. In addition, VarCard.io extracts the gene‐disease correlation for the queried gene, as assessed by ClinGen.All patients with missense variants found in probands presenting to the New York University Inherited Arrhythmia program were included. All missense variants found in each patient were included. We compared the annotation of AlphaMissense via VarCard.io to the ClinVar11 database, as well as to the annotation of an independent commercial classification engine, Franklin12 (Genoox, Palo Alto, CA). ClinVar annotations were recorded as per American College of Medical Genetics and Genomics guidelines to be either pathogenic, LP, VUS, benign, or LB. If the variant did not appear on the ClinVar database, it was treated in this study as a VUS unless otherwise specified. If a variant had a mixed classification (ie, >1 entry in ClinVar), it was treated as a VUS if at least 1 annotation was a VUS.All genes in the database were assessed using the ClinGen13 framework (ClinGen, National Institutes of Health, Bethesda, MD) as to their known gene–disease correlation. Patient records were reviewed, and the clinical phenotype was assessed by using all clinical data available, including patient's history, ECG tracings, provocative tests, and imaging (eg, echocardiography, cardiac magnetic resonance imaging, calcium pyrophosphate imaging). The analysis was done by 2 physicians trained in assessing cardiogenetic cases (A.H. and M.C.) and the senior author (L.J.), who is the director of our institution's cardiogenetic program. All decisions were reached via consensus of all 3 assessors, and the phenotype analysis was performed before the collection of VarCard.io's results. The genotype–phenotype correlation was considered as positively concordant if (1) a patient had a variant in a gene with known gene–disease correlation compatible with the designated clinical phenotype for this gene by ClinGen, and (2) the variant was classified as pathogenic/LP (positive concordance). Alternatively, negative concordance was determined if (1) a variant was found in a gene not compatible with the patient's clinical phenotype by ClinGen, and (2) the variant was classified as benign/LB. Otherwise, the genotype–phenotype was considered as discordant (ie, a patient with a pathogenic/LP variant in a gene not compatible with the patients' phenotype). Patients were excluded from analysis if their phenotype was indeterminable due to missing data. Patients included in the study provided written informed consent before inclusion, which was approved by the New York University Langone Health Institutional Review Board committee in accordance with the Helsinki Declaration.
The KCNH2 gene, whose channel (HERG) functional characteristics are well studied, was taken as a case study. Previously published results14, 15 of the KCNH2 variant's channel current in comparison with wild type was compared with the VarCard.io results to assess Varcard.io's ability to predict the functional implications of an KCNHN2 variant.
Discrete variables are reported as numbers and rates, and 95% CIs are calculated using the Clopper–Pearson method. The statistical significance of rate differences was assessed using the χ2 test. Results were considered significant when P values were <0.05. All calculations were done using R version 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria).
Results
We reviewed 623 probands in our original database, of which 266 patients were identified as meeting the inclusion criteria of having at least 1 genetic variant that was classified as a missense variant in a gene recognized to be associated with a cardiac phenotype by ClinGen (Figure 1). These patients had a total of 339 different genetic variants. The most common phenotype was long QT syndrome, followed by nonischemic cardiomyopathy and hypertrophic cardiomyopathy (Figure 1). The most frequently involved genes were SCN5A (implicated in Brugada syndrome, dilated cardiomyopathy, and long QT syndrome), followed by KCNQ1, KCNH2, and RyR2 (Figure 2).
BrS indicates Brugada syndrome; CPVT, catecholaminergic polymorphic ventricular tachycardia; HCM, hypertrophic cardiomyopathy; LQTS, long QT syndrome; NICMP, nonischemic cardiomyopathy; PCCD, progressive cardiac conduction defect; and SQTS, short QT syndrome.
LB indicates likely benign; LP, likely pathogenic; and VUS, variant of unknown significance.
Of the 339 variants tested in VarCard.io, 328 variants had ClinVar entries, of which 110 (33.5%) variants had a clinically significant variant annotation of pathogenicity (ie, pathogenic, LP, benign, or LB without VUS entries), and the rest were classified as VUS or had mixed classification (Table).
Variant Classification According to ClinVar
CSV indicates clinically significant variant; LB, likely benign; LP, likely pathogenic; and VUS, variant of unknown significance.
For the 230 (67.8%) VUSs (ie, variants with at least 1 ClinVar entry as VUS or not reported), 198 variants were reclassified by VarCard.io as either LP or LB, yielding a reclassification rate of 86.1% (95% CI, 80.9–90.3). These reclassifications were to LB in 111 variants (56.1% [95% CI, 48.8–63.1]) and to LP in 87 variants (43.9% [95% CI, 36.9–51.2]; Figure 3). Only 32 variants (13.9%) had an ambiguous annotation by VarCard.io. Of the 10 most common genes in the cohort, DSP gene variants and ANK2 gene variants had the highest reclassification rate (100%), and KCNH2 and KCNQ1 had the lowest reclassification rates of 37.5% and 18.8%, respectively (Table S1).
VUS indicates variant of unknown significance.
Of the 109 variants with a CSV ClinVar annotation (ie, variants without any VUS entry), 105 remained a CSV, and the other 4 (3.7%) were annotated as ambiguous by VarCard.io. The overall agreement rate between ClinVar and VarCard.io on CSVs (ie, same annotation in both systems) was 90.5% ([95% CI, 83.2–95.3]). This agreement rate was similar for 81 variants with ClinVar pathogenic/PL annotation and for 24 variants with ClinVar benign/LB annotation (91.4% and 87.5%, respectively; P=0.865).
Of the 303 variants with a CSV annotation by VarCard.io, 269 variants were related to a patient with a defined phenotype. Of these, 258 variants (95.9% [95% CI, 92.8–97.9]) had a concordant genotype–phonotype prediction: 138 (96.5% [95% CI, 92–98.9]) with positive concordance and 120 (95.2% [95% CI, 89.9–98.2]) with negative concordance. There were 34 variants (11.2%) in patients without sufficient clinical data to establish a defined phenotype and thus were not included in this analysis. For the significantly smaller group of patients with CSV annotation on ClinVar, of 69 patients with sufficient clinical data to establish a defined phenotype, 61 (88.4% [95% CI, 78.4–94.8]) had concordant genotype–phonotype prediction, and 8 (11.5% [95% CI, 5.1–21.6]) had a discordant prediction.
Using Franklin on the 230 VUS variants appearing in ClinVar resulted in reclassification to either pathogenic/LP or benign/LB in only 80 variants, giving a reclassification rate of 34.8% (95% CI, 28.6–41.3), significantly lower than the 86.1% (95% CI, 80.9–90.3) VarCard.io reclassification rate (P<0.001). For the 100 variants with CSV annotation in both ClinVar and Franklin, agreement was 100% (95% CI, 96.4–100).
As the KCNH2 gene is well studied, we compared VarCard.io's results with previously published14, 15 data regarding function assessment of the 24 KCNH2 variants in our cohort. We were able to procure data for 11 variants, of which 3 were classified as LB and 8 as LP. All 8 LP variants had KCHN currents of <50% of wild‐type channels. Of the 3 LB variants, 2 had a normal current and one borderline (p. Arg176Trp 48% current compared with wild type channels).
Discussion
In this work, we aimed to evaluate the performance of AlphaMissense, interfaced through VarCard.io, an open searchable web‐based engine for variant pathogenicity prediction, on a real‐world cohort of patients with heritable arrhythmia and cardiomyopathy syndromes (central illustration). Our main findings are: Using VarCard.io, we reclassify 86.1% of VUSs to either LP or LB classes. This reclassification rate was significantly greater than the reclassification rate of 35% observed in a widely used commercial variant annotation engine (Franklin).Assigned annotations by VarCard.io were highly concordant with the observed clinical phenotypes, with 95.9% genotype–phenotype concordance.We found a 90.5% agreement rate between ClinVar and VarCard.io on variants with pathogenic or benign annotations.
Using VarCard.io, we reclassify 86.1% of VUSs to either LP or LB classes. This reclassification rate was significantly greater than the reclassification rate of 35% observed in a widely used commercial variant annotation engine (Franklin).
Assigned annotations by VarCard.io were highly concordant with the observed clinical phenotypes, with 95.9% genotype–phenotype concordance.
We found a 90.5% agreement rate between ClinVar and VarCard.io on variants with pathogenic or benign annotations.
A diagnosis of VUS in a gene with cardiac manifestations, occurring in up to 40% of genetic tests,4, 5, 6, 7 is a pressing clinical problem that may have direct implications on patients' risk assessment, specific therapy, possible preventive strategies (such as in PKP2 mutation carriers), family screening opportunities, and patients' anxiety.16, 17 As genetic panels become increasingly available and encompass more genes, the incidence of VUSs is also increasing, stressing the need for reliable and accessible variant annotation strategies.
The AlphaMissense variant pathogenicity model has the potential to facilitate the field of genetic testing by significantly enhancing the performance of computational VUS reclassification. The use of AlphaMissense was previously shown to result in the classification of 88.8% of 69.5 million variants present on gnomAD9 to LP or LB, compatible with our finding of an 86.1% reclassification rate for ClinVar‐reported VUSs.
The major difference in architecture between AlphaMissesne and prevailing popular computational models such as SIFT18 and REVEL19 is that these models are largely based on labels of pathogenicity guided by sequence homology between species and prevalence in population databases. This introduces inherent biases, as some relatively rare variants are pathogenic, and many cardiac conditions are characterized by age‐dependent penetrance. In addition, traditional supervised artificial intelligence models are often trained on human‐provided labels, thereby preserving a circular logic and bias.
The high performance achieved by AlphaMissense is the result of the model's unique self‐supervised architecture, built and trained on the basis of the principles of a large language model. In the case of a language model, each word is represented by a vector (embedding) that captures semantic relationships between words, allowing the model to reason about language, that is, predict the compatibility of any word in any given context. In the case of AlphaMissense and variant prediction, vectors represent AAs, and their compatibility in each genomic position is computed on the basis of complex properties that capture their “context” within the protein. To further enhance performance, the model is then fine‐tuned on a defined set of variants with well‐established annotations based on extreme population frequency properties.
While our results demonstrate VarCard.io's utility, it is important to acknowledge certain limiting aspects. As AlphaMissense is an artificial intelligence model with predictions based on complex, convoluted representations, there is no simple way to derive linear mechanistic explanations for the different predictions and their pathogenesis in the clinical phenotype. In addition, AlphaMissense does not infer on penetrance or predict how a variant would interact with a patient's specific multiomic inventory to create the specific phenotype (ie, would not predict if a specific SCN5A variant would cause Brugada syndrome or long QT syndrome). Furthermore, assessing clinical concordance of genes and phenotypes might be biased as both a gene's function and possibly pathogenicity and a patient's phenotype may be complex and not straightforward to assess for this study's purposes. Variants may also have incomplete penetrance and age‐related penetrance; thus, a patient not having a phenotype at a specific time point does not necessarily mean a specific variant is not pathogenic in nature. Finally, as suggested by the American College of Medical Genetics and Genomics, computational predictions are 1 component in a matrix of criteria recommended for the clinical evaluation of genetic variants. Thus, although the PP3 criterion was recently upgraded from to “supporting” to “moderate” or “strong,”20 it should still be considered within the context of other features.
In conclusion, despite the above limitations, VarCard.io, propelled by AlphaMissense, may be a highly efficient tool for cardiac genetic variant interpretation. The AlphaMissense notable performance in assessing variants that are classified as VUS in ClinVar demonstrates the potential of artificial intelligence as an important step forward in the field of clinical cardiovascular genetics.
