Bringing doctors together

or

A free, private network for verified physicians

Leading voices

Dr. Peter DeBlieux

Critical Care · LSUHSC

View

Dr. Lillian Emlet

Critical Care · University of Pittsburgh

View

Dr. Rory Spiegel

Critical Care · Georgetown University

View

Dr. Stefan Wheat

General Emergency Medicine · UW

View

Dr. Ryan Stanton

General Emergency Medicine · ACEP

View

Dr. Jim Dahle

General Emergency Medicine · The White Coat Investor

View

Dr. Mohamed Hagahmed

General Emergency Medicine · University of Pittsburgh

View

Dr. Sorabh Khandelwal

General Emergency Medicine · OSU

View

Dr. Jennifer Wiler

General Emergency Medicine · UCD

View

Dr. Gita Pensa

General Emergency Medicine · Brown University

View
  • Discover your community

  • Stay current

  • A profile that represents you

  • Find your people

About RoonPrivacyTerms

    Overview

    Validation of a 0-/2-Hour High-Sensitivity Cardiac Troponin Algorithm for Suspected Acute Coronary Syndrome in the Emergency Department.

    Sep 30, 2025DOI: 10.1161/jaha.125.043756Journal ArticleAHA

    Written by: Dustin G MarkJie HuangKeane K LeeDana R SaxDustin W BallardDavid R VinsonMary E ReedKaiser Permanente CREST Network Investigators

    Find these authors on Roon

    DM
    Dr. Dustin Mark
    Critical Care · Kaiser Foundation Hospital

    METHODS

    We performed a retrospective cohort study of adult (aged 18 years and older) ED patient encounters within KPNC between January 1, 2023 and June 30, 2024. KPNC operates 21 community medical center‐based EDs serving a diverse population with over 1.5 million annual ED visits. KPNC health plan members include approximately 33% of the population in areas served and are representative of the demographic and socioeconomic diversity of the surrounding and statewide population.17 All EDs are staffed exclusively by board‐certified emergency physicians.Patients were included if they had a chief complaint of chest pain or discomfort and hs‐cTnI testing within 6 hours of ED arrival, which was performed at the discretion of the treating clinician. Consistent with pragmatic prospective validation studies of hs‐cTn‐I algorithms in the ED, we did not include patients without chest pain or discomfort who underwent troponin testing given a significantly lower prevalence of type 1 MI in those scenarios, particularly in US practice environments.2, 15, 18, 19 Patients were excluded if they had an ED diagnosis of ST‐segment–elevation MI, an ED disposition of left against medical advice, lack of active KPNC health plan coverage at the index visit and during the following month (for completeness of outcome capture), or if there was a study eligible encounter within 30 days prior. Reporting adhered to the Standards for Reporting Diagnostic Accuracy guidelines. The KPNC Institutional Review Board approved the study with a waiver of informed consent. Requests to access the data set from qualified researchers trained in human subject confidentiality protocols may be sent to KPNC at kpnc.irb@kp.org.KPNC introduced the Access hsTnI assay across all facility laboratories on November 16, 2022. Lithium heparin plasma samples were run on the Beckman Access 2 immunoassay system, which has a reported assay limit of quantitation (20% coefficient of variation) of 2.0 ng/L, a 10% coefficient of variation of 4.1 ng/L, and an estimated maximal SD of 0.89 ng/L at a mean concentration of 8.9 ng/L (total assay imprecision).20 Sex‐specific 99th centile upper reference limits (12 ng/L for women, 20 ng/L for men) were adopted per manufacturer recommendations. At the same time, an hs‐cTnI‐based algorithm was introduced for the risk stratification of patients with possible non–ST‐segment–elevation acute coronary syndrome (the KPNC NSTE‐ACS algorithm).The KPNC NSTE‐ACS algorithm was designed as a hybrid of the HIGH‐STEACS pathway rule‐out criteria (Figure S1) and the ESC 0‐/2‐hour protocol rule‐in criteria (Figure S2), as described in Figure 1, and places patients into 1 of 4 risk strata (very low risk, low risk, indeterminate risk, or high risk). Risk‐based recommendations were as follows: (1) expedited ED discharge for very‐low‐risk and low‐risk encounters (rule‐out); (2) serial troponin testing and consideration of urgent stress testing for indeterminate risk encounters (observation), and (3) admission/consultation for high‐risk encounters (rule‐in). Of note, the presence of an ischemic‐appearing ECG was a disqualification for expedited ED discharge and resulted in a minimum designation of indeterminate risk.8, 21The KPNC NSTE‐ACS evaluation algorithm was available for reference as shown, including risk stratum assignment (very low, low, indeterminate, or high) with corresponding disposition and management recommendations. Hs‐cTnI results were obtained using the Access hsTnI assay (Beckman‐Coulter). Key shared features between the KPNC NSTE‐ACS algorithm and the parent algorithms (HIGH‐STEACS pathway and ESC 0‐/2‐hour protocol) are shown in a separate table, highlighted in green. CHF indicates congestive heart failure; ESC, European Society of Cardiology; HIGH‐STEACS, High‐Sensitivity Troponin in the Evaluation of Patients With Acute Coronary Syndrome; hs‐cTnI, high‐sensitivity cardiac troponin I; KPNC, Kaiser Permanente Northern California; LR, likelihood ratio; MI, myocardial infarction; NPV, negative predictive value; NSTE‐ACS, non‐ST‐segment–elevation acute coronary syndrome; PE, pulmonary embolism; and URL, upper reference limit (ie, 99th centile).Very‐low‐risk designations (principal rule‐out criterion) were based on an hs‐cTnI value <5 ng/L being obtained a minimum of 3 hours from chest pain onset, consistent with American College of Cardiology expert consensus guidance, and allowed for a single‐troponin rule‐out.8 Low‐risk designations (secondary rule‐out criterion) were made if the highest of the 0‐ and 2‐hour hs‐cTnI values was between 5 ng/L and the 99th centile and there was a 0‐/2‐hour difference (“delta”) in hs‐cTnI values of <3 ng/L. This delta was chosen to approximate 2 SDs of the total assay imprecision estimate. High‐risk designations (rule‐in criterion) were an hs‐cTnI value of ≥50 ng/L or a 0‐/2‐hour delta of ≥20 ng/L.6 Remaining hs‐cTnI profiles were classified as indeterminate risk. For the purposes of this study, patients with only 1 hs‐cTnI measurement within 6 hours of ED arrival and that resulted between 5 and 49 ng/L are reported separately from the indeterminate risk group as having “insufficient data” for risk designation.Educational sessions were provided to all KPNC emergency physicians before implementation, and clinical reference weblinks to the algorithm were available within the electronic health record (Epic, Verona, WI). In addition, electronic clinical decision support (eCDS) for algorithm interpretation was available to emergency physicians via Risk Stratification (RISTRA), a web‐based platform embedded within the electronic health record, which also allowed for prospective confirmation of algorithm reference during a given clinical encounter.22 Notably, there was no specific prompt to use the RISTRA eCDS outside of the educational sessions, and thus all RISTRA eCDS use was clinician initiated. Importantly, clinicians were reminded, during education sessions and within the RISTRA eCDS, that these algorithms are decision aids and not substitutes for clinical judgment in disposition and treatment decisions.Demographics and comorbidities were electronically extracted from the electronic health record. Comorbidities were determined using International Classification of Diseases, Tenth Revision (ICD‐10) diagnoses within the active problem list at the time of the index encounter, including resolved problems when indicated (Supplemental Methods 1). Presence of ischemic‐appearing ECGs was determined using a validated text string analysis methodology of the final ECG interpretation, as previously described (Supplemental Methods 2).23The primary outcome was a composite of 30‐day MI or death, inclusive of index events. Secondary outcomes were ED disposition (discharge, observation, or hospital admission), 30‐day noninvasive (functional or anatomic) or invasive coronary testing, and 30‐day coronary revascularization (percutaneous or surgical). Death was determined using a composite death database that draws from internal KPNC mortality records, the California Department of Public Health Vital Records, and the Social Security Death Index. Procedural outcomes and coronary tests were determined using ICD‐10 or Current Procedural Terminology diagnostic codes. MI outcomes were ascertained using both automatic criteria and adjudication by manual chart review. Automatic criteria for MI were an internal ICD‐10 diagnostic code for non‐ST‐segment–elevation MI or ST‐segment–elevation MI along with at least 1 hs‐cTnI value > 99th centile within the 30‐day outcome window (in accordance with the fourth universal definition of MI24), or any corresponding ICD‐10 diagnostic code within an outside claims database (no laboratory confirmation required). Adjudication of MI by manual chart review was conducted if there was an internal ICD‐10 diagnostic code for MI without an hs‐cTnI value > 99th centile within the 30‐day outcome window, or if there was both an hs‐cTnI value > 99th centile and an ICD‐10 diagnosis of acute coronary syndrome, type 2 MI, or unstable angina within 30 days following a very low‐risk or low‐risk encounter (to better ensure algorithm rule‐out safety). Manual review was independently performed by 2 emergency physicians using all available clinical notes, cardiac studies, and ECGs within the electronic health records to determine the presence or absence of a type 1 MI, as defined by the fourth universal definition of MI, with disagreements adjudicated by a cardiologist. Details on respective ICD‐10 and procedural codes are available in Supplemental Methods 3 and Supplemental Methods 4.Outcome incidence was reported for each risk stratum, and test characteristics were reported for the primary outcome and its components, including sensitivity, specificity, NPV, positive predictive value (PPV), and likelihood ratio (LR), including negative LR (LR−) or positive LR (LR+) where applicable. Algorithm overall rule‐out performance was assessed using the combination of the principal (very‐low‐risk stratum) and secondary (low‐risk stratum) rule‐out criteria, and algorithm rule‐in performance was assessed using the high‐risk stratum. Additionally, test characteristics for the principal and secondary rule‐out criteria were assessed individually. Principal rule‐out criterion test characteristics were calculated inclusive of encounters with insufficient data, because those encounters would not have been classified as very low risk (ie, hs‐cTnI was not < 5 ng/L). For secondary rule‐out criterion test characteristic calculations, to allow for a binary test definition and better represent clinical decision making in that scenario, encounters designated as very low risk were not included. We report algorithm efficacy as the proportion of patients meeting the respective rule‐out criteria.Subgroup analyses of rule‐out and rule‐in test characteristics for the primary outcome were performed according to age (< 45, 45–64, ≥ 65 years), sex, history of chronic kidney disease (CKD), and known CAD. CKD was further categorized as stage 4+ (estimated glomerular filtration rate < 30 mL/min per 1.73 m2), stage 3 (estimated glomerular filtration rate 30–59 mL/min per 1.73 m2), or absent. CAD was categorized as either ischemic (prior coronary revascularization or MI), nonischemic, or absent. To further characterize independent risks by subgroup, and to account for correlations due to repeat patient encounters, we modeled the primary outcome (dependent variable) within the rule‐out and rule‐in groups separately using a generalized estimating equation (binomial family and logit link) controlling for the above subgroups in addition to diabetes, smoking, obesity, peripheral artery disease, cerebrovascular disease, and facility, reported as adjusted odds ratios (aOR). Finally, to assess the performance of the algorithm in clinical practice, we examined the incidence of outcomes within risk strata according to final ED disposition (discharge versus observation or admission), reporting outcome differences as relative risks (RR).Outcome incidences and test characteristics were estimated with a general estimating equation approach to account for repeat encounters, and the delta method was used to calculate 95% CIs.25 Differences in proportions were assessed using chi‐square or Fisher’s exact tests. P values < 0.05 were considered statistically significant. Analyses were performed with Stata 17.0 (StataCorp, College Station, TX).Sensitivity analyses for algorithm performance and test characteristics were performed on the subset of encounters in which RISTRA eCDS was accessed by clinicians to aid in KPNC NSTE‐ACS algorithm interpretation. As noted previously, clinicians accessed RISTRA eCDS without any real‐time prompts, and thus this subset of encounters might be considered as having a more specific indicator of clinical concern for possible ACS as compared with the entire study cohort.

    RESULTS

    There were 104 025 encounters among 93 433 patients in the study cohort (Figure 2). Of unique patients, 91.1% had 1 encounter, 7.1% had 2 encounters, and 1.7% had > 2 encounters (maximum of 9 encounters). The median age was 59 years, 55% were female sex, 25% had diabetes, 13% had ischemic CAD, and 3% had stage 4+ CKD (Table 1). A primary outcome of 30‐day MI or death occurred in 5.5% of encounters (30‐day MI in 5.0% and death in 0.7%), with 81% of encounters resulting in discharge directly from the ED (Table 2). Noninvasive coronary testing, invasive coronary testing, and coronary revascularization were performed within 30 days in 19%, 6.0%, 2.9% of encounters, respectively. The incidence of the primary outcome by risk stratum (Table 2 and Figure S3) was as follows: very low risk (0.1% [95% CI, 0.1%–0.1%]), low risk (1.2% [95% CI, 1.1%–1.4%]), indeterminate risk (6.6% [95% CI, 6.2%–7.1%]) and high risk (60.2% [95% CI, 59.1%–61.2%]) with corresponding LRs of 0.02, 0.20, 1.14, and 24.3, respectively.Summary of 104 025 study encounters, with distribution by risk stratum (very low, low, indeterminate, high) and summary disposition recommendation (rule‐out, observation, rule‐in) according to the KPNC NSTE‐ACS evaluation algorithm. Note that 11 182 encounters had insufficient troponin data to allow for definitive risk stratum assignment, though were at least excluded from the very low risk stratum (ie, hs‐cTnI value not <5 ng/L). AMA indicates against medical advice; ED, emergency department; hs‐cTnI, high‐sensitivity cardiac troponin I; KPNC, Kaiser Permanente Northern California; MI, myocardial infarction; and NSTE‐ACS, non‐ST elevation acute coronary syndrome.Study Cohort Demographics and CharacteristicsIQR indicates interquartile range. * Includes American Indian/Alaska Native, Native Hawaiian/Other Pacific Islander, declined to state, unknown, and other.Primary and Secondary OutcomesMI indicates myocardial infarction.Rule‐out criteria were met in 73224 (70%) encounters with a sensitivity of 95.4% (95% CI, 94.8%–96.0%), a specificity of 83.5% (95% CI, 83.2%–83.7%), an LR− of 0.05 (95% CI, 0.05%–0.06), and an NPV of 99.7% (95% CI, 99.6%–99.7%, Table 3 and Figure S3). Sensitivity and NPV for the primary outcome were marginally higher for the principal rule‐out criterion with a sensitivity of 98.8% (95% CI, 98.5%–99.1%) and an NPV of 99.9% (95% CI, 99.9%–99.9%) as compared with the secondary rule‐out criterion, which had a sensitivity of 96.6% (95% CI, 96.2%–97.1%) and an NPV of 98.8% (95% CI, 98.6%–99.0%). Further test characteristics for the primary outcome and its individual components are presented in Table 3.Test Characteristics of Rule‐Out Criteria for the Primary Outcome and ComponentsFN indicates false negative; FP, false positive; hs‐cTnI, high‐sensitivity cardiac troponin I; LR−, negative likelihood ratio; MI, myocardial infarction; NPV, negative predictive value; TN, true negative; and TP, true positive.Test characteristics for the rule‐out criteria were calculated after excluding encounters with insufficient troponin data for risk category assignment (n=11 182). Reference cohort n=92 843.Test characteristics for the primary rule‐out criterion (hs‐cTnI <5 ng/L, very low risk stratum) were calculated with inclusion of encounters with insufficient troponin data (n=11 182) because all such encounters had a troponin value of at least 5 ng/L and thus would not have been considered very low risk. Reference cohort n=104 025.To both allow for a binary test definition and characterize the incremental value of the secondary rule‐out criterion (hs‐cTnI 5 ng/L to 99th centile with a 2‐hour delta <3 ng/L), test characteristics were calculated after exclusion of encounters with insufficient troponin data (n=11 182) and encounters classified as very low risk (n=58 530). Reference cohort n=34 313.Subgroup analysis of the rule‐out criteria (Figure 3) revealed marginally lower NPV for male sex and more notable decreases in NPV with increasing age and degrees of CKD or CAD. The largest difference in NPV was encounters with stage 4+ CKD versus no CKD (96.1% versus 99.7%, absolute difference 3.8% [95% CI, 2.5%–5.7%, P<0.001]) followed by ischemic CAD versus no CAD (98.6% versus 99.8%, absolute difference 1.2% [95% CI, 0.9%–1.5%], P<0.001). All decreases in NPV were attributable to both increases in outcome incidence and decreases in specificity, as sensitivities remained stable (range 94.4% to 96.1%, Table S1). Separate examination of principal and secondary rule‐out criteria revealed similar findings (Figures S4 and S5). Multivariable analysis (Table S2) showed that age ≥ 65 years had the strongest independent association with the primary outcome among rule‐out encounters (aOR, 7.2 [95% CI, 3.7–14.3]) followed by stage 4+ CKD (aOR, 5.2 [95% CI, 3.2–8.3]), and ischemic CAD (aOR, 2.2 [95% CI, 1.6–3.1]).Forest plot summary of rule‐out criteria negative predictive value for 30‐day myocardial infarction or death within key clinical subgroup strata (by age, sex, coronary artery disease, or chronic kidney disease). CAD indicates coronary artery disease; CKD, chronic kidney disease; and NPV, negative predictive value.Rule‐in criteria were met in 7242 (7%) of encounters with a primary outcome sensitivity of 80.4% (95% CI, 79.3%–81.4%), specificity of 96.7% (95% CI, 96.6%–96.8%), an LR+ of 24.3 (95% CI, 23.3–25.3), and a PPV of 60.2% (95% CI, 59.1%–61.2%, Table S3 and Figure S3). Subgroup analysis (Table S4 and Figure S6) showed that PPV point estimates dropped below 50% for age <45 years (43.4% [95% CI, 39.2%–47.6%]), stage 4+ CKD (45.8% [95% CI, 42.4%–48.7%]) and nonischemic CAD (49.1% [95% CI, 45.2%–52.9%]), The relative decreases in PPV among these subgroups was consistent in multivariate modeling (Table S5).Overall, 84 446 (81%) of study encounters resulted in ED discharge, of which 450 (0.53% [95% CI, 0.48%–0.58%]) experienced the primary outcome of 30‐day MI or death (Table S6). ED discharges varied in proportion by risk strata: very low risk (95%), low risk (80%), indeterminate risk (59%), high risk (11%), and insufficient data (81%). Of encounters meeting rule‐out criteria, 67 304/73 224 (92%) were discharged directly from the ED. The incidence of the primary outcome in the rule‐out group following ED discharge was 126/67 304 (0.19%, Table S7). Comparatively, among rule‐out encounters resulting in observation or admission, the primary outcome occurred in 123/5920 (2.1%). Thus, selection of rule‐out encounters for ED discharge effectively provided an added layer of risk stratification (RR, 0.09 [95% CI, 0.7–0.12]).Similar degrees of risk discernment through selection for ED discharge were noted in subgroup analysis of rule‐out encounters, specifically among those subgroups with the highest independent association with the primary outcome (Table S7 and Figure S7). For encounters with age ≥ 65 years, 18063/21136 (85%) were discharged from the ED with lower primary outcome events compared with encounters selected for observation or admission (72/18 063 [0.39%] versus 90/3073 [2.93%], RR, 0.13 [95% CI, 0.10–0.18]); in CKD4+, 429/642 (67%) were discharged (primary outcome in 7/429 [1.6%] versus 18/213 [8.5%], RR, 0.19 [95% CI, 0.08–0.44]); and in known ischemic CAD, 4924/6102 (81%) were discharged (primary outcome in 43/4924 [0.87%] versus 42/1178 [3.6%], RR, 0.24 [95% CI, 0.16–0.37]).Demographics for encounters with clinician‐initiated use of RISTRA eCDS (n=3259) were comparable to the main cohort (Table S8), though the primary outcome occurred less frequently (3.0% versus 5.5%) with a lower proportion of encounters meeting rule‐in criteria (3% versus 7%, Table S9 and Figure S8), likely being reflective of the limited use of eCDS in overtly high‐risk scenarios (eg, markedly abnormal hs‐cTnI values). Rule‐out criteria performed similarly in the RISTRA eCDS encounters as compared with the main cohort (primary outcome sensitivity, 95.7% [95% CI, 89.6%–98.3%] and NPV, 99.8% [95% CI, 99.6%–99.9%], Figure S8 and Table S10). Notably, NPV was higher for the secondary rule‐out criterion (99.7% [95% CI, 98.9%–99.9%]) as compared with the full study cohort, mostly attributable to a significantly lower incidence of death (0% versus 0.68%, P=0.02).

    DISCUSSION

    Our findings support the use of the KPNC NSTE‐ACS algorithm to safely identify patients at low risk for 30‐day MI or death in US ED populations. The NPV for 30‐day MI or death in our study (99.7%) is consistent with the early rule‐out performance of the HIGH‐STEACS pathway (from which the KPNC NSTE‐ACS rule‐out criteria were derived) when using an hs‐cTnI assay (NPVs of 99.5% and 99.6%).7, 10 Though the NPV for the secondary rule‐out criterion (ie, low risk stratum) in our study was lower at 98.8%, this is in part attributable to our use of all‐cause death as an outcome. For example, in a randomized controlled trial of the HIGH‐STEACS pathway (HISTORIC [High‐Sensitivity Cardiac Troponin on Presentation to Rule Out Myocardial Infarction] trial), <25% of the composite 30‐day outcomes in the rule‐out group were attributable to cardiac death without MI, whereas in our study >50% were due to all‐cause death without MI, suggesting that up to one half of our death outcomes (and one quarter of our composite outcomes) were noncardiac in nature. Furthermore, when considering patients with clinical concern for possible NSTE‐ACS (as opposed to the broader umbrella of chest pain complaints), the true NPV for the secondary rule‐out criterion is arguably better approximated by the sensitivity analysis of encounters with RISTRA eCDS use (99.7%).We found that advanced age (≥ 65 years), known ischemic CAD (history of MI or coronary revascularization), and CKD (either stage 3 or stage 4+) resulted in significantly lower NPV for 30‐day MI or death among encounters meeting rule‐out criteria (point estimates of 99.2%, 98.6%, 98.9%, and 96.1%, respectively). On multivariable analysis, age ≥ 65 years was the strongest independent factor, which may reflect inclusion of noncardiac death in our composite outcome. However, age and known ischemic CAD subgroups were also investigated in the original HIGH‐STEACS validation study using a composite outcome of 30‐day MI or cardiac death with results nearly identical to our study (NPV for age >65 of 98.9%; NPV for ischemic CAD 98.7%).7 Comparably diminished NPVs in the setting of advanced age or ischemic CAD have also been observed with the various ESC algorithms, including the 0‐/1‐hour protocol,16, 26, 27, 28 0‐/2‐hour protocol (using the Access hsTnI assay),15 and the older 0‐/3‐hour protocol.7 A lower rule‐out NPV in the presence of CKD, on the other hand, has not been as widely emphasized, with most investigations highlighting the limited PPV of elevated hs‐cTn values in the presence of CKD, which we likewise observed.29, 30, 31, 32, 33 In all the forementioned instances, lower specificities and higher outcome incidences were the principal driving factors leading to decreased NPVs. Clinical awareness of these limitations is warranted, with the following caveats: (1) applying additional clinical exclusions to achieve the commonly cited benchmark of ≤ 1% risk for adverse cardiac events may be “a bridge too far” owing to significant losses in efficacy (ie, fewer patients ruled out) with only marginal improvements in safety,34, 35, 36 and (2) the pretest probability threshold for functional cardiac testing in the evaluation of possible NSTE‐ACS may be as high as 2%.37The KPNC NSTE‐ACS algorithm rule‐in criteria (derived from the 0‐/2‐hour ESC protocol) also proved effective with a PPV of 60%, mirroring the results of a recent evaluation of the 0‐/2‐hour ESC protocol using the Access hsTnI assay in a separate US population.15 As mentioned, rule‐in criteria PPV was reduced in the setting of CKD29, 30, 31, 32, 33 as well as in the setting of known CAD (nonischemic more than ischemic). Notably, younger age (< 45 years) had the lowest associated PPV, a finding also seen when applying the ESC 0‐/1‐hour protocol using a hs‐cTnT assay, perhaps reflecting a higher relative risk for non‐ACS causes of hs‐cTn elevations (eg, from pulmonary embolism) and lower risk of death in younger populations.27Considering the heterogenous performance of the rule‐out criteria within clinical subgroups, it is reassuring that, in community ED practice, emergency physicians were able to select patients for discharge with improved safety compared with strict rule‐out criteria application and with less loss of efficacy as compared with superimposed clinical risk scores (eg, HEART [History, Electrocardiogram, Age, Risk factors, Troponin] score).35 Similarly efficacious improvements in safety were noted in a “real‐world” study of the ESC 0‐/1‐hour algorithm.2 It is further reassuring that this parsing was similarly effective within the higher risk subgroups of rule‐out encounters. These findings reinforce the importance of clinical judgment when using diagnostic algorithms, which should always be employed as assistive rather than deterministic tools. It is also important to acknowledge that although 30‐day MI or death was relatively infrequent among “rule‐out” patients selected for observation or admission (2.1%), this remains above the 1% to 2% risk threshold for outpatient management advocated by the American College of Emergency Physicians.38 Ultimately, identifying a structured means by which gains in safety can be reliably and consistently achieved without sacrificing efficacy remains an elusive end,33, 35, 36 though the application of machine learning technology appears promising.39, 40, 41As a retrospective study, we were limited in our ability to identify patients with verified clinical concern for NSTE‐ACS, particularly encounters without a chief complaint of chest pain or discomfort, and thus our findings may not be applicable to alternative presentations of ACS (eg, isolated dyspnea). The use of diagnostic codes to adjudicate MI outcomes is also a limitation, though agreement has been shown to be good compared with prospective outcome adjudication42 and we were able to perform targeted manual chart review to improve capture of type 1 MIs following rule‐out encounters that may have been miscoded as ACS, type 2 MIs, or unstable angina, thus further bolstering the safety assessment of the algorithm. The practice setting (an integrated health care system) and exclusion of patients without active health plan insurance (to maximize outcome capture) may also limit the generalizability of our findings; further validation studies in different practice environments and patient populations would strengthen the validity of our results. Longer outcome follow‐up periods may provide further insight, though the examined 30‐day outcome window is more pertinent to ED disposition decisions and tends to correlate well with 1‐year outcomes, especially among rule‐out encounters.10

    CONCLUSIONS

    The KPNC NSTE‐ACS algorithm demonstrated excellent overall performance using the Access hsTnI assay when applied to ED patients across a large, integrated community health system in the United States. Though rule‐out criteria performance was modestly diminished in the presence of known ischemic CAD or advanced CKD, consistent with other reports, emergency physicians effectively mitigated these risks in the process of disposition decision making, providing reassurance that the “real world” algorithm performance conforms to current standards of care.

    Appendix

    Kaiser Permanente CREST Network investigators: Dr. Dustin Mark; Dr. Jie Huang; Dr. Dana Sax; Dr. David Vinson; Dr. Dustin Ballard; Dr. Mary Reed.

    Sources of Funding

    Kaiser Permanente Northern California Delivery Science Program, Pleasanton, CA.

    Disclosures

    None.

    Supporting information

    Data S1–S4 (Supplemental Methods)Tables S1–S10Figures S1–S8
    DMDr. Dustin Mark

    Dr. Dustin Mark

    Emergency Medicine · Critical Care
    Kaiser Foundation Hospital
    dustinm