Bringing doctors together

or

A free, private network for verified physicians

Leading voices

Dr. Shashank Sinha

Advanced Heart Failure · Inova Fairfax Hospital

View

Dr. Javed Butler

Advanced Heart Failure · Baylor Health Care System

View

Dr. Andrew Sauer

Advanced Heart Failure · Saint Luke's

View

Dr. Anu Lala

Advanced Heart Failure · Mount Sinai

View

Dr. Robert Mentz

Advanced Heart Failure · Duke University

View

Dr. Sunny Varshney

Advanced Heart Failure · Stanford

View

Dr. Stephanie Hsiao

Advanced Heart Failure · Stanford

View

Dr. Michelle Kittleson

Advanced Heart Failure · Cedars-Sinai

View

Dr. Adrian Hernandez

Advanced Heart Failure · Duke

View

Dr. Dorairaj Prabhakaran

General Cardiology

View
  • Discover your community

  • Stay current

  • A profile that represents you

  • Find your people

About RoonPrivacyTerms

    Overview

    Impact of Residential Social Deprivation on Prediction of Heart Failure in Patients With Type 2 Diabetes: External Validation and Recalibration of the WATCH-DM Score Using Real World Data.

    Feb 8, 2024DOI: 10.1161/circoutcomes.123.010166Journal ArticleAHA

    Written by: Salil V DeoSadeer Al-KindiIssam MotairekDavid McAllisterAnoop S V ShahYakov E ElgudinEiran Z GorodeskiSalim ViraniMark C PetrieSanjay RajagopalanNaveed Sattar

    Find these authors on Roon

    EG
    Dr. Eiran Gorodeski
    Advanced Heart Failure · UH

    Introduction:

    Cardiovascular disease (CVD) is the most common cause of morbidity and mortality in patients with type 2 diabetes mellitus (T2DM)1. Regardless of baseline ventricular systolic function, patients with T2DM have 2–3-fold increased risk for CVD, especially heart failure (both preserved and reduced ejection fraction)1. The expanding armamentarium of cardio-reno-protective medications (e.g Glucagon like Peptide-1 receptor agonist (GLP1RA),Sodium Glucose Co-transporter-2 receptor inhibitor(SGLT2i), and non-steroidal mineralo-corticosteroid receptor antagonist (MRA) in T2DM have recently transformed pharmacotherapy with clinical guidelines now recommending that these agents be initiated in T2DM patients at high-risk for CVD 2. These medications also uniquely reduce the risk for incident heart failure hospitalization (HFH) 3. At present, however, these medications remain underutilized in clinical practice 4–6Accurate HF risk prediction in patients with T2DM is important to allocate resources, and focus targeted therapies, especially SGLT2i, and may help overcome clinical inertia. Many risk models currently exist for predicting risk of HF in patients with T2DM 7–10.Most of these models have been derived from studies that may not reflect real-world population (clinical trial data or prospective studies). For example, the WATCH-DM score and RECODe risk scores were developed using the ACCORD trial, while the TRS-HFDM was created using the SAVOR-TIMI 53 study. In fact, a recent systematic review reports only one score, the QDIABETES, was created using primary care data from the UK8. As prior literature reports differences in patient characteristics between trial or prospectively developed cohorts and real-world data, there is a need to evaluate these models using real-world data 11, 12. Therefore, it is unclear whether risk prediction calculated from these models can be generalized to diverse patients in real-world clinical settings. Additionally, studies have already demonstrated that the social determinants of health (SDoH) are associated with an increased risk of cardiovascular disease and HF 13–15.Prior studies have linked social deprivation with risk of HF in patients T2DM. Despite this, current risk scores do not account for the SDoH; this represents a missed opportunity to improve patient care. Recent efforts to develop polysocial risk scores incorporating a variety of socioeconomic factors, demographics, and clinical data can provide improved risk prediction to enhance resource allocation. However, research is needed to better characterize the extent to which SDoH interacts with clinical risk scores to predict disease. A prior study demonstrated mis-calibration of the American Heart Association/American College of Cardiology Pooled Cohort Equations Risk Model for ASCVD risk prediction in patients residing in socially deprived neighborhoods13. Recent studies also demonstrated increased HF rates in patients with T2DM residing in deprived neighborhoods 15 16. However, despite these observations, the UK-based QDIABETES score is the only heart failure hospitalization risk score that included measure of social deprivation 17.From the various recent scores developed for predicting heart failure in T2DM, the WATCH-DM score is externally validated in multiple cohorts (mostly trials, and one single-center electronic health record dataset with small number of HF events) 18 It uses readily available clinical data and provides a simple point scoring system. We, therefore, sought to externally validate the WATCH-DM score in a large contemporary US-based multi-center real-world electronic health data. We further sought to test the hypothesis that residential SDoH may influence the scores predictive accuracy and, if necessary, recalibrate the score using SDoH.

    Research Design and Methods

    The data that support the findings of this study are only available to researchers credentialled to conduct research at the VA. Code used for statistical analyses are available at: https://github.com/svd09/recalibrate_watchdm_sdi. The study was approved by the Research and Development Committee of the Northeast Ohio VA Healthcare system and individual patient consent was waived.The Department of Veterans Affairs (VA) healthcare system, with approximately 9 million beneficiaries, is the largest single healthcare provider in the US. The primary source of data for this study was the Corporate Data Warehouse, a central secure repository of information obtained from the electronic health records of US Veterans receiving in-patient and out-patient care. Researchers can link clinical information with data regarding laboratory and other tests to create an accurate detailed health trajectory for each patient. Information regarding clinical encounters occurring outside the VA healthcare system is also provided in the Corporate Data Warehouse when this care has been paid by the VA. We initially identified US Veterans that received outpatient care with a diagnosis of diabetes mellitus in 2010 across all VA healthcare locations and defined their first visit during this year as they index visit. We excluded patients that had current or prior heart failure. We used a time-period of 1 year prior to the index visit to collect information regarding their clinical characteristics and laboratory results. We used this information to obtain the points for each covariate included in the WATCH-DM score and then obtained the total points for each patient. We observed missing information regarding body mass index (22.5%), creatinine (15%), HbA1c (1.2%), systolic blood pressure (1.6%), and diastolic blood pressure (1.6%). Age, prior myocardial infarction, and prior coronary artery bypass grafting did not have any missing data. We filled the missing fields using mean imputation, with the mean values obtained from the non-missing information for each variable. We identified the patients’ residential ZIP code at the time of the index visit.The 2015 SDI is a composite metric of socioeconomic status derived at the ZIP code level19. This index is derived using 7 domains from the American Community Survey identified through factor analysis: Income [Percent population less than 100% FPL (population under 0.99 /total population)], education [Percent population 25 years or more with less than 12 years of education (population with less than high school diploma or 12 years of education/total population], Employment [Percent Non-employed (not in labor force + unemployed) / (civilian + not in the labor force) for the population 16–64 years], Housing [Percent population living in renter-occupied housing units (Renter occupied housing units/ (Owner-occupied housing units + Renter occupied housing units), Percent population living in crowded housing units (Tenure by Occupants Per Room – a population with ≥ 1.01 occupants per room in Owner-occupied housing units and Renter occupied housing units) / total population], Household Characteristics [Percent single-parent households with dependents < 18 years (total single-parent households (male and female) with dependents <18 years)/total population)], Transportation [Percent population with no car (population with no vehicle available/total population)], and demographics [Percent high needs population, namely (population under 5 years of age + women between the ages of 15–44 years + everyone 65 years and over)/total population]. The final SDI measure is a weighted total of all these measures and ranges from 0 (least socially deprived) to 100 (most socially deprived). For our study, using the residential ZIP code derived SDI we grouped patients as follows: group 1 (least deprived) SDI: ≤ 20, group2: 21 – 40, group 3: 41 – 60, group 4: 61 – 80, and group 5 (most deprived) 81 – 100 (Figure S1). These cut-offs were used to represent fixed percentiles throughout the US ZIP codes (meaning that do not vary by cohort used) with each subsequent group representing a higher level of social deprivation as derived from the patients residential ZIP code.The WATCH-DM score used in our study includes the following covariates: age, body mass index, systolic blood pressure, diastolic blood pressure, serum creatinine, high density lipoprotein cholesterol concentration, prior myocardial infarction, prior coronary artery bypass grafting and the HbA1c level. Users score patients on each covariate and obtain a point total. They can then use a look-up table which converts the points into 5 groups from group I (very low risk) to group V (very high risk) with a corresponding 5-year risk estimate (Table S1).In our study, we defined the incidence of heart failure as the first occurrence of heart failure hospitalization over the five-year follow-up period, i.e., being admitted with a primary diagnosis of heart failure ascertained by the International Classification of Diseases (ICD)9th (428.xx) and 10th (I50.x) version codes. In patients that had heart failure hospitalization, we further obtained the admission date for that event.We reported continuous and categorical data as mean (standard deviation (SD)) or count(percentage) respectively. For patients that died during follow-up, we obtained their date of death and calculated the duration from the index date and death date. For those alive, we censored their follow-up at 5 years, as all patients that were alive continued to receive care at VA medical centers beyond that time-period. We used the Kaplan Meier method to obtain the cumulative incidence for heart failure hospitalization. We first fit and then validated the WATCH-DM score in the whole cohort. After calculating the WATCH-DM score for all 1,065,691 patients we divided them into five groups- from group I (very low risk) to group V (very high risk). We initially evaluated the discrimination metrics of the score. Discrimination is the ability of the risk model to separate patients into groups with incrementally increasing risks. To evaluate this, we calculated and then plotted the heart failure hospitalization incidence of each WATCH-DM risk group and graphically evaluated the extent of curve separation between groups. We fitted a cox proportional hazards model with the WATCH-DM group as the covariate and evaluated the relative risk for heart failure hospitalization in each group with group I as the reference. We then refitted this model including the SDI quintile as a second covariate. To examine the incremental benefit of adding the SDI to the original WATCH-DM score, we compared model fits using the Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC), an analysis of variance (ANOVA) test. We calculated the Harrell’s c-statistic to quantify the discrimination of the score. Risk score calibration is its ability to accurately predict the absolute risk at the studied timepoint, i.e, calibration measures the difference between the expected(predicted) and observed (actual) cohort risk. We initially split the data into deciles of the predicted risk. We created a calibration plot and numerically evaluated the calibration with the Greenwood Nam D’Agostino (GND) test20. The GND test is a goodness-of-fit test like the Hosmer Lemeshow test for Cox models and can be applied for evaluating external calibration. Patients are grouped into n (normally deciles) of predicted risk, and the ratio of the observed and estimated events for each group is calculated to obtain a Χ2 value and the corresponding p-value with n-1 degrees of freedom. The calibration plot is a graphical representation of the expected (x-axis) and observed (y-axis) estimates21. The model calibration was visually assessed against a diagonal line, which is the ‘perfect’ calibration fit.Using the same methods stated above, we then externally validated the risk score in each quintile of the SDI. Our preliminary observations demonstrated wide variation in the incidence of heart failure hospitalization across quintiles of the SDI. We, therefore, further re-calibrated the WATCH-DM score according to the SDI quintile groups. We used the calibration-in-the-large method to obtain a correction factor which was then included in the prediction model to obtain the revised risk estimates 22. Briefly, the correction factor is a point estimate obtained from the observed and expected risk estimates in each quintile of the SDI. The supplement contains more details regarding correction factor calculation. The risk prediction obtained after re-calibration was then tested with the calibration plots and GND test. Finally, to evaluate the potential clinical usefulness of the re-calibrated model, we fit decision curves to evaluate its ‘net benefit’23.

    Results

    Our validation cohort included 1,065,691 patients (mean age 67.45 years, 3.33% female patients, 75.55% White patients, and 6.26% Hispanic patients) that received outpatient care during 2010 at VA medical centers nationwide. Patients in our cohort resided in 34,596/41,704 (82.9%) unique zip codes in the US. For patients included in our cohort, the ZIP code level median household income was $ 46,791 (IQR: $37,963, $58,689) and the median household value was $ 119,600 (IQR: $84,000, $181,500). The median single parent households, median unemployment and median poverty levels observed in our data were 14% (IQR: 9, 19), 7.9% (IQR: 5, 11.5) and 12.4% (7, 19.5) respectively. The prevalence of preexisting coronary artery disease, cerebrovascular disease, peripheral arterial disease was 31.02%, 11.14% and 14.97% respectively; 13.22% have chronic kidney disease (eGFR < 60 ml/min/m2) and an additional 2.07% were dialysis dependent (Table S2). The mean (SD) WATCH-DM score in our cohort was 14.84 (3.66), the median score was 15 (Inter-quartile range: 12,17) and the maximum was 33 (Figure S2). From group I to group V of the WATCH-DM score we observed a gradual increase in the prevalence of all traditional cardiovascular risk factors (Table S2).Among 1,065,691 patients, 180,383 died over the 5-year follow-up period, resulting in a cumulative mortality rate of 2.75 (95% CI: 2.72, 2.78) %, 9.41 (95% CI: 9.31, 9.47) % and 17.03 (95% CI: 16.96, 17.10) % at 1, 3 and 5 years respectively (Figure S2). After adjusting for age, the cumulative mortality rate is 2%, 7.2% and 13.6% at 1,3, and 5 years respectively. The cumulative incidence of heart failure hospitalization for the whole cohort was 1.06%, 3.14% and 5.39% at 1, 3, and 5 years respectively. The cumulative incidence for 5-year heart failure hospitalization was incrementally higher with each subsequent WATCH-DM score group from 3.18% in group I to 11.00% in group V (Figure 1; Table S3). We also observed that the relative risk for heart failure hospitalization increased for each subsequent group; considering group I reference group, those in group II had a 38% higher relative risk for heart failure hospitalization [HR 1.38 (95% CI:1.34, 1.42)], while it was more than three times higher in group V [HR 3.41 (95% CI: 3.32, 3.50)] (Table S3). Harrell’s c-statistic was 62.21 (61.84, 62.58) demonstrating modest overall discrimination. The WATCH-DM score had acceptable calibration (E/O ratio = 1.02); while the GND test reported mis-calibration (p-value < 0.001), the calibration plot demonstrated that the overall calibration of the model is acceptable. (Figure S4).We linked zip code data from 1,012,315 patients (94.4% of the whole study cohort) with the Social Deprivation Index; 142,306 (14.05%) belonged to the least deprived quintile (Q1) while 191,619 (18.92%) belonged to the fifth quintile (Q5), the most deprived category. Patients in Q1 were much more likely to be White, while 38.3% of the patients in the SDI quintile 5 (most deprived) were black (Table S4). While the mean WATCH-DM score incrementally reduced from Q1 (mean:15.2, SD:3.6) to Q5 (mean:14.5, SD:3.7) of the SDI (Figure S2), we observed a gradual increase in the 5-year cumulative incidence for heart failure hospitalization across the quintiles of the SDI (Table S5). We observed that in quintiles 1 and 5 of the SDI, 5-year heart failure hospitalization rates did not differ much according to the patient’s race (Figure S4). Including the SDI to the original WATCH-DM score improved model fit (SDI covariate p-value < 0.001; model coefficient 0.131). The model including both variables also reported lower AIC (1480283 vs 1478687) and BIC (1480292 vs 1478705) values. The discrimination (Harrell’s c-statistic) for the WATCH-DM score across quintiles of the SDI was like that observed for the whole cohort. However, the WATCH-DM score was not well calibrated to predict the 5-year heart failure hospitalization across SDI quintile. As observed from the E/O ratio, while it over-predicted the risk in Q1 & 2, it under-predicted the risk in Q5 (Figure 2; black dots, line in Figure 3). The chi-square value for the Greenwood Nam D’Agostino test was, therefore, also correspondingly high in each quintile (Table in Figure 3). We recalibrated the WATCH-DM score using the correction factor (Details provided in the supplement) and observed that the mean predicted risk reduced in Q1 (O/E ratio: 0.72) & Q2 (O/E ratio: 0.73) of the SDI, while it increased in SDI groups Q4 (O/E ratio: 1.05) & Q5 (O/E ratio; 1.25) (Figure 2). After using the derived correction factor to recalculate the predicted risk, we observed better calibration as reported by reduced chi-square values for the GND test and calibration plot lines that were closer to the ideal reference line (red dots, line in figure 3). Using decision curve analysis, we also confirmed that the re-calibrated WATCH-DM score provided a potential ‘net benefit’ when used in clinical practice (Figure S5). After re-calibration, the predicted cumulative risk for heart failure hospitalization in the very high-risk group (WATCH-DM score ≥ 19) was 8.01% and 13.26% respectively in quintiles 1 (least deprived) and 5 (most deprived) of the SDI (Table S6).We have packaged our results in an online calculator available at: https://svd09.shinyapps.io/sdi-watch-dm/. Users can enter specific patient data including the residential zip code. This dashboard provides the SDI quintile for that zip code, the 5-year heart failure hospitalization risk according to the WATCH-DM Score and the re-calibrated 5-year heart failure hospitalization risk. Figure 4 depicts a screenshot of our app (Figure 4).

    Discussion

    We externally validated the WATCH-DM score, a model that predicts 5-year incident heart failure hospitalization in patients with diabetes mellitus, in a large cohort of US Veterans. Our main finding is that beyond the original WATCH-DM score, the level of social deprivation (as measured by the SDI) is associated with the 5-year risk for heart failure hospitalization. In patients residing in least deprived ZIP codes, the WATCH-DM score overestimated observed risk, while it underestimated the risk in patients residing in the most deprived ZIP codes. We provide readers with a recalibrated WATCH-DM score which accounts for SoDH and is available for clinical use via a simple online dashboard.It is now known that social deprivation is associated with increased HF risk. In an earlier study, we reported that more socially deprived US counties had higher heart failure related mortality rates14. Data from the Southern Community Cohort Study demonstrated that each inter-quartile increase in neighborhood deprivation index was associated with a 12% increase in risk of incident HF24. However, we feel that the association between social deprivation and cardiovascular diseases, including heart failure may be even stronger in patients with T2DM. In a cohort study of 900 patients investigating the association between the SDI and ischemic events after percutaneous coronary intervention, researchers found that the SDI was associated with increased risk for events only in patients with diabetes (adjusted HR for Q4 vs Q1 1.72 [1.01–2.92], P=0.04) but not in patients without T2D (P=0.39) 23. A recent study from Sweden examined the association between neighborhood-level social deprivation and HF risk specifically in patients with T2DM 14. In this population study of 434,000 patients, authors reported that the adjusted risk of heart failure was higher in deprived neighborhoods (HR for high deprivation vs low deprivation 1.11, 95% CI 1.06–1.16, in men & 1.15, 95% CI 1.09–1.21, in women) 14. In our recently published study that examined this issue in US Veterans, we showed that SDI was associated with increased risk for the first heart failure hospitalization as well as recurrent events in T2DM patients16. In this analysis, we observed that despite a decline in the WATCH-DM scores across the SDI, the observed heart failure hospitalization rates increased. This fact further demonstrates that social deprivation influences the risk of heart failure hospitalization independently of the covariates present in the WATCH-DM score, specifically, age, BMI, glycemic control, and kidney function. Together, this information supports the notion that social deprivation metrics influence heart failure risks that are not accounted for by the traditional cardiovascular risk scores.This study adds to the emerging literature that focuses on social and environmental factors as determinants of residual risk in cardiovascular disease. Recent studies have focused on developing poly-social risk scores that augment the routinely included clinical and demographic factors25. For example, Javed et al. developed and validated a poly-social risk score that included seven different SDoH domains and reported that it had good discriminatory capacity to predict atherosclerotic cardiovascular disease26. Similarly, Zhao et al examined social risk and incident T2DM in the UK Biobank study and showed that a poly-social risk score (derived from three SDoH domains) could reliably predict incident T2DM27. Hence, these studies support our observations regarding the important role that SDoH play in determining cardiovascular risk.Given the above discussion, it is thus important to investigate, and if necessary, recalibrate cardiovascular prediction models within the context of social deprivation. However, we observed in our review of current literature, as did others, that very few risk scores account for the social determinants of health. An observational cohort study of 100,000 patients in the US reported that the AHA/ACC pooled cohort equation, a model to predict the incidence of atherosclerotic cardiovascular disease, showed poorer discrimination (c-index 0.8 versus 0.7) in patients residing in deprived neighborhoods 24. Similar observations were also reported using data from Scotland25. However, to our knowledge, our study is the first to recalibrate a well-validated prediction score based on neighborhood social deprivation. Via this study, we further hope to demonstrate how including social deprivation measures may help to refine risk estimates produced by clinical risk scores. Recent efforts have developed poly-social risk scores, that incorporate multiple domains of SDoH, and linked them with cardiovascular risk. TheseThis also highlights the fact that socioeconomic factors mediate the HF risk through mechanisms other than those related to traditional risk factors included in the WATCH-DM score. We used a simple tool like the ZIP code to determine the SDI index. Zip codes are commonly available in electronic medical records making our recalibrated risk model easy to use for practicing physicians. We have packaged our risk model into a tool that clinicians can routinely use on their phones. We hope that our study and application will encourage clinicians to use this score in the shared decision-making process with patients. It is also important to note that even a small improvement in model fit with the addition of SDoH can be considered ‘socially meaningful’, given that socioeconomic factors affect entire populations. Hence, the population impact of even a minor change in social deprivation burden can be significant. This speaks to the value of including social determinants in risk models not only from a statistical perspective, but also as key upstream constructs that shape more downstream diabetes and cardiovascular outcomes. Social factors are often ignored in traditional risk assessment, representing missed opportunities to understand their effects across the full cascade of cardiovascular disease development. Given the evolving data science approaches, capturing geospatial SDoH and environmental data in real time may aide in cardiovascular risk prediction28. These modified/recalibrated risk scores incorporating SDoH can help concentrate resources like community-based care programs and screening initiatives to those at the highest medical risk. Recently, the Center for Medicare and Medicaid Services introduced a new policy to incorporate SDoH data collection with the aim of promoting equitable healthcare access and delivery29. A recent review highlighted multiple missed opportunities in improving the care of people with T2D30. Authors recommended a comprehensive patient evaluation as the first step towards providing high-quality diabetes care; our study clearly demonstrates that understanding the patients SDoH is an important part of such an evaluation.This study should be interpreted within the context of cohort limitation. First, our cohort is predominantly male, which is reflective of the VA population (although included numerically high number of women [n= 35432]). The relative predominance of men may limit generalization of the current findings to women, and additional external validation of the recalibrated model in a mixed gender population is thus required. Second, our cohort consists of patients with diabetes mellitus that received outpatient care at VA medical centers; therefore, compared to a population-based cohort, our patients likely have a higher baseline risk. While this would impact the absolute event rate, it does not change the mis calibration that we observe across the SDI groups. Therefore, as a benchmark for readers to understand the absolute increase in heart failure hospitalization risk, we report both the original and recalibrated 5-year heart failure hospitalization in our ònline risk calculator portal` (link provided in the manuscript).Third, social deprivation metrics at smaller area levels like census tracts may provide a more accurate measure for each patient. However, these measures are not available for routine use in clinical practice, while patient generally knows their residential ZIP code. ZIP code information is also readily available in other large databases like that provided by the Center for Medicare and Medicaid Services. We feel that although the census tract level SDI may be a more accurate marker, ZIP code level information is more applicable for routine use. Lastly, our definition for heart failure hospitalization is obtained from International Classification of Disease codes, not prospectively clinically adjudicated information. However, this will again, not impact the difference in heart failure rates seen across the spectrum of social deprivation. Importantly, we were able to externally validate the WATCH-DM risk model in a large real-world cohort. Our large geospatially coded cohort provided further proof that neighborhood-level deprivation metrics can be helpful to refine risk estimates above and beyond clinical covariates. Adoptions of such measures in prediction models is likely to improve accuracy.

    Conclusion

    Our large external validation study using real-world data demonstrated that the WATCH-DM score is acceptable for use in a real-world cohort of patients with type 2 diabetes mellitus. We observed that the 5-year risk for incident heart failure hospitalization is modulated by social deprivation. To aid clinicians caring for this patient population we produced a simple-to-use online SDI recalibrated WATCH-DM calculator. Our study highlights the wider need to consider the social determinants of health in future risk prediction modeling.
    EGDr. Eiran Gorodeski

    Dr. Eiran Gorodeski

    Cardiology · Advanced Heart Failure & Transplantation
    University Hospitals of Cleveland
    eiran
    Cleveland, OH, USA