The rapid proliferation of artificial intelligence (AI) is increasingly influencing strategic decisions in radiology departments. As radiology department chairpersons (heretofore “chairs”) make financial and operational decisions, they must grapple with the reality that new AI methods hold both potential promise and unintended negative consequences. To successfully implement AI solutions, chairs must navigate the necessary shared governance, ethical considerations, fiscal realities, sustainability, and other driving factors [1,2]. Because chairs’ views of AI will likely contribute to both the short- and long-term implementations of AI tools in radiology practice, we conducted a cross-sectional survey to understand their assessments and future decisions related to AI in their departments.Authors of radiology-focused surveys have polled different audiences including radiologists [3–5], patients [6], and primary care providers [7], but none have specifically concentrated on radiology chairs. Surveys about AI in radiology have been conducted for varied reasons including to determine specific technologies being implemented [3], opportunities and threats [4], implementation approaches [5,6], and the AI features most important to encourage confident use [7]. In this study, our goal was to understand how radiology chairs, specifically in academic departments, viewed AI, conceptualized the future of AI in their practices, and planned to approach decisions related to implementation in clinical practice.Our survey was designed to gain an understanding in a few specific dimensions: (1) whether chairs have distinct levels of optimism or pessimism related to generative AI (GenAI) specifically, (2) what factors are driving decisions about AI implementation in their practice, and (3) where on the medical imaging life cycle they intend to focus their departmental efforts related to AI in the near term. AI has evolved over the last decade. Discriminative AI has existed for many years, and small-scale adoption has been typical. With the same input, discriminative models will generate the same output. More recently, increased computational capacity is fueling the proliferations of GenAI models that, with the same input, will generate varied and myriad outputs, creating unique opportunities and challenges. AI solutions hold potential for every step along the medical imaging life cycle, which we derived from the concept of the “imaging value chain” [8,9]. Each step along this value chain holds varied impact for distinct stakeholders.We were interested in understanding a nuanced perspective of chair decision making and prioritization by considering a framework of modern leadership approaches. In a classic publication, Grint codified that unique approaches required for leaders to solve “tame” (simple, local, familiar, formulaic) problems versus “wicked” (complex, universal but environmentally dependent, novel, involving controversial trade-offs) problems [10]. Because the adoption of AI in biomedicine contains these distinct types of problems (tame and wicked) [11], mapping leadership priorities on this axis of tame versus wicked can give a more enlightening assessment of what leaders are prioritizing and resultant outcomes that may emerge [12,13]. Chairs are stewards of a multidimensional mission typically including excellence in clinical care, research, teaching, and potentially other core areas.We embarked on this study to generate hypotheses because we knew sample size would be limited, given the small number of chairs of academic radiology departments across the nation. We focused primarily on the clinical aspects of this leadership role, understanding there are important interactions and dependencies. At the outset we posited that chairs would be more concerned about GenAI specifically; chairs would rather address local opportunities and challenges, rather than more global issues that AI might ameliorate or exacerbate; and chairs would be most interested in implementing use cases representing steps in the imaging value chain that support stakeholders within their department as opposed to stakeholders outside their departments.
MATERIALS AND METHODS
We conducted a web-based cross-sectional survey of radiology chairs who were active members of the Society of Chairs of Academic Radiology Departments (SCARD), and the data from this survey are used with the permission of SCARD. The survey adhered to standards set by the Consensus-Based Checklist for Reporting of Survey Studies [14]. The study was reviewed and approved by the local institutional review board and determined to meet the criteria for exempt human subjects as defined under 45 CFR 46, thus explicit written consent was waived. Still, potential respondents were explicitly informed that choosing to continue with the survey, beyond the introduction, represented confirmation of willingness to participate in the project.The survey was designed by the authors and the University of Wisconsin Survey Center. The survey had two sections (e-only Appendix): first, an introduction that included definitions and confirmation of willingness to participate and second, a series of six multipart questions. We solicited participation via a personalized, one-time use link provided by email, which opened a web-based survey that used the Qualtrics (https://www.qualtrics.com/) platform. Each link provided to a given e-mail was unique for that individual to prevent any participant from responding more than once. We sent the survey link via e-mail on October 5, 2023, to the 112 active members of SCARD. Potential participants were given two e-mail reminders after the initial e-mail invitation. In addition, the importance of the survey was discussed at the fall SCARD meeting on October 20, 2023, during a session focused on hot topics in radiology. Survey responses were downloaded from Qualtrics to a secure university server, deidentified of names and contact information, and used by the research team with only an arbitrary study ID to preserve confidentiality. The survey was concluded on October 23, 2023, for presentation and analysis. Early results were presented at the 2023 RSNA meeting.The 112 active members of SCARD represent chairs of academic radiology department across the country. Membership in SCARD is limited to sitting or interim chairs of radiology departments affiliated with Liaison Committee for Medical Education accredited medical schools in the United States and Canada. SCARD members are a cohort of representatives from the majority of departments across the United States including chairs of various department sizes. We believed that the length of the survey would determine the level of participation in this busy group, thus we did not include demographic questions in the survey and limited the overall number of questions to six.Definitions were provided to respondents at the beginning of the survey. SCARD members are a sophisticated group of leaders who need to stay on the cutting edge of the technology relevant to radiology practice; however, a minority likely have formal training in computer science or specializations that included training in machine learning or AI. Thus, our definitions were meant to clarify high-level concepts so that answers were aligned with the intended meaning of the questions. We defined AI and machine learning, noting their tendency to continuously learn and adapt. We also conveyed the specific aspects of GenAI that may warrant differential consideration. Because GenAI is relatively new to radiologists, we reiterated a short definition embedded in question 2. Questions that refer to “AI” without subclassification include all types of AI, a designation used throughout this text.Data collection and frequency analysis for survey questions was performed with the R statistical software, version 4.3.0. Data analyses was performed using R (v. 4.4.0) and was executed in two stages. The first stage involved tabulating and summarizing item responses as frequencies and percentages, noting any missing values or incomplete responses. The second stage involved investigating hypotheses of interest and estimating the effect with a supporting 95% confidence interval. Standard errors were used for testing, and interval estimation used a finite population correction factor to account for sampling without replacement from a limited number of SCARD members. P values are provided only for descriptive purposes and have not been adjusted for multiple testing in this exploratory research.For questions 3 and 4, we created separate dominance matrices [15] to rank survey items related to their importance (most to least). A dominance matrix provides an appropriate order for a hierarchy based on probabilistic dominance [16], displaying how responses for a given item relate to each other in terms of importance expressed by respondents. The decomposition of this matrix gives the Kendall-Wei (K-W) rank [17,18] for each item, which we used to rank those survey items from most to least important. We also computed a measure of departure from unanimous agreement (d) among responders [18] with respect to the rankings, with d = 0 representing unanimous agreement among respondents and larger values measuring a departure from unanimity.
RESULTS
We received 43 responses from SCARD members, a response rate of 38% (43 of 112). Tables 1 to 4 depict the results, with graphics conveying: in shades of green: optimistic, likely to implement, or more important,in yellow: neutral responses, andin shades of red: pessimistic, unlikely to implement, and less important.in shades of green: optimistic, likely to implement, or more important,in yellow: neutral responses, andin shades of red: pessimistic, unlikely to implement, and less important.Participants responded with optimism about AI in general and GenAI specifically (Table 1) with more than 80% saying they were at least slightly (ie, extremely, very, or slightly) optimistic. There was no difference (P = .175; McNemar’s test) between the 93% (40 of 43) of chairs who were to some degree optimistic about AI in general and the 86% (37 of 43) who were to some degree optimistic about GenAI. However, the number of chairs who were extremely optimistic about AI in general versus GenAI dropped from 17 to 8 individuals, respectively.We randomly ordered the elements of question 3 in the survey. However, in our analysis we laid out responses according to their place on the tame versus wicked spectrum [10] based on the criteria of published articles in our judgement (Table 2). Most respondents believed that four factors were very or extremely important: quality and efficiency (43 of 43, 100%), burnout (41 of 43, 95%), health care costs (22 of 43, 51%), and equity (47 of 43, 63%)—four responses that spanned the tame versus wicked spectrum. By constructing the dominance matrix, we sort the six items comprising question 3 according to their decreasing K-W rank (Fig. 1). Counts shown in the matrix convey the number of times the item listed in the row was rated higher than the item listed in the column. For example (first row, last column), 38 responders stated quality or efficiency was more important than educating the community about AI; there was only one instance in which salary was rated higher than quality or efficiency (fifth row, first column). If responders were unanimous in their rankings, then all counts below the main diagonal (shown in gray shading) would be zero. The degree to which the observed ranking of items deviates from unanimity is measured by the asymmetry of the sum of upper- and lower-triangular elements of the matrix, with lower values (those closer to zero) indicating less departure from unanimous agreement. In the case of these six items, the measure of departure from unanimity was d = 0.22, reflecting relative agreement among chairs.Question 4 options, regarding how likely chairs were to implement AI in the next 12 months according to the imaging life cycle, were ordered based on the chronological sequence of the imaging value chain (Table 3). One survey responder neglected to answer three elements of the imaging value chain (Question 4:1 [preorder workflow], Question 4:7 [care coordination], and Question 4:8 [downstream workflow]), and a separate responder did not answer Question 4:2 (ordering). Most respondents believed that two steps in the imaging value chain were very or extremely important: postprocessing (26 of 43, 60%) and interpretation (26 of 43, 60%). At least a third believed that image acquisition (18 of 43, 42%) met this threshold of being very or extremely important. Overlaying K-W rankings for the eight options in Question 4 (Fig. 2) demonstrated that chairs are most likely to implement AI in those steps that serve their departmental constituency. In the case of these eight items, d = 0.53, implying less consistency among the responders as compared with the same measure seen in question 3 (d = 0.22). To better visualize these priorities, we mapped the imaging value chain to convey the main stakeholders involved in each step. To do so, we laid out the imaging value chain in a reverse “C,” arranged clockwise starting from the top left corner (Fig. 3). This mapping matches the imaging value chain elements to stakeholders; steps on the left primarily serve the health system and referring clinicians, and steps on the right serve the team members within the radiology department (eg, radiologists, technologists, scribes).When asked about their perception of the level of concern of individuals within their departments about being replaced by AI, chairs seemed confident this concern was minimal (Table 4). Chairs reported that 84% (36 of 43) of radiologists and 88% (38 of 43) of technologists were not at all or only a little worried about being displaced (P = .43; McNemar’s test). Their feelings about residents and fellows seemed somewhat lower, with 72% (31 of 43) saying they perceived trainees to be not at all or a little worried. This amounts to an approximate 12 (95% confidence interval: 5–28; P = .04, McNemar’s test) percentage point reduction compared with radiologists and a 16 (95% confidence interval: 2–21; P = .02, McNemar’s test) percentage point reduction compared with technologists.Responses to open-ended question 6 added nuance to the survey (e-only online supplement only). For example, several quotes captured that the cost of AI is driving decision making: “too expensive,” “funding is the biggest limitation,” and “high interest, but hospital has shown little interest supporting.” Furthermore, realities in practice pressure chairs to prioritize particular goals, for example: “given national work force shortage—prioritize AI that increases efficiency of radiologists and technologists.” Finally, important trade-offs were mentioned: “AI can decrease dose, but at the expense of image quality.”
DISCUSSION
Chairs have high levels of optimism for AI in clinical practice and do not seem to have substantially diminished optimism for GenAI. Chairs seem to prioritize issues that can be quantified as local versus tame, like quality and efficiency, while also supporting more wicked versus universal problems like equity—opinions that demonstrated substantial agreement or unanimity across survey participants. Despite the availability of AI to support a plethora of tasks throughout the imaging value chain [8,9,19], chairs seem to be focused, with slightly less unanimity, on tasks that most closely impact their constituency: radiologists, technologists, and scribes. They do not perceive that other stakeholders in their departments are particularly concerned about being replaced, though some responded that trainees feel a bit more vulnerable, a phenomenon that has previously been shown in the literature [20]. The open-ended questions at the end of the survey revealed challenges about which our standardized survey questions did not specifically ask, including trade-offs that needed to be balanced (eg, radiation dose versus image quality). Although radiology chairs are enthusiastic to use AI to target local operational priorities and challenges, they also identify cost as an obstacle to implementation.Chair optimism for AI likely indicates leadership support and that implementation will likely increase. Although chairs envision they will focus on local stakeholder priorities, AI implementation geared toward radiologists will likely be complemented by AI driven by or intended for other stakeholders. AI implementations are not necessarily always noticeable to radiology chairs (nor under their control) and may be implemented by the health system or other stakeholder groups (eg, primary care). For example, AI care pathway tools have been widely adopted via electronic health record–embedded clinical decision support to influence image examination ordering. Furthermore, many imaging equipment vendors have equipped CT and MR scanners with AI algorithms to optimize image reconstruction. Equipment for other modalities like ultrasound use AI algorithms for segmentation and automatic structure identification and labeling. These and other sources of AI will complement the initiatives that chairs champion for their departments.Surveys in the literature have previously shed light on the evolution and adoption of AI in radiology practice. Surveys of radiology society members in Italy (2019) [4] and the United States (2020) [21] revealed both optimism and concerns. In Italy, 77% of radiologists were optimistic and 90% were not concerned about being replaced. In the United States, 34% of respondents were using AI, and that use was in the same steps of the imaging value chain on which chairs in our survey focused. Of those radiologists not using AI, 80% saw no benefit and 35% could not justify expense [3]. The former sentiment regarding no benefit was not shared by US chairs in our survey 4 years later, although the latter concern about cost was indeed reflected in several chairs’ free text responses. Prior surveys emphasized the importance of distinct stakeholder perspectives on AI in radiology including the necessity to collaborate across stakeholders to optimize AI implementation [22].Radiology leaders regularly publish reflections [1,23,24] on the AI revolution and implications for radiology. The most recent dimension in this revolution has been around GenAI. We were surprised that chairs were not more pessimistic about GenAI related to some of the emerging risks like “hallucination” and the potential to propagate any biases inherent in the dataset on which the algorithm was trained [25]. Even the US FDA is struggling to regulate GenAI tools in health care largely because the output is not reproducible as the models continue to train on new data [26]. These challenges of limited generalizability or reproducibility may translate into medical errors or bias, wicked problems that may emerge in the future. With the increasing publications documenting both the pitfalls of AI and mitigation strategies [27–30], perhaps radiology chairs are optimistic that careful implementation will minimize these risks. Thankfully, chair responses related to implementation priorities indicated that they will likely be committed to tackling tame as well as wicked problems [10] posed by AI, even if these unintended consequences of AI are not foreseeable or viewed with pessimism currently.Although we prioritized questions about leadership decisions related to AI impact on the clinical mission, with less focus on education and research (apart from asking chairs to rank “educating the community about AI” in question 4), these mission areas are interrelated. Because academic radiology careers combine these and other activities, AI support for more routine tasks, like travel reimbursement, may have a global impact to increase focus on primary mission areas. Investigating the important interactions of AI support for all mission areas as well as other burdens of radiology “work” is ripe for further research.A strength of our study is that we successfully polled chairs who will likely be key drivers of implementation and adoption of AI in radiology practice. Weaknesses of our study include limited response rate of 38%, although on par with historical SCARD survey response rates. The resulting small numbers limit our ability to draw statistical conclusions. In addition, we did not specifically measure demographics of respondents due to concerns that these additional questions would influence survey completion and therefore response rate. We do have a sense of the demographics of SCARD, and we do not believe those members who participated would share substantially different views about AI than those who did not participate. Keeping up with rapidly accelerating AI offerings is a daunting task, knowledge that is necessary to make responsible decisions.We did not ask chair participants questions about their AI knowledge, as do some surveys about AI [6,21,31], which established a correlation between participants’ level of perceived AI understanding and AI ratings. For radiologists, those with greater self-assessed knowledge of AI were more likely to approve of its introduction [5]. Chair comfort level and decision making may depend on their knowledge of AI, a potential area of future study. Our definitions and ordering of tame and wicked drivers of decision making are admittedly subjective.Based on our survey, chairs seem optimistic and generally prepared to navigate this early phase of AI adoption in radiology practice by focusing on implementation goals important for their constituency, tackling challenges across the tame versus wicked spectrum, and monitoring constituencies for replacement anxiety. Future research into key concepts will be crucially important for effective implementation of AI in practice. For example, the amount of specialized knowledge that leaders need to make informed or optimal decisions about AI implementation, the balance AI implementation trade-offs such as image quality versus radiation dose or expense versus benefits that chairs should attempt to achieve. Purposeful investment in education and research to support leadership decision making in diagnostic radiology will likely accelerate the positive contributions AI will make to our field in the future.
EM
Dr. Elizabeth Morris
Radiology · Breast ImagingUniversity of California, Daviselizabethm