Performance Enhancement
By Aine Cryts
Radiology Today
Vol. 22 No. 7 P. 18
Breast imaging study finds training translates to better clinical outcomes.
The factors that impact a radiologist’s performance with screening mammography aren’t well understood, but, in a study published in the June issue of Radiology, researchers looked into seven characteristics to increase understanding of their impact. Geography, breast subspecialization, performance of diagnostic mammography, and performance of diagnostic ultrasound were the most influential factors that impacted mammography interpretive performance, per the study. Researchers relied, in part, on 11 years of screening mammography performance metrics from the National Mammography Database (NMD), which means that mammograms from women of all ages were included.
More than 31% of the radiologists included in the study had acceptable performance for all metrics, based on external benchmarks, according to an announcement by the Harvey L. Neiman Health Policy Institute about the study. The Neiman Institute studies the role and value of radiology and radiologists in evolving health care delivery and payment systems, as well as the impact of medical imaging on the cost, quality, safety, and efficiency of health care.
Between 52% and 77% of radiologists demonstrated performance in the acceptable range, per the Neiman Institute.
According to the study, which included 1,223 radiologists nationally, radiologists in the Western and Midwestern regions of the United States were more likely to have acceptable performance across a range of metrics. Cindy Lee, MD, FACMQ, FSBI, an assistant professor of radiology at New York University (NYU) Grossman School of Medicine and the study’s first author, says it is difficult to explain the geographical differences among radiologists’ performance in the study, but it could be an opportunity for further study.
Subspecialization and Experience
Mammography is considered the most effective screening test to reduce breast cancer mortality, as supported by many randomized clinical trials and large observational studies, according to the authors. For example, a 1997 study published in Cancer found that mammographic screening can reduce breast cancer mortality by 45% in women under 50 years of age. A 2012 study published in The Lancet found that in the United Kingdom, where women between the ages of 50 and 70 years are screened every three years, mammography resulted in a relative risk reduction of 20%. Still, there are many variables associated with whether a woman is called back for a biopsy, if that biopsy is performed, and the percentage of instances where breast cancer is found, Lee says.
Lee explained in the announcement that assessing performance across measures holistically, instead of relying on individual metrics alone, supports guidance provided in the ACR BI-RADS Atlas. The BI-RADS Atlas provides standardized breast imaging terminology, report organization, assessment structure, and a classification system for mammography, ultrasound, and MRI of the breast.
But the recent study provides no easy answers for patients, Lee says. That’s because the study doesn’t necessarily provide guidance on the optimal characteristics of radiologists who interprets mammograms. That said, factors such as subspecialization in breast imaging as a result of fellowship training and the experience gained by radiologists interpreting mammography for many years do seem to be linked to better performance, she says.
Andrew Rosenkrantz, MD, lead study author and a professor and director of health policy in the NYU Grossman School of Medicine, said in the announcement that most mammograms performed in the United States are read by general radiologists, rather than by breast subspecialty radiologists, who comprise less than 10% of all radiologists. According to Rosenkrantz, who is also a senior affiliate research fellow with the Neiman Institute, in the long term, the demand for all radiologists to interpret mammograms will increase, due to the country’s aging population and increased compliance with mammography screening guidelines: “Hence, attention to the interpretive screening performance of all radiologists is critical, and strategies to improve interpretive accuracy among generalists would be beneficial.”
Stamatia Destounis, MD, FACR, FSBI, FAIUM, owner of Elizabeth Wende Breast Care in Rochester, New York, who wasn’t involved in the study, says it’s expected that radiologists who are subspecialized in breast imaging will perform better in screening mammography metrics.
Multiple Sources
Radiologist characteristics were extracted from the Centers for Medicare & Medicaid Services’ (CMS) databases and included demographics, subspecialization, and clinical practice pattern for this study. Parameters included the following:
• Recall rate was defined as the percentage of screening mammography exams that were assessed as BI-RADS 0, 3, 4, or 5, which implied that additional imaging evaluation is required; the acceptable range was between 5% and 12%.
• Cancer detection rate was defined as the number of women diagnosed with screening-detected invasive breast cancers per 1,000 women screened; the acceptable range was ≥2.5 per 1,000 exams.
• The acceptable range for positive predictive value for abnormal cases at screening was between 3% and 8%.
• The acceptable range for positive predictive value for abnormal cases with biopsy recommended was between 20% and 40%.
• The acceptable range for positive predictive value for biopsy performed was between the 25th and 75th percentile of the study sample.
• The acceptable range for invasive cancer detection rate, which was defined as the number of women diagnosed with screening-detected invasive breast cancers per 1,000 women screened, was ≥25th percentile of the study sample.
• The acceptable range for the percentage of ductal carcinoma in situ, which is an early, noninvasive form of breast cancer, was ≥25th percentile of the study sample.
The performance metrics studied represent the basic auditing measures for screening and diagnostic mammography, according to the BI-RADS Atlas, and are, according to the study, widely used as performance metrics in the literature. “The proportion of ductal carcinoma in situ (DCIS) and invasive cancers among all breast cancer diagnos[es], as represented by DCIS and invasive cancer detection rate, respectively, are important indicators of overall patient prognosis and outcome,” the authors wrote.
The seven metrics included in the study provide “quantifiable evidence in pursuit of the three major goals of breast cancer screening,” the researchers wrote: first, to find a high percentage of breast cancers in a screening population; second, to locate the cancers in an acceptable range of additional imaging and biopsy recommendations, to minimize morbidity and cost; and third, to find increased numbers of early-stage, node-negative cancers, which are more likely to be curable.
Women of All Ages
Distinguishing this study was the inclusion of data on all women who had mammograms; this was provided by the NMD, which uses data captured from radiology practices to provide comparative information for national and regional benchmarking. Facilities that provide data receive semiannual feedback reports that encompass benchmark data including cancer detection rates, positive predictive value rates, and recall rates.
The study authors note that a limitation of some research analyzing mammographic screening performance using CMS claims data is that those data exclude women 65 years of age and younger, adding, “[O]ur study overcame this pitfall by leveraging the performance measures in the NMD, which included mammography performed in women of all ages.”
Researchers also accessed CMS databases including the following:
• Physician Compare, which contains information for all providers participating in the Medicare fee-for-service program; and
• CMS Provider Utilization and Payment Data, namely the Physician and Other Supplier Public Use File, which has information about services provided to Medicare fee-for-service beneficiaries; this is aggregated each year by a combination of provider and service as defined by Healthcare Common Procedural Coding System codes. Specifically, these data were used to classify radiologists in regard to their ratio of screening to diagnostic mammograms and the performance of diagnostic mammography, breast ultrasound, and breast biopsy.
Billed claims were assigned to radiology subspecialities using a claims-based subspecialty classification system based on the Neiman Imaging Types of Services (NITOS), which are codes that provide a variable that can be merged into datasets by Healthcare Common Procedure Coding System code to classify imaging procedures by modality and anatomic site. The NITOS codes, which were developed by the Neiman Institute in 2015, are designed to be used along with or instead of Berenson-Eggers Type of Service (BETOS) for imaging analyses; the BETOS coding system was, according to CMS, developed primarily to analyze the growth in Medicare expenditures.
Experts Call for More Training
“If you’re in practice as a general radiologist and [you] really enjoy breast imaging, there are very few opportunities to come back for additional training and subspecialization,” Lee says. “We really don’t have that infrastructure ... and it’s [greatly] needed.”
Lee, who received fellowship training in breast and ultrasound imaging, says better training for radiologists will help improve outcomes for patients. Radiology leaders should use audits of their department or imaging center to determine which radiologists are performing better and share best practices among radiologists, she says. Another recommendation is to shift relative value units associated with breast imaging to allow general radiologists to increase their level of expertise.
While that helps with radiologists currently in practice, she also calls for an increase in breast imaging fellowships during radiologists’ early training. “There are plenty of people who want more training, but there are very limited fellowship spots, and they’re just when you’re fresh out of graduation,” Lee says.
In addition, Lee advocates for communicating with medical students about breast imaging as an “interactive” subspecialty within radiology. She values her ability to connect with patients. “[Breast imagers] see each patient during our diagnosis,” Lee says. “We tell them, ‘I felt your lump. It’s real. But it’s a benign finding.’”
Janine Katzen, MD, chair of the Breast Imaging Fellowship Match Committee at the Society of Breast Imaging, says the breast imaging fellowship match was established in 2017 “to provide a fair and transparent process for all applications.” She says there have been increasing numbers of programs participating in the match each year. It started with 76 programs in 2017 and rose to 83 programs in 2021. While 147 fellowship positions were offered in 2017, the number stands at 169 in 2021.
“Currently, there are more available fellowship positions than applications,” explains Katzen, who is also an assistant professor of radiology at Weill Cornell Medicine-Cornell University and an assistant attending radiologist at NewYork-Presbyterian/Weill Cornell Medical Center. In 2021, more than 78% of available positions were filled, she says.
“These programs offer a minimum of six months of dedicated breast imaging training, with the great majority of programs providing a full year of dedicated breast imaging training,” Katzen says. “Given the increasing complexity of breast imaging datasets, this training is beneficial to both radiologists and our patients.”
Destounis, who is breast imaging fellowship trained and a member of Radiology Today’s Editorial Advisory Board, also advocates for “more concerted effort” early in medical school and within radiology residency programs to bring more trainees into the breast imaging field.
— Aine Cryts is a health care writer based in the Boston area.
