How cardiologists addressed bias in a clinical algorithm

Not only is Joseph Wright a physician, he’s the chief health equity officer for the American Academy of Pediatrics. So he was skeptical when, at his annual checkup in January, an alert popped up on his doctor’s computer screen.

A calculator embedded in Wright’s health record had automatically pulled in his data, including the blood test he had done that morning. The calculator’s result indicated his arteries could be narrowing dangerously: He should start taking a statin right away, his doctor said.

Wright knew well that Black patients are at higher risk for heart disease and stroke, and about 30% more likely to die from heart disease than white patients. That’s why the calculator had included his race — along with his age, cholesterol, and blood pressure among other traits — to predict his risk. But he also knew — better than most — that there was nothing inherent to his physiology as a Black man that easily explained that higher risk. Dispelling the myth of biological differences between races is part of his work.

“The scientist in me was very curious,” said Wright, wondering how accurate the race-based calculator was for him. And the patient in him couldn’t fully trust his doctor’s recommendation to lower his cholesterol with medicines that he might not need.

Wright isn’t the only one with reservations. For several years, cardiologists and public health researchers have been sparring over race’s role in the calculator his doctor used, one of their most common tools to predict the risk of strokes and heart attacks in people who haven’t had them before. It’s one of dozens of clinical algorithms that have come into question for factoring patients’ race into their results — a practice that critics say may harm Black patients more than it helps them.

Race, surely, is trying to tell clinicians and researchers something important when it shows up as a powerful predictor of disease. It’s sending up a flare, calling on them to hunt out the hidden forces that drive up patients’ risk. Often, it’s a crude proxy for the social and economic conditions, including structural racism, that shape patients’ lives — and their chances at good health. Piecing these social factors together is a central goal as the movement to strike harmful uses of race from clinical tools gathers momentum.

The health care system prioritizes documenting a patient’s medical history, neatly summarized in lab results and numerical billing codes, over the more complicated parts of their life story. But research has suggested these more complex influences — often called social determinants or drivers of health — can account for up to 80% of a patient’s health outcomes. Is a patient within driving distance of quality health care? Can they afford to transport themselves there, and to pay for appointments? Do their local stores stock fresh produce?

Including such factors could be a way to produce more accurate clinical algorithms while enhancing social justice. And unbeknownst to Wright’s doctor, cardiologists were already beginning to upend the paradigm he’d confidently used for so long. In a first among broadly targeted disease-risk calculators in the United States, they were developing a tool that, instead of race, uses social factors to personalize predictions.

Framingham Heart Study
For many years, clinicians relied on data from the long-running Framingham Heart Study to predict heart disease risk. Study volunteers like Patricia McNamara, shown having an eye exam in 1952, were nearly all white. Frank C. Curtin/AP

Six days before Wright’s appointment, the American Heart Association quietly published a new online calculator to predict the risk of heart attack, stroke, and heart failure called PREVENT. The update was meant to capture a new understanding of the interplay between cardiovascular, kidney, and metabolic disease, incorporating new clinical variables like BMI and kidney function.

The AHA also designed the new calculator without race — an important ethical and scientific shift for the organization that, along with the American College of Cardiology, maintains the country’s clinical practice guidelines for the care of cardiovascular disease. It was an approach borne of the AHA’s commitment in 2020 to tackle the structural racism that contributes to Black Americans dying of heart disease at much higher rates than others.

Throughout the two years the process took, one stark question kept surfacing, said Sadiya Khan, a preventive cardiologist and epidemiologist at Northwestern University who worked to develop PREVENT. “Do you do more good than harm when race is included as a predictor, or do you do more harm than good?”

It was the same question some cardiologists had been asking since 2013, when race was included in the risk calculator for atherosclerotic cardiovascular disease (ASCVD) used in Wright’s appointment.

Since 1998, clinicians had estimated risk using equations based on data from the National Institutes of Health’s storied, long-running Framingham Heart Study, which began tracking patients in a town outside Boston in 1948. The problem is, those patients were nearly all white.

When cardiologists began developing the next generation of equations, they aimed to be more equitable by including data on Black Americans. “At any given level of risk factors, Black Americans were at higher risk for heart attacks and strokes than white Americans were,” said Donald Lloyd-Jones, a cardiovascular epidemiologist and former AHA president who co-developed the so-called pooled cohort equations.

Their risk predictions were still imperfect: They were based on relatively few Black patients, and had barely any data from other racial groups, including Hispanic and Asian Americans. “On balance, it meant that we were actually probably slightly over-predicting risk, and therefore potentially slightly over-treating Black Americans,” said Lloyd-Jones.

As cardiologists began to question the use of race in their tools again, some wondered whether that overprediction was a problem. Statins are underused by many who need them; was it really so problematic to put them in the hands of some extra patients who don’t? “The ASCVD estimator tool potentially skews care in a disparate way that drives therapy to patients of color, as opposed to depriving patients of color,” said Wright. “That’s almost a flip of the way that disparities have played out where race is involved.”

Khan and her colleagues had these concerns in mind as they began work on the PREVENT calculator, using the electronic health records and research participation of a racially diverse group of 3 million patients. They wanted to be very careful that leaving out race wouldn’t underestimate the risk for Black patients. “We want to make sure that the PREVENT calculator works accurately and precisely for everyone,” said Khan. “But our question was, can we do it without including race as a predictor?”

After testing its predictive performance against another pool of 3 million, their answer was clear. While the previous ASCVD calculator overestimated risk, PREVENT performed well across ages, sexes, and racial and ethnic groups.

Even more meaningful is what the calculator added. “Race is a stand-in,” said Lloyd-Jones. “It’s a surrogate for these other things that are more directly related to your health status than the color of your skin.” Nutritional status. Access to education. Financial well-being. All of these can impact cardiovascular health over time.

PREVENT includes these social factors in a bid to refine its predictions.

It does so by giving clinicians the option of factoring in a patient’s ZIP code to get more personalized risk estimates. Those five digits are used to call up a community’s social deprivation index, a combination of seven measures that reflect an area’s socioeconomic footing, including rates of poverty and unemployment.

“It’s not just a question of removing race,” said Mitchell Elkind, the AHA’s chief clinical science officer. “It’s actually a more active proposal to include social determinants of health.”

That’s still a controversial idea. So far, specialties that have removed race from their clinical tools have done so by replacing it with other clinical characteristics. Factoring in a patient’s hypertension resulted in similar accuracy in a tool to predict the risk of attempting a vaginal birth after a cesarean section, for example, and a calculator used to assess a child’s risk of urinary tract infections factored in fever duration and UTI history instead of race.

Many question whether adding social determinants to medical predictions could simply replicate the pitfalls of race correction. Instead of disadvantaging patients according to the color of their skin, they worry socially stratified calculators could do the same for patients of lower socioeconomic status.

And medicine’s understanding of these factors is still extremely crude. “Just walk down your street,” said Elkind. “There are plenty of people in your ZIP code who are going to have different social determinants of health.”

Khan and the other researchers who developed PREVENT would have liked to include more direct measures of a patient’s housing status, or income, or education. But the 6 million real-world health records they used to design and test the calculators rarely include those factors.

These data aren’t collected because some patients don’t want to share sensitive details about their lives, and some clinicians are uncomfortable asking. Many health systems struggle to muster the resources to collect a patient’s medical history, let alone a fuller life history. And even if they could collect more detailed information, it would take a whole other set of tools to make sure that doctors could do something useful with that information.

“It gets a little bit intimidating for doctors, because it’s no longer a pill. You’re talking about how systems are created to reinforce disadvantage,” said Arnab Ghosh, a social scientist and internal medicine doctor at Weill Cornell. “You’re taking this outside the realm of what’s comfortable for doctors.”

Doctors will soon have to get more comfortable, as the federal agencies that regulate health information technology and reimbursement and health care accreditation organizations begin to incentivize and require health systems to collect certain social data points.

The Centers for Medicare and Medicaid Services mandated that by May 2025, federally funded hospitals and health care systems report data on patients’ food insecurity, interpersonal safety, housing insecurity, transportation insecurity, and utilities. Standards are emerging to turn abstract concepts like community support and food security into quantifiable, prediction-friendly measures.

But even when medical records give a fuller picture of patients’ lives, it’s not a foregone conclusion that the new data will improve risk scores.

When researchers tested PREVENT, adding the optional ZIP-code-based deprivation index to its baseline risk factors only minimally improved the calculator’s ability to discriminate between patients with and without cardiovascular disease. And in a recent study, Ghosh found that replacing race in the ASCVD calculator with several social determinants didn’t improve its predictive abilities.

Social determinants impact health through complex mechanisms, he said: “Your access to health care, whether you trust the health care system, whether the doctors treat you in a particular way.” Those influences are baked into a lifetime of diagnoses and clinical readings. Teasing them out is a huge challenge for doctors — and potentially an insurmountable one for statistical models.

Wright frequently confronts the use of race in medicine, and so he wondered about the accuracy of the race-based calculator that led his doctor to recommend a statin. GAVIN MCINTYRE FOR STAT

Back at his doctor’s office, Joseph Wright knew all of the unspoken risks his race injected into his cardiovascular risk score, including the accumulated signal of decades of racism. Together, they manifested in a single percentage: his odds of having a stroke or a heart attack in the next 10 years. And he knew, for him — a well-educated, well-compensated physician in his mid-60s, even one who, OK, could stand to exercise more and lose a few pounds — the calculator could well be overestimating his personal risk.

Wright was meeting the doctor he saw that day for the first time. Without the benefit of knowing Wright as a person, and having given him physicals for years, it was easy for him to default to a number on a screen, one that said, without question, that Wright needed a statin.

“I would like to think that we can personalize our approaches to the care of individual patients as much as we can,” Wright reflected on the visit.

It’s not that going on a statin would be so bad. It’s a cheap drug, and though there are side effects, it’s relatively safe — the same reasons why researchers had determined that, on balance, it was acceptable for the ASCVD calculator to slightly overestimate risk for Black patients.

But Wright pushed back, and his doctor offered an objective data point: a coronary calcium score to measure the level of plaque build-up in his arteries. They’d send Wright in for a CT scan, then see where they were.

As he waited for the results, Wright went online. On January 11, the AHA had published the PREVENT calculator for doctors to use, two months after it had shared the research behind its development. No risk predictor is perfectly accurate for every patient: Using the new, race-redacted tool, Wright wasn’t surprised to see his 10-year risk had dropped from intermediate to borderline. He’d been right, he thought, to question taking those statins.

In a recent analysis, researchers showed that the number of Americans flagged as needing statins could drop by up to a third if the new PREVENT tool was widely implemented in the same manner as the old tool — from 45.4 million to 28.3 million. The groups most impacted by that recalculation? Black patients and adults between 70 and 75.

PREVENT, its creators acknowledge, is an experiment. If it can retain accuracy in predicting heart disease risk without using race, that is considered one success. It’s not meant to solve all of cardiology’s problems, or reduce the massive disparities in heart disease, which have complex and multiple roots.

The ultimate goal is for PREVENT to replace the ASCVD calculator in clinical guidelines, said Elkind. But that won’t happen until researchers determine how PREVENT — and its proposal to incorporate social drivers into risk prediction — should be used in real-world clinical practice, a process that could take years.

In the meantime, many physicians will continue to use the race-based ASCVD equations. For patients — especially those who aren’t uniquely positioned to advocate for themselves like Wright was — the work can’t happen fast enough. Even after a clinical guideline changes, health systems and individual clinicians are often slow to adopt new ways of doing things.

“My concern is, how long will it take?” said Wright. When will the new race-free scores pop up during patients’ visits? And how much longer will it be before a physician can look at a patient’s social risk and begin to offer community resources alongside a prescription? “I think that’s the real challenge,” he said.

A few days later, Wright’s calcium score came in. It was zero.

Wright was surprised to see the reality of race-based predictions play out so plainly, to see his actual biology refute the calculator’s score, in black and white.

His doctor wrote him a note: They’d hold off on the daily statin.

STAT’s coverage of  health inequities is supported by a grant from the Commonwealth Fund. Our financial supporters are not involved in any decisions about our journalism.


link

By admin

Leave a Reply

Your email address will not be published. Required fields are marked *