Race and gender

Subterms

    More stories

    • 100 Shares149 Views

      in Data Management & Statistics

      The downside of machine learning in health care

      by

      While working toward her dissertation in computer science at MIT, Marzyeh Ghassemi wrote several papers on how machine-learning techniques from artificial intelligence could be applied to clinical data in order to predict patient outcomes. “It wasn’t until the end of my PhD work that one of my committee members asked: ‘Did you ever check to see how well your model worked across different groups of people?’”

      That question was eye-opening for Ghassemi, who had previously assessed the performance of models in aggregate, across all patients. Upon a closer look, she saw that models often worked differently — specifically worse — for populations including Black women, a revelation that took her by surprise. “I hadn’t made the connection beforehand that health disparities would translate directly to model disparities,” she says. “And given that I am a visible minority woman-identifying computer scientist at MIT, I am reasonably certain that many others weren’t aware of this either.”

      In a paper published Jan. 14 in the journal Patterns, Ghassemi — who earned her doctorate in 2017 and is now an assistant professor in the Department of Electrical Engineering and Computer Science and the MIT Institute for Medical Engineering and Science (IMES) — and her coauthor, Elaine Okanyene Nsoesie of Boston University, offer a cautionary note about the prospects for AI in medicine. “If used carefully, this technology could improve performance in health care and potentially reduce inequities,” Ghassemi says. “But if we’re not actually careful, technology could worsen care.”

      It all comes down to data, given that the AI tools in question train themselves by processing and analyzing vast quantities of data. But the data they are given are produced by humans, who are fallible and whose judgments may be clouded by the fact that they interact differently with patients depending on their age, gender, and race, without even knowing it.

      Furthermore, there is still great uncertainty about medical conditions themselves. “Doctors trained at the same medical school for 10 years can, and often do, disagree about a patient’s diagnosis,” Ghassemi says. That’s different from the applications where existing machine-learning algorithms excel — like object-recognition tasks — because practically everyone in the world will agree that a dog is, in fact, a dog.

      Machine-learning algorithms have also fared well in mastering games like chess and Go, where both the rules and the “win conditions” are clearly defined. Physicians, however, don’t always concur on the rules for treating patients, and even the win condition of being “healthy” is not widely agreed upon. “Doctors know what it means to be sick,” Ghassemi explains, “and we have the most data for people when they are sickest. But we don’t get much data from people when they are healthy because they’re less likely to see doctors then.”

      Even mechanical devices can contribute to flawed data and disparities in treatment. Pulse oximeters, for example, which have been calibrated predominately on light-skinned individuals, do not accurately measure blood oxygen levels for people with darker skin. And these deficiencies are most acute when oxygen levels are low — precisely when accurate readings are most urgent. Similarly, women face increased risks during “metal-on-metal” hip replacements, Ghassemi and Nsoesie write, “due in part to anatomic differences that aren’t taken into account in implant design.” Facts like these could be buried within the data fed to computer models whose output will be undermined as a result.

      Coming from computers, the product of machine-learning algorithms offers “the sheen of objectivity,” according to Ghassemi. But that can be deceptive and dangerous, because it’s harder to ferret out the faulty data supplied en masse to a computer than it is to discount the recommendations of a single possibly inept (and maybe even racist) doctor. “The problem is not machine learning itself,” she insists. “It’s people. Human caregivers generate bad data sometimes because they are not perfect.”

      Nevertheless, she still believes that machine learning can offer benefits in health care in terms of more efficient and fairer recommendations and practices. One key to realizing the promise of machine learning in health care is to improve the quality of data, which is no easy task. “Imagine if we could take data from doctors that have the best performance and share that with other doctors that have less training and experience,” Ghassemi says. “We really need to collect this data and audit it.”

      The challenge here is that the collection of data is not incentivized or rewarded, she notes. “It’s not easy to get a grant for that, or ask students to spend time on it. And data providers might say, ‘Why should I give my data out for free when I can sell it to a company for millions?’ But researchers should be able to access data without having to deal with questions like: ‘What paper will I get my name on in exchange for giving you access to data that sits at my institution?’

      “The only way to get better health care is to get better data,” Ghassemi says, “and the only way to get better data is to incentivize its release.”

      It’s not only a question of collecting data. There’s also the matter of who will collect it and vet it. Ghassemi recommends assembling diverse groups of researchers — clinicians, statisticians, medical ethicists, and computer scientists — to first gather diverse patient data and then “focus on developing fair and equitable improvements in health care that can be deployed in not just one advanced medical setting, but in a wide range of medical settings.”

      The objective of the Patterns paper is not to discourage technologists from bringing their expertise in machine learning to the medical world, she says. “They just need to be cognizant of the gaps that appear in treatment and other complexities that ought to be considered before giving their stamp of approval to a particular computer model.” More

    • 88 Shares129 Views

      in Data Management & Statistics

      Exploring the human stories behind the data

      by

      Shaking in the back of a police cruiser, handcuffs digging into his wrists, Brian Williams was overwhelmed with fear. He had been pulled over, but before he was asked for his name, license, or registration, a police officer ordered him out of his car and into back of the police cruiser, saying into his radio, “Black male detained.” The officer’s explanation for these actions was: “for your safety and mine.”

      Williams walked away from the experience with two tickets, a pair of bruised wrists, and a desire to do everything in his power to prevent others from experiencing the utter powerlessness he had felt.

      Now an MIT senior majoring in biological engineering and minoring in Black studies, Williams has continued working to empower his community. Through experiences in and out of the classroom, he has leveraged his background in bioengineering to explore interests in public health and social justice, specifically looking at how the medical sector can uplift and support communities of color.

      Williams grew up in a close-knit family and community in Broward County, Florida, where he found comfort in the routine of Sunday church services, playing outside with friends, and cookouts on the weekends. Broward County was home to him — a home he felt deeply invested in and indebted to.

      “It takes a village. The Black community has invested a lot in me, and I have a lot to invest back in it,” he says.

      Williams initially focused on STEM subjects at MIT, but in his sophomore year, his interests in exploring data science and humanities research led him to an Undergraduate Research Opportunities Program (UROP) project in the Department of Political Science. Working with Professor Ariel White, he analyzed information on incarceration and voting rights, studied the behavior patterns of police officers, and screened 911 calls to identify correlations between how people described events to how the police responded to them.

      In the summer before his junior year, Williams also joined MIT’s Civic Data Design Lab, where he worked as a researcher for the Missing Data Project, which uses both journalism and data science to visualize statistics and humanize the people behind the numbers. As the project’s name suggests, there is often much to be learned from seeking out data that aren’t easily available. Using datasets and interviews describing how the pandemic affected Black communities, Williams and a team of researchers created a series called the Color of Covid, which told the stories behind the grim statistics on race and the pandemic.

      The following year, Williams undertook a research-and-development internship with the biopharmaceutical company Amgen in San Francisco, working on protein engineering to combat autoimmune diseases. Because this work was primarily in the lab, focusing on science-based applications, he saw it as an opportunity to ask himself: “Do I want to dedicate my life to this area of bioengineering?” He found the issue of social justice to be more compelling.

      At the same time, Williams was drawn toward tackling problems the local Black community was experiencing related to the pandemic. He found himself thinking deeply about how to educate the public, address disparities in case rates, and, above all, help people.

      Working through Amgen’s Black Employee Resource Group and its Diversity, Inclusion, and Belonging Team, Williams crafted a proposal, which the company adopted, for addressing Covid-19 vaccination misinformation in Black and Brown communities in San Mateo and San Francisco County. He paid special attention to how to frame vaccine hesitancy among members of these communities, understanding that a longstanding history of racism in scientific discovery and medicine led many Black and Brown people to distrust the entire medical industry.

      “Trying to meet people where they are is important,” Williams says.

      This experience reinforced the idea for Williams that he wanted to do everything in his power to uplift the Black community.

      “I think it’s only right that I go out and I shine bright because it’s not easy being Black. You know, you have to work twice as hard to get half as much,” he says.

      As the current political action co-chair of the MIT Black Students’ Union (BSU), Williams also works to inspire change on campus, promoting and participating in events that uplift the BSU. During his Amgen internship, he also organized the MIT Black History Month Takeover Series, which involved organizing eight events from February through the beginning of spring semester. These included promotions through social media and virtual meetings for students and faculty. For his leadership, he received the “We Are Family” award from the BSU executive board.

      Williams prioritizes community in everything he does, whether in the classroom, at a campus event, or spending time outside in local communities of color around Boston.

      “The things that really keep me going are the stories of other people,” says Williams, who is currently applying to a variety of postgraduate programs. After receiving MIT endorsement, he applied to the Rhodes and Marshall Fellowships; he also plans to apply to law school with a joint master’s degree in public health and policy.

      Ultimately, Williams hopes to bring his fight for racial justice to the policy level, looking at how a long, ongoing history of medical racism has led marginalized communities to mistrust current scientific endeavors. He wants to help bring about new legislation to fix old systems which disproportionately harm communities of color. He says he aims to be “an engineer of social solutions, one who reaches deep into their toolbox of social justice, pulling the levers of activism, advocacy, democracy, and legislation to radically change our world — to improve our social institutions at the root and liberate our communities.” While he understands this is a big feat, he sees his ambition as an asset.

      “I’m just another person with huge aspirations, and an understanding that you have to go get it if you want it,” he says. “You feel me? At the end of the day, this is just the beginning of my story. And I’m grateful to everyone in my life that’s helping me write it. Tap in.” More