Mental health

Subterms

    Latest story

    100 Shares149 Views

    Subtle biases in AI can influence emergency decisions

    by

    It’s no secret that people harbor biases — some unconscious, perhaps, and others painfully overt. The average person might suppose that computers — machines typically made of plastic, steel, glass, silicon, and various metals — are free of prejudice. While that assumption may hold for computer hardware, the same is not always true for computer software, which is programmed by fallible humans and can be fed data that is, itself, compromised in certain respects.

    Artificial intelligence (AI) systems — those based on machine learning, in particular — are seeing increased use in medicine for diagnosing specific diseases, for example, or evaluating X-rays. These systems are also being relied on to support decision-making in other areas of health care. Recent research has shown, however, that machine learning models can encode biases against minority subgroups, and the recommendations they make may consequently reflect those same biases.

    A new study by researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the MIT Jameel Clinic, which was published last month in Communications Medicine, assesses the impact that discriminatory AI models can have, especially for systems that are intended to provide advice in urgent situations. “We found that the manner in which the advice is framed can have significant repercussions,” explains the paper’s lead author, Hammaad Adam, a PhD student at MIT’s Institute for Data Systems and Society. “Fortunately, the harm caused by biased models can be limited (though not necessarily eliminated) when the advice is presented in a different way.” The other co-authors of the paper are Aparna Balagopalan and Emily Alsentzer, both PhD students, and the professors Fotini Christia and Marzyeh Ghassemi.

    AI models used in medicine can suffer from inaccuracies and inconsistencies, in part because the data used to train the models are often not representative of real-world settings. Different kinds of X-ray machines, for instance, can record things differently and hence yield different results. Models trained predominately on white people, moreover, may not be as accurate when applied to other groups. The Communications Medicine paper is not focused on issues of that sort but instead addresses problems that stem from biases and on ways to mitigate the adverse consequences.

    A group of 954 people (438 clinicians and 516 nonexperts) took part in an experiment to see how AI biases can affect decision-making. The participants were presented with call summaries from a fictitious crisis hotline, each involving a male individual undergoing a mental health emergency. The summaries contained information as to whether the individual was Caucasian or African American and would also mention his religion if he happened to be Muslim. A typical call summary might describe a circumstance in which an African American man was found at home in a delirious state, indicating that “he has not consumed any drugs or alcohol, as he is a practicing Muslim.” Study participants were instructed to call the police if they thought the patient was likely to turn violent; otherwise, they were encouraged to seek medical help.

    The participants were randomly divided into a control or “baseline” group plus four other groups designed to test responses under slightly different conditions. “We want to understand how biased models can influence decisions, but we first need to understand how human biases can affect the decision-making process,” Adam notes. What they found in their analysis of the baseline group was rather surprising: “In the setting we considered, human participants did not exhibit any biases. That doesn’t mean that humans are not biased, but the way we conveyed information about a person’s race and religion, evidently, was not strong enough to elicit their biases.”

    The other four groups in the experiment were given advice that either came from a biased or unbiased model, and that advice was presented in either a “prescriptive” or a “descriptive” form. A biased model would be more likely to recommend police help in a situation involving an African American or Muslim person than would an unbiased model. Participants in the study, however, did not know which kind of model their advice came from, or even that models delivering the advice could be biased at all. Prescriptive advice spells out what a participant should do in unambiguous terms, telling them they should call the police in one instance or seek medical help in another. Descriptive advice is less direct: A flag is displayed to show that the AI system perceives a risk of violence associated with a particular call; no flag is shown if the threat of violence is deemed small.  

    A key takeaway of the experiment is that participants “were highly influenced by prescriptive recommendations from a biased AI system,” the authors wrote. But they also found that “using descriptive rather than prescriptive recommendations allowed participants to retain their original, unbiased decision-making.” In other words, the bias incorporated within an AI model can be diminished by appropriately framing the advice that’s rendered. Why the different outcomes, depending on how advice is posed? When someone is told to do something, like call the police, that leaves little room for doubt, Adam explains. However, when the situation is merely described — classified with or without the presence of a flag — “that leaves room for a participant’s own interpretation; it allows them to be more flexible and consider the situation for themselves.”

    Second, the researchers found that the language models that are typically used to offer advice are easy to bias. Language models represent a class of machine learning systems that are trained on text, such as the entire contents of Wikipedia and other web material. When these models are “fine-tuned” by relying on a much smaller subset of data for training purposes — just 2,000 sentences, as opposed to 8 million web pages — the resultant models can be readily biased.  

    Third, the MIT team discovered that decision-makers who are themselves unbiased can still be misled by the recommendations provided by biased models. Medical training (or the lack thereof) did not change responses in a discernible way. “Clinicians were influenced by biased models as much as non-experts were,” the authors stated.

    “These findings could be applicable to other settings,” Adam says, and are not necessarily restricted to health care situations. When it comes to deciding which people should receive a job interview, a biased model could be more likely to turn down Black applicants. The results could be different, however, if instead of explicitly (and prescriptively) telling an employer to “reject this applicant,” a descriptive flag is attached to the file to indicate the applicant’s “possible lack of experience.”

    The implications of this work are broader than just figuring out how to deal with individuals in the midst of mental health crises, Adam maintains.  “Our ultimate goal is to make sure that machine learning models are used in a fair, safe, and robust way.” More

    More stories

    • 63 Shares149 Views

      in Data Management & Statistics

      Meet the 2022-23 Accenture Fellows

      by

      Launched in October 2020, the MIT and Accenture Convergence Initiative for Industry and Technology underscores the ways in which industry and technology can collaborate to spur innovation. The five-year initiative aims to achieve its mission through research, education, and fellowships. To that end, Accenture has once again awarded five annual fellowships to MIT graduate students working on research in industry and technology convergence who are underrepresented, including by race, ethnicity, and gender.This year’s Accenture Fellows work across research areas including telemonitoring, human-computer interactions, operations research,  AI-mediated socialization, and chemical transformations. Their research covers a wide array of projects, including designing low-power processing hardware for telehealth applications; applying machine learning to streamline and improve business operations; improving mental health care through artificial intelligence; and using machine learning to understand the environmental and health consequences of complex chemical reactions.As part of the application process, student nominations were invited from each unit within the School of Engineering, as well as from the Institute’s four other schools and the MIT Schwarzman College of Computing. Five exceptional students were selected as fellows for the initiative’s third year.Drew Buzzell is a doctoral candidate in electrical engineering and computer science whose research concerns telemonitoring, a fast-growing sphere of telehealth in which information is collected through internet-of-things (IoT) connected devices and transmitted to the cloud. Currently, the high volume of information involved in telemonitoring — and the time and energy costs of processing it — make data analysis difficult. Buzzell’s work is focused on edge computing, a new computing architecture that seeks to address these challenges by managing data closer to the source, in a distributed network of IoT devices. Buzzell earned his BS in physics and engineering science and his MS in engineering science from the Pennsylvania State University.

      Mengying (Cathy) Fang is a master’s student in the MIT School of Architecture and Planning. Her research focuses on augmented reality and virtual reality platforms. Fang is developing novel sensors and machine components that combine computation, materials science, and engineering. Moving forward, she will explore topics including soft robotics techniques that could be integrated with clothes and wearable devices and haptic feedback in order to develop interactions with digital objects. Fang earned a BS in mechanical engineering and human-computer interaction from Carnegie Mellon University.

      Xiaoyue Gong is a doctoral candidate in operations research at the MIT Sloan School of Management. Her research aims to harness the power of machine learning and data science to reduce inefficiencies in the operation of businesses, organizations, and society. With the support of an Accenture Fellowship, Gong seeks to find solutions to operational problems by designing reinforcement learning methods and other machine learning techniques to embedded operational problems. Gong earned a BS in honors mathematics and interactive media arts from New York University.

      Ruby Liu is a doctoral candidate in medical engineering and medical physics. Their research addresses the growing pandemic of loneliness among older adults, which leads to poor health outcomes and presents particularly high risks for historically marginalized people, including members of the LGBTQ+ community and people of color. Liu is designing a network of interconnected AI agents that foster connections between user and agent, offering mental health care while strengthening and facilitating human-human connections. Liu received a BS in biomedical engineering from Johns Hopkins University.

      Joules Provenzano is a doctoral candidate in chemical engineering. Their work integrates machine learning and liquid chromatography-high resolution mass spectrometry (LC-HRMS) to improve our understanding of complex chemical reactions in the environment. As an Accenture Fellow, Provenzano will build upon recent advances in machine learning and LC-HRMS, including novel algorithms for processing real, experimental HR-MS data and new approaches in extracting structure-transformation rules and kinetics. Their research could speed the pace of discovery in the chemical sciences and benefits industries including oil and gas, pharmaceuticals, and agriculture. Provenzano earned a BS in chemical engineering and international and global studies from the Rochester Institute of Technology. More

    • 100 Shares99 Views

      in Data Management & Statistics

      How artificial intelligence can help combat systemic racism

      by

      In 2020, Detroit police arrested a Black man for shoplifting almost $4,000 worth of watches from an upscale boutique. He was handcuffed in front of his family and spent a night in lockup. After some questioning, however, it became clear that they had the wrong man. So why did they arrest him in the first place?

      The reason: a facial recognition algorithm had matched the photo on his driver’s license to grainy security camera footage.

      Facial recognition algorithms — which have repeatedly been demonstrated to be less accurate for people with darker skin — are just one example of how racial bias gets replicated within and perpetuated by emerging technologies.

      “There’s an urgency as AI is used to make really high-stakes decisions,” says MLK Visiting Professor S. Craig Watkins, whose academic home for his time at MIT is the Institute for Data, Systems, and Society (IDSS). “The stakes are higher because new systems can replicate historical biases at scale.”

      Watkins, a professor at the University of Texas at Austin and the founding director of the Institute for Media Innovation​, researches the impacts of media and data-based systems on human behavior, with a specific concentration on issues related to systemic racism. “One of the fundamental questions of the work is: how do we build AI models that deal with systemic inequality more effectively?”

      Play video

      Artificial Intelligence and the Future of Racial Justice | S. Craig Watkins | TEDxMIT

      Ethical AI

      Inequality is perpetuated by technology in many ways across many sectors. One broad domain is health care, where Watkins says inequity shows up in both quality of and access to care. The demand for mental health care, for example, far outstrips the capacity for services in the United States. That demand has been exacerbated by the pandemic, and access to care is harder for communities of color.

      For Watkins, taking the bias out of the algorithm is just one component of building more ethical AI. He works also to develop tools and platforms that can address inequality outside of tech head-on. In the case of mental health access, this entails developing a tool to help mental health providers deliver care more efficiently.

      “We are building a real-time data collection platform that looks at activities and behaviors and tries to identify patterns and contexts in which certain mental states emerge,” says Watkins. “The goal is to provide data-informed insights to care providers in order to deliver higher-impact services.”

      Watkins is no stranger to the privacy concerns such an app would raise. He takes a user-centered approach to the development that is grounded in data ethics. “Data rights are a significant component,” he argues. “You have to give the user complete control over how their data is shared and used and what data a care provider sees. No one else has access.”

      Combating systemic racism

      Here at MIT, Watkins has joined the newly launched Initiative on Combatting Systemic Racism (ICSR), an IDSS research collaboration that brings together faculty and researchers from the MIT Stephen A. Schwarzman College of Computing and beyond. The aim of the ICSR is to develop and harness computational tools that can help effect structural and normative change toward racial equity.

      The ICSR collaboration has separate project teams researching systemic racism in different sectors of society, including health care. Each of these “verticals” addresses different but interconnected issues, from sustainability to employment to gaming. Watkins is a part of two ICSR groups, policing and housing, that aim to better understand the processes that lead to discriminatory practices in both sectors. “Discrimination in housing contributes significantly to the racial wealth gap in the U.S.,” says Watkins.

      The policing team examines patterns in how different populations get policed. “There is obviously a significant and charged history to policing and race in America,” says Watkins. “This is an attempt to understand, to identify patterns, and note regional differences.”

      Watkins and the policing team are building models using data that details police interventions, responses, and race, among other variables. The ICSR is a good fit for this kind of research, says Watkins, who notes the interdisciplinary focus of both IDSS and the SCC. 

      “Systemic change requires a collaborative model and different expertise,” says Watkins. “We are trying to maximize influence and potential on the computational side, but we won’t get there with computation alone.”

      Opportunities for change

      Models can also predict outcomes, but Watkins is careful to point out that no algorithm alone will solve racial challenges.

      “Models in my view can inform policy and strategy that we as humans have to create. Computational models can inform and generate knowledge, but that doesn’t equate with change.” It takes additional work — and additional expertise in policy and advocacy — to use knowledge and insights to strive toward progress.

      One important lever of change, he argues, will be building a more AI-literate society through access to information and opportunities to understand AI and its impact in a more dynamic way. He hopes to see greater data rights and greater understanding of how societal systems impact our lives.

      “I was inspired by the response of younger people to the murders of George Floyd and Breonna Taylor,” he says. “Their tragic deaths shine a bright light on the real-world implications of structural racism and has forced the broader society to pay more attention to this issue, which creates more opportunities for change.” More

    • 125 Shares199 Views

      in Data Management & Statistics

      New integrative computational neuroscience center established at MIT’s McGovern Institute

      by

      With the tools of modern neuroscience, researchers can peer into the brain with unprecedented accuracy. Recording devices listen in on the electrical conversations between neurons, picking up the voices of hundreds of cells at a time. Genetic tools allow us to focus on specific types of neurons based on their molecular signatures. Microscopes zoom in to illuminate the brain’s circuitry, capturing thousands of images of elaborately branched dendrites. Functional MRIs detect changes in blood flow to map activity within a person’s brain, generating a complete picture by compiling hundreds of scans.

      This deluge of data provides insights into brain function and dynamics at different levels — molecules, cells, circuits, and behavior — but the insights remain compartmentalized in separate research silos for each level. An innovative new center at MIT’s McGovern Institute for Brain Research aims to leverage them into powerful revelations of the brain’s inner workings.

      The K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center will create advanced mathematical models and computational tools to synthesize the deluge of data across scales and advance our understanding of the brain and mental health.

      The center, funded by a $24 million donation from philanthropist Lisa Yang and led by McGovern Institute Associate Investigator Ila Fiete, will take a collaborative approach to computational neuroscience, integrating cutting-edge modeling techniques and data from MIT labs to explain brain function at every level, from the molecular to the behavioral.

      “Our goal is that sophisticated, truly integrated computational models of the brain will make it possible to identify how ‘control knobs’ such as genes, proteins, chemicals, and environment drive thoughts and behavior, and to make inroads toward urgent unmet needs in understanding and treating brain disorders,” says Fiete, who is also a brain and cognitive sciences professor at MIT.

      “Driven by technologies that generate massive amounts of data, we are entering a new era of translational neuroscience research,” says Yang, whose philanthropic investment in MIT research now exceeds $130 million. “I am confident that the multidisciplinary expertise convened by the ICoN center will revolutionize how we synthesize this data and ultimately understand the brain in health and disease.”

      Connecting the data

      It is impossible to separate the molecules in the brain from their effects on behavior — although those aspects of neuroscience have traditionally been studied independently, by researchers with vastly different expertise. The ICoN Center will eliminate the divides, bringing together neuroscientists and software engineers to deal with all types of data about the brain.

      “The center’s highly collaborative structure, which is essential for unifying multiple levels of understanding, will enable us to recruit talented young scientists eager to revolutionize the field of computational neuroscience,” says Robert Desimone, director of the McGovern Institute. “It is our hope that the ICoN Center’s unique research environment will truly demonstrate a new academic research structure that catalyzes bold, creative research.”

      To foster interdisciplinary collaboration, every postdoc and engineer at the center will work with multiple faculty mentors. In order to attract young scientists and engineers to the field of computational neuroscience, the center will also provide four graduate fellowships to MIT students each year in perpetuity. Interacting closely with three scientific cores, engineers and fellows will develop computational models and technologies for analyzing molecular data, neural circuits, and behavior, such as tools to identify patterns in neural recordings or automate the analysis of human behavior to aid psychiatric diagnoses. These technologies and models will be instrumental in synthesizing data into knowledge and understanding.

      Center priorities

      In its first five years, the ICoN Center will prioritize four areas of investigation: episodic memory and exploration, including functions like navigation and spatial memory; complex or stereotypical behavior, such as the perseverative behaviors associated with autism and obsessive-compulsive disorder; cognition and attention; and sleep. Models of complex behavior will be created in collaboration with clinicians and researchers at Children’s Hospital of Philadelphia.

      The goal, Fiete says, is to model the neuronal interactions that underlie these functions so that researchers can predict what will happen when something changes — when certain neurons become more active or when a genetic mutation is introduced, for example. When paired with experimental data from MIT labs, the center’s models will help explain not just how these circuits work, but also how they are altered by genes, the environment, aging, and disease. These focus areas encompass circuits and behaviors often affected by psychiatric disorders and neurodegeneration, and models will give researchers new opportunities to explore their origins and potential treatment strategies.

      “Lisa Yang is focused on helping the scientific community realize its goals in translational research,” says Nergis Mavalvala, dean of the School of Science and the Curtis and Kathleen Marble Professor of Astrophysics. “With her generous support, we can accelerate the pace of research by connecting the data to the delivery of tangible results.” More

    • 63 Shares99 Views

      in Data Management & Statistics

      Finding common ground in Malden

      by

      When disparate groups convene around a common goal, exciting things can happen.

      That is the inspiring story unfolding in Malden, Massachusetts, a city of about 60,000 — nearly half people of color — where a new type of community coalition continues to gain momentum on its plan to build a climate-resilient waterfront park along its river. The Malden River Works (MRW) project, recipient of the inaugural Leventhal City Prize, is seeking to connect to a contiguous greenway network where neighboring cities already have visitors coming to their parks and enjoying recreational boating. More important, the MRW is changing the model for how cities address civic growth, community engagement, equitable climate resilience, and environmental justice.                                                                                        

      The MRW’s steering committee consists of eight resident leaders of color, a resident environmental advocate, and three city representatives. One of the committee’s primary responsibilities is providing direction to the MRW’s project team, which includes urban designers, watershed and climate resilience planners, and a community outreach specialist. MIT’s Kathleen Vandiver, director of the Community Outreach Education and Engagement Core at MIT’s Center for Environmental Health Sciences (CEHS), and Marie Law Adams MArch ’06, a lecturer in the School of Architecture and Planning’s Department of Urban Studies and Planning (DUSP), serve on the project team.

      “This governance structure is somewhat unusual,” says Adams. “More typical is having city government as the primary decision-maker. It is important that one of the first things our team did was build a steering committee that is the decision maker on this project.”

      Evan Spetrini ’18 is the senior planner and policy manager for the Malden Redevelopment Authority and sits on both the steering committee and project team. He says placing the decision-making power with the steering committee and building it to be representative of marginalized communities was intentional. 

      “Changing that paradigm of power and decision-making in planning processes was the way we approached social resilience,” says Spetrini. “We have always intended this project to be a model for future planning projects in Malden.”

      This model ushers in a new history chapter for a city founded in 1640.

      Located about six miles north of Boston, Malden was home to mills and factories that used the Malden River for power, and a site for industrial waste over the last two centuries. Decades after the city’s industrial decline, there is little to no public access to the river. Many residents were not even aware there was a river in their city. Before the project was under way, Vandiver initiated a collaborative effort to evaluate the quality of the river’s water. Working with the Mystic River Watershed Association, Gradient Corporation, and CEHS, water samples were tested and a risk analysis conducted.

      “Having the study done made it clear the public could safely enjoy boating on the water,” says Vandiver. “It was a breakthrough that allowed people to see the river as an amenity.”

      A team effort

      Marcia Manong had never seen the river, but the Malden resident was persuaded to join the steering committee with the promise the project would be inclusive and of value to the community. Manong has been involved with civic engagement most of her life in the United States and for 20 years in South Africa.

      “It wasn’t going to be a marginalized, token-ized engagement,” says Manong. “It was clear to me that they were looking for people that would actually be sitting at the table.”

      Manong agreed to recruit additional people of color to join the team. From the beginning, she says, language was a huge barrier, given that nearly half of Malden’s residents do not speak English at home. Finding the translation efforts at their public events to be inadequate, the steering committee directed more funds to be made available for translation in several languages when public meetings began being held over Zoom this past year.

      “It’s unusual for most cities to spend this money, but our population is so diverse that we require it,” says Manong. “We have to do it. If the steering committee wasn’t raising this issue with the rest of the team, perhaps this would be overlooked.”

      Another alteration the steering committee has made is how the project engages with the community. While public attendance at meetings had been successful before the pandemic, Manong says they are “constantly working” to reach new people. One method has been to request invitations to attend the virtual meetings of other organizations to keep them apprised of the project.

      “We’ve said that people feel most comfortable when they’re in their own surroundings, so why not go where the people are instead of trying to get them to where we are,” says Manong.

      Buoyed by the $100,000 grant from MIT’s Norman B. Leventhal Center for Advanced Urbanism (LCAU) in 2019, the project team worked with Malden’s Department of Public Works, which is located along the river, to redesign its site and buildings and to study how to create a flood-resistant public open space as well as an elevated greenway path, connecting with other neighboring cities’ paths. The park’s plans also call for 75 new trees to reduce urban heat island effect, open lawn for gathering, and a dock for boating on the river.

      “The storm water infrastructure in these cities is old and isn’t going to be able to keep up with increased precipitation,” says Adams. “We’re looking for ways to store as much water as possible on the DPW site so we can hold it and release it more gradually into the river to avoid flooding.”

      The project along the 2.3-mile-long river continues to receive attention. Recently, the city of Malden was awarded a 2021 Accelerating Climate Resilience Grant of more than $50,000 from the state’s Metropolitan Area Planning Council and the Barr Foundation to support the project. Last fall, the project was awarded a $150,015 Municipal Vulnerability Preparedness Action Grant. Both awards are being directed to fund engineering work to refine the project’s design.

      “We — and in general, the planning profession — are striving to create more community empowerment in decision-making as to what happens to their community,” says Spetrini. “Putting the power in the community ensures that it’s actually responding to the needs of the community.”

      Contagious enthusiasm

      Manong says she’s happy she got involved with the project and believes the new governance structure is making a difference.

      “This project is definitely engaging with communities of color in a manner that is transformative and that is looking to build a long-lasting power dynamic built on trust,” she says. “It’s a new energized civic engagement and we’re making that happen. It’s very exciting.”

      Spetrini finds the challenge of creating an open space that’s publicly accessible and alongside an active work site professionally compelling.

      “There is a way to preserve the industrial employment base while also giving the public greater access to this natural resource,” he says. “It has real implications for other communities to follow this type of model.”

      Despite the pandemic this past year, enthusiasm for the project is palpable. For Spetrini, a Malden resident, it’s building “the first significant piece of what has been envisioned as the Malden River Greenway.” Adams sees the total project as a way to build social resilience as well as garnering community interest in climate resilience. For Vandiver, it’s the implications for improved community access.

      “From a health standpoint, everybody has learned from Covid-19 that the health aspects of walking in nature are really restorative,” says Vandiver. “Creating greater green space gives more attention to health issues. These are seemingly small side benefits, but they’re huge for mental health benefits.”

      Leventhal City Prize’s next cycle

      The Leventhal City Prize was established by the LCAU to catalyze innovative, interdisciplinary urban design, and planning approaches worldwide to improve both the environment and the quality of life for residents. Support for the LCAU was provided by the Muriel and Norman B. Leventhal Family Foundation and the Sherry and Alan Leventhal Family Foundation.

      “We’re thrilled with inaugural recipients of the award and the extensive work they’ve undertaken that is being held up as an exemplary model for others to learn from,” says Sarah Williams, LCAU director and a professor in DUSP. “Their work reflects the prize’s intent. We look forward to catalyzing these types of collaborative partnership in the next prize cycle.”

      Submissions for the next cycle of the Leventhal City Prize will open in early 2022.    More