Data Management & Statistics

Subterms

    More stories

    • 75 Shares149 Views

      in Data Management & Statistics

      Can machine-learning models overcome biased datasets?

      by

      Artificial intelligence systems may be able to complete tasks quickly, but that doesn’t mean they always do so fairly. If the datasets used to train machine-learning models contain biased data, it is likely the system could exhibit that same bias when it makes decisions in practice.

      For instance, if a dataset contains mostly images of white men, then a facial-recognition model trained with these data may be less accurate for women or people with different skin tones.

      A group of researchers at MIT, in collaboration with researchers at Harvard University and Fujitsu Ltd., sought to understand when and how a machine-learning model is capable of overcoming this kind of dataset bias. They used an approach from neuroscience to study how training data affects whether an artificial neural network can learn to recognize objects it has not seen before. A neural network is a machine-learning model that mimics the human brain in the way it contains layers of interconnected nodes, or “neurons,” that process data.

      The new results show that diversity in training data has a major influence on whether a neural network is able to overcome bias, but at the same time dataset diversity can degrade the network’s performance. They also show that how a neural network is trained, and the specific types of neurons that emerge during the training process, can play a major role in whether it is able to overcome a biased dataset.

      “A neural network can overcome dataset bias, which is encouraging. But the main takeaway here is that we need to take into account data diversity. We need to stop thinking that if you just collect a ton of raw data, that is going to get you somewhere. We need to be very careful about how we design datasets in the first place,” says Xavier Boix, a research scientist in the Department of Brain and Cognitive Sciences (BCS) and the Center for Brains, Minds, and Machines (CBMM), and senior author of the paper.  

      Co-authors include former MIT graduate students Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, and Spandan Madan, a corresponding author who is currently pursuing a PhD at Harvard; Tomotake Sasaki, a former visiting scientist now a senior researcher at Fujitsu Research; Frédo Durand, a professor of electrical engineering and computer science at MIT and a member of the Computer Science and Artificial Intelligence Laboratory; and Hanspeter Pfister, the An Wang Professor of Computer Science at the Harvard School of Enginering and Applied Sciences. The research appears today in Nature Machine Intelligence.

      Thinking like a neuroscientist

      Boix and his colleagues approached the problem of dataset bias by thinking like neuroscientists. In neuroscience, Boix explains, it is common to use controlled datasets in experiments, meaning a dataset in which the researchers know as much as possible about the information it contains.

      The team built datasets that contained images of different objects in varied poses, and carefully controlled the combinations so some datasets had more diversity than others. In this case, a dataset had less diversity if it contains more images that show objects from only one viewpoint. A more diverse dataset had more images showing objects from multiple viewpoints. Each dataset contained the same number of images.

      The researchers used these carefully constructed datasets to train a neural network for image classification, and then studied how well it was able to identify objects from viewpoints the network did not see during training (known as an out-of-distribution combination). 

      For example, if researchers are training a model to classify cars in images, they want the model to learn what different cars look like. But if every Ford Thunderbird in the training dataset is shown from the front, when the trained model is given an image of a Ford Thunderbird shot from the side, it may misclassify it, even if it was trained on millions of car photos.

      The researchers found that if the dataset is more diverse — if more images show objects from different viewpoints — the network is better able to generalize to new images or viewpoints. Data diversity is key to overcoming bias, Boix says.

      “But it is not like more data diversity is always better; there is a tension here. When the neural network gets better at recognizing new things it hasn’t seen, then it will become harder for it to recognize things it has already seen,” he says.

      Testing training methods

      The researchers also studied methods for training the neural network.

      In machine learning, it is common to train a network to perform multiple tasks at the same time. The idea is that if a relationship exists between the tasks, the network will learn to perform each one better if it learns them together.

      But the researchers found the opposite to be true — a model trained separately for each task was able to overcome bias far better than a model trained for both tasks together.

      “The results were really striking. In fact, the first time we did this experiment, we thought it was a bug. It took us several weeks to realize it was a real result because it was so unexpected,” he says.

      They dove deeper inside the neural networks to understand why this occurs.

      They found that neuron specialization seems to play a major role. When the neural network is trained to recognize objects in images, it appears that two types of neurons emerge — one that specializes in recognizing the object category and another that specializes in recognizing the viewpoint.

      When the network is trained to perform tasks separately, those specialized neurons are more prominent, Boix explains. But if a network is trained to do both tasks simultaneously, some neurons become diluted and don’t specialize for one task. These unspecialized neurons are more likely to get confused, he says.

      “But the next question now is, how did these neurons get there? You train the neural network and they emerge from the learning process. No one told the network to include these types of neurons in its architecture. That is the fascinating thing,” he says.

      That is one area the researchers hope to explore with future work. They want to see if they can force a neural network to develop neurons with this specialization. They also want to apply their approach to more complex tasks, such as objects with complicated textures or varied illuminations.

      Boix is encouraged that a neural network can learn to overcome bias, and he is hopeful their work can inspire others to be more thoughtful about the datasets they are using in AI applications.

      This work was supported, in part, by the National Science Foundation, a Google Faculty Research Award, the Toyota Research Institute, the Center for Brains, Minds, and Machines, Fujitsu Research, and the MIT-Sensetime Alliance on Artificial Intelligence. More

    • 113 Shares179 Views

      in Data Management & Statistics

      Research aims to mitigate chemical and biological airborne threats

      by

      When the air harbors harmful matter, such as a virus or toxic chemical, it’s not always easy to promptly detect this danger. Whether spread maliciously or accidentally, how fast and how far could hazardous plumes travel through a city? What could emergency managers do in response?

      These were questions that scientists, public health officials, and government agencies probed with an air flow study conducted recently in New York City. At 120 locations across all five boroughs of the city, a team led by MIT Lincoln Laboratory collected safe test particles and gases released earlier in subway stations and on streets, tracking their journeys. The exercise measured how far the materials traveled and what their concentrations were when detected.

      The results are expected to improve air dispersion models, and in turn, help emergency planners improve response protocols if a real chemical or biological event were to take place. 

      The study was performed under the Department of Homeland Security (DHS) Science and Technology Directorate’s (S&T) Urban Threat Dispersion Project. The project is largely driven by Lincoln Laboratory’s Counter–Weapons of Mass Destruction (CWMD) Systems Group to improve homeland defenses against airborne threats. This exercise followed a similar, though much smaller, study in 2016 that focused mainly on the subway system within Manhattan.

      “The idea was to look at how particles and gases move through urban environments, starting with a focus on subways,” says Mandeep Virdi, a researcher in the CWMD Systems Group who helped lead both studies.

      The particles and gases used in the study are safe to disperse. The particulates are primarily composed of maltodextrin sugar, and have been used in prior public safety exercises. To enable researchers to track the particles, the particles are modified with small amounts of synthetic DNA that acts as a unique “barcode.” This barcode corresponds to the location from which the particle was released and the day of release. When these particles are later collected and analyzed, researchers can know exactly where they came from.

      The laboratory’s team led the process of releasing the particles and collecting the particle samples for analysis. A small sprayer is used to aerosolize the particles into the air. As the particles flow throughout the city, some get trapped in filters set up at the many dispersed collection sites. 

      To make processes more efficient for this large study, the team built special filter heads that rotated through multiple filters, saving time spent revisiting a collection site. They also developed a system using NFC (near-field communication) tags to simplify the cataloging and tracking of samples and equipment through a mobile app. 

      The researchers are still processing the approximately 5,000 samples that were collected over the five-day measurement campaign. The data will feed into existing particle dispersion models to improve simulations. One of these models, from Argonne National Laboratory, focuses on subway environments, and another model from Los Alamos National Laboratory simulates above-ground city environments, taking into account buildings and urban canyon air flows.

      Together, these models can show how a plume would travel from the subway to the streets, for example. These insights will enable emergency managers in New York City to develop more informed response strategies, as they did following the 2016 subway study.

      “The big question has always been, if there is a release and law enforcement can detect it in time, what do you actually do? Do you shut down the subway system? What can you do to mitigate those effects? Knowing that is the end goal,” Virdi says. 

      A new program, called the Chemical and Biological Defense Testbed, has just kicked off to further investigate those questions. Trina Vian at Lincoln Laboratory is leading this program, also under S&T funding.

      “Now that we’ve learned more about how material transports through the subway system, this test bed is looking at ways that we can mitigate that transport in a low-regret way,” Vian says.

      According to Vian, emergency managers don’t have many options other than to evacuate the area when a biological or chemical sensor is triggered. Yet current sensors tend to have high false-alarm rates, particularly in dirty environments. “You really can’t afford to make that evacuation call in error. Not only do you undermine people’s trust in the system, but also people can become injured, and it may actually be a non-threatening situation.”

      The goal of this test bed is to develop architectures and technologies that could allow for a range of appropriate response activities. For example, the team will be looking at ways through which air flow could be constrained or filtered in place, without disrupting traffic, while responders validate an alarm. They’ll also be testing the performance of new chemical and biological sensor technologies.

      Both Vian and Virdi stress the importance of collaboration for carrying out these large-scale studies, and in tackling the problem of airborne dangers in general. The test bed program is already benefiting by using equipment provided through the CWMD Alliance, a partnership of DHS and the Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense.

      A team of nearly 175 personnel worked together on the air flow exercise, spanning the Metropolitan Transportation Authority, New York City Transit, New York City Police Department, Port Authority of New York and New Jersey, New Jersey Transit, New York City Department of Environmental Protection, the New York City Department of Health and Mental Hygiene, the National Guard Weapons of Mass Destruction Civil Support Teams, the Environmental Protection Agency, and Department of Energy National Laboratories, in addition to S&T and Lincoln Laboratory.

      “It really was all about teamwork,” Virdi reflects. “Programs like this are why I came to Lincoln Laboratory. Seeing how the science is applied in a way that has real actionable results and how appreciative agencies are of what we’re doing has been rewarding. It’s exciting to see your program through, especially one as intense as this.” More

    • 100 Shares99 Views

      in Data Management & Statistics

      Professor Emery Brown has big plans for anesthesiology

      by

      Emery N. Brown — the Edward Hood Taplin Professor of Medical Engineering and of Computational Neuroscience at MIT, an MIT professor of health sciences and technology, an investigator with The Picower Institute for Learning and Memory at MIT, and the Warren M. Zapol Professor of Anaesthesia at Harvard Medical School and Massachusetts General Hospital (MGH) — clearly excels at many roles. Renowned internationally for his anesthesia and neuroscience research, he embodies a unique blend of anesthesiologist, statistician, neuroscientist, educator, and mentor to both students and colleagues. Notably, Brown is one of the most decorated clinician-scientists in the country; he is one of only 25 people — and the first African-American, statistician, and anesthesiologist — to be elected to all three National Academies (Science, Engineering, and Medicine).

      Now, he is handing off one of his many key roles and responsibilities. After almost 10 years, Brown is stepping down as co-director of the Harvard-MIT Program in Health Sciences and Technology (HST). He will turn his energies toward working to develop a new joint center between MIT and MGH that uses the study of anesthesia to design novel approaches to controlling brain states. While a goal of the new center will be to improve anesthesia and intensive care unit management, according to Brown, it will also study related problems such as treating depression, insomnia, and epilepsy, as well as enhancing coma recovery.

      Founded in 1970, HST is one of the oldest interdisciplinary educational programs focused on training the next generation of clinician-scientists and engineers, who learn to translate science, engineering, and medical research into clinical practice, with the aim of improving human health. The MIT Institute for Medical Engineering and Science (IMES), where Brown is associate director, is HST’s home at MIT. Brown was the first HST co-director after the establishment of IMES in 2012; Wolfram Goessling is the Harvard University co-director of HST.

      “Emery has been an exemplary leader for HST during his tenure, and has helped it become a hub for the training of world-class scientists, engineers, and clinicians,” says Anantha Chandrakasan, dean of the MIT School of Engineering and the Vannevar Bush Professor of Electrical Engineering and Computer Science. “I am deeply grateful for his many years of service and wish him well as he moves on to new endeavors.”

      Elazer R. Edelman, director of IMES, calls Brown “a phenom who has been dedicated to our programs for years.”

      “With his thoughtful leadership and understated style, Emery made many contributions to the HST community,” Edelman continues. “On a personal note, this is bittersweet for me, as Emery has been a partner and mentor in my role as IMES director. And while I know that he will always be there for me, as he has been for all of us at IMES and HST, I will miss our late-night calls and midday conferences on matters of import for MIT, IMES, and HST.”

      Brown says “it was an honor and a privilege to co-direct HST with Wolfram.”

      “The students, staff, and faculty are simply amazing,” Brown continues. “Although, now more than 50 years old, HST remains at the vanguard for training PhD and MD students to work at the intersection between engineering, science, and medicine.”

      Goessling also thanks Brown for his leadership: “I truly valued Emery’s partnership and friendship, working together to deepen ties between the MIT and Harvard sides of HST. I am particularly grateful for working with Emery on our combined diversity efforts, leading to the HST Diversity Ambassadors initiative that made HST a better and stronger program.”

      According to Edelman, Brown was instrumental in the transition to new paradigms and relationships with HMS in the context of IMES. In 2014, he led the establishment of clear criteria for HST faculty membership, thereby strengthening the community of faculty experts who train students and provide research opportunities. More recently, he provided guidance through the turmoil of the ongoing Covid-19 pandemic, including the transition to online instruction and the return to the classroom. And Brown has always been a strong supporter of student diversity efforts, serving as an advocate and advisor to HST students.

      Brown holds BA, MA, and PhD degrees from Harvard University, and an MD from Harvard Medical School. He has been recognized with many awards, including the 2020 Swartz Prize in Theoretical and Computational Neuroscience, the 2018 Dickson Prize in Science, and an NIH Director’s Pioneer Award. Brown also served on President Barack Obama’s BRAIN Initiative Working Group. Among his many accomplishments, he has been cited for developing neural signal processing algorithms to characterize how neural systems represent and transmit information, and for unlocking the neurophysiology of how anesthetics produce the states of general anesthesia.

      Edelman says the process is underway to name a successor to Brown as co-director of HST at MIT. More

    • 125 Shares129 Views

      in Data Management & Statistics

      3 Questions: What a single car can say about traffic

      by

      Vehicle traffic has long defied description. Once measured roughly through visual inspection and traffic cameras, new smartphone crowdsourcing tools are now quantifying traffic far more precisely. This popular method, however, also presents a problem: Accurate measurements require a lot of data and users.

      Meshkat Botshekan, an MIT PhD student in civil and environmental engineering and research assistant at the MIT Concrete Sustainability Hub, has sought to expand on crowdsourcing methods by looking into the physics of traffic. During his time as a doctoral candidate, he has helped develop Carbin, a smartphone-based roadway crowdsourcing tool created by MIT CSHub and the University of Massachusetts Dartmouth, and used its data to offer more insight into the physics of traffic — from the formation of traffic jams to the inference of traffic phase and driving behavior. Here, he explains how recent findings can allow smartphones to infer traffic properties from the measurements of a single vehicle.  

      Q: Numerous navigation apps already measure traffic. Why do we need alternatives?

      A: Traffic characteristics have always been tough to measure. In the past, visual inspection and cameras were used to produce traffic metrics. So, there’s no denying that today’s navigation tools apps offer a superior alternative. Yet even these modern tools have gaps.

      Chief among them is their dependence on spatially distributed user counts: Essentially, these apps tally up their users on road segments to estimate the density of traffic. While this approach may seem adequate, it is both vulnerable to manipulation, as demonstrated in some viral videos, and requires immense quantities of data for reliable estimates. Processing these data is so time- and resource-intensive that, despite their availability, they can’t be used to quantify traffic effectively across a whole road network. As a result, this immense quantity of traffic data isn’t actually optimal for traffic management.

      Q: How could new technologies improve how we measure traffic?

      A: New alternatives have the potential to offer two improvements over existing methods: First, they can extrapolate far more about traffic with far fewer data. Second, they can cost a fraction of the price while offering a far simpler method of data collection. Just like Waze and Google Maps, they rely on crowdsourcing data from users. Yet, they are grounded in the incorporation of high-level statistical physics into data analysis.

      For instance, the Carbin app, which we are developing in collaboration with UMass Dartmouth, applies principles of statistical physics to existing traffic models to entirely forgo the need for user counts. Instead, it can infer traffic density and driver behavior using the input of a smartphone mounted in single vehicle.

      The method at the heart of the app, which was published last fall in Physical Review E, treats vehicles like particles in a many-body system. Just as the behavior of a closed many-body system can be understood through observing the behavior of an individual particle relying on the ergodic theorem of statistical physics, we can characterize traffic through the fluctuations in speed and position of a single vehicle across a road. As a result, we can infer the behavior and density of traffic on a segment of a road.

      As far less data is required, this method is more rapid and makes data management more manageable. But most importantly, it also has the potential to make traffic data less expensive and accessible to those that need it.

      Q: Who are some of the parties that would benefit from new technologies?

      A: More accessible and sophisticated traffic data would benefit more than just drivers seeking smoother, faster routes. It would also enable state and city departments of transportation (DOTs) to make local and collective interventions that advance the critical transportation objectives of equity, safety, and sustainability.

      As a safety solution, new data collection technologies could pinpoint dangerous driving conditions on a much finer scale to inform improved traffic calming measures. And since socially vulnerable communities experience traffic violence disproportionately, these interventions would have the added benefit of addressing pressing equity concerns. 

      There would also be an environmental benefit. DOTs could mitigate vehicle emissions by identifying minute deviations in traffic flow. This would present them with more opportunities to mitigate the idling and congestion that generate excess fuel consumption.  

      As we’ve seen, these three challenges have become increasingly acute, especially in urban areas. Yet, the data needed to address them exists already — and is being gathered by smartphones and telematics devices all over the world. So, to ensure a safer, more sustainable road network, it will be crucial to incorporate these data collection methods into our decision-making. More

    • 88 Shares149 Views

      in Data Management & Statistics

      Probing how proteins pair up inside cells

      by

      Despite its minute size, a single cell contains billions of molecules that bustle around and bind to one another, carrying out vital functions. The human genome encodes about 20,000 proteins, most of which interact with partner proteins to mediate upwards of 400,000 distinct interactions. These partners don’t just latch onto one another haphazardly; they only bind to very specific companions that they must recognize inside the crowded cell. If they create the wrong pairings — or even the right pairings at the wrong place or wrong time — cancer or other diseases can ensue. Scientists are hard at work investigating these protein-protein relationships, in order to understand how they work, and potentially create drugs that disrupt or mimic them to treat disease.

      The average human protein is composed of approximately 400 building blocks called amino acids, which are strung together and folded into a complex 3D structure. Within this long string of building blocks, some proteins contain stretches of four to six amino acids called short linear motifs (SLiMs), which mediate protein-protein interactions. Despite their simplicity and small size, SLiMs and their binding partners facilitate key cellular processes. However, it’s been historically difficult to devise experiments to probe how SLiMs recognize their specific binding partners.

      To address this problem, a group led by Theresa Hwang PhD ’21 designed a screening method to understand how SLiMs selectively bind to certain proteins, and even distinguish between those with similar structures. Using the detailed information they gleaned from studying these interactions, the researchers created their own synthetic molecule capable of binding extremely tightly to a protein called ENAH, which is implicated in cancer metastasis. The team shared their findings in a pair of eLife studies, one published on Dec. 2, 2021, and the other published Jan. 25.

      “The ability to test hundreds of thousands of potential SLiMs for binding provides a powerful tool to explore why proteins prefer specific SLiM partners over others,” says Amy Keating, professor of biology and biological engineering and the senior author on both studies. “As we gain an understanding of the tricks that a protein uses to select its partners, we can apply these in protein design to make our own binders to modulate protein function for research or therapeutic purposes.”

      Most existing screens for SLiMs simply select for short, tight binders, while neglecting SLiMs that don’t grip their partner proteins quite as strongly. To survey SLiMs with a wide range of binding affinities, Keating, Hwang, and their colleagues developed their own screen called MassTitr.

      The researchers also suspected that the amino acids on either side of the SLiM’s core four-to-six amino acid sequence might play an underappreciated role in binding. To test their theory, they used MassTitr to screen the human proteome in longer chunks comprised of 36 amino acids, in order to see which “extended” SLiMs would associate with the protein ENAH.

      ENAH, sometimes referred to as Mena, helps cells to move. This ability to migrate is critical for healthy cells, but cancer cells can co-opt it to spread. Scientists have found that reducing the amount of ENAH decreases the cancer cell’s ability to invade other tissues — suggesting that formulating drugs to disrupt this protein and its interactions could treat cancer.

      Thanks to MassTitr, the team identified 33 SLiM-containing proteins that bound to ENAH — 19 of which are potentially novel binding partners. They also discovered three distinct patterns of amino acids flanking core SLiM sequences that helped the SLiMs bind even tighter to ENAH. Of these extended SLiMs, one found in a protein called PCARE bound to ENAH with the highest known affinity of any SLiM to date.

      Next, the researchers combined a computer program called dTERMen with X-ray crystallography in order understand how and why PCARE binds to ENAH over ENAH’s two nearly identical sister proteins (VASP and EVL). Hwang and her colleagues saw that the amino acids flanking PCARE’s core SliM caused ENAH to change shape slightly when the two made contact, allowing the binding sites to latch onto one another. VASP and EVL, by contrast, could not undergo this structural change, so the PCARE SliM did not bind to either of them as tightly.

      Inspired by this unique interaction, Hwang designed her own protein that bound to ENAH with unprecedented affinity and specificity. “It was exciting that we were able to come up with such a specific binder,” she says. “This work lays the foundation for designing synthetic molecules with the potential to disrupt protein-protein interactions that cause disease — or to help scientists learn more about ENAH and other SLiM-binding proteins.”  

      Ylva Ivarsson, a professor of biochemistry at Uppsala University who was not involved with the study, says that understanding how proteins find their binding partners is a question of fundamental importance to cell function and regulation. The two eLife studies, she explains, show that extended SLiMs play an underappreciated role in determining the affinity and specificity of these binding interactions.

      “The studies shed light on the idea that context matters, and provide a screening strategy for a variety of context-dependent binding interactions,” she says. “Hwang and co-authors have created valuable tools for dissecting the cellular function of proteins and their binding partners. Their approach could even inspire ENAH-specific inhibitors for therapeutic purposes.”

      Hwang’s biggest takeaway from the project is that things are not always as they seem: even short, simple protein segments can play complex roles in the cell. As she puts it: “We should really appreciate SLiMs more.” More

    • 100 Shares149 Views

      in Data Management & Statistics

      The downside of machine learning in health care

      by

      While working toward her dissertation in computer science at MIT, Marzyeh Ghassemi wrote several papers on how machine-learning techniques from artificial intelligence could be applied to clinical data in order to predict patient outcomes. “It wasn’t until the end of my PhD work that one of my committee members asked: ‘Did you ever check to see how well your model worked across different groups of people?’”

      That question was eye-opening for Ghassemi, who had previously assessed the performance of models in aggregate, across all patients. Upon a closer look, she saw that models often worked differently — specifically worse — for populations including Black women, a revelation that took her by surprise. “I hadn’t made the connection beforehand that health disparities would translate directly to model disparities,” she says. “And given that I am a visible minority woman-identifying computer scientist at MIT, I am reasonably certain that many others weren’t aware of this either.”

      In a paper published Jan. 14 in the journal Patterns, Ghassemi — who earned her doctorate in 2017 and is now an assistant professor in the Department of Electrical Engineering and Computer Science and the MIT Institute for Medical Engineering and Science (IMES) — and her coauthor, Elaine Okanyene Nsoesie of Boston University, offer a cautionary note about the prospects for AI in medicine. “If used carefully, this technology could improve performance in health care and potentially reduce inequities,” Ghassemi says. “But if we’re not actually careful, technology could worsen care.”

      It all comes down to data, given that the AI tools in question train themselves by processing and analyzing vast quantities of data. But the data they are given are produced by humans, who are fallible and whose judgments may be clouded by the fact that they interact differently with patients depending on their age, gender, and race, without even knowing it.

      Furthermore, there is still great uncertainty about medical conditions themselves. “Doctors trained at the same medical school for 10 years can, and often do, disagree about a patient’s diagnosis,” Ghassemi says. That’s different from the applications where existing machine-learning algorithms excel — like object-recognition tasks — because practically everyone in the world will agree that a dog is, in fact, a dog.

      Machine-learning algorithms have also fared well in mastering games like chess and Go, where both the rules and the “win conditions” are clearly defined. Physicians, however, don’t always concur on the rules for treating patients, and even the win condition of being “healthy” is not widely agreed upon. “Doctors know what it means to be sick,” Ghassemi explains, “and we have the most data for people when they are sickest. But we don’t get much data from people when they are healthy because they’re less likely to see doctors then.”

      Even mechanical devices can contribute to flawed data and disparities in treatment. Pulse oximeters, for example, which have been calibrated predominately on light-skinned individuals, do not accurately measure blood oxygen levels for people with darker skin. And these deficiencies are most acute when oxygen levels are low — precisely when accurate readings are most urgent. Similarly, women face increased risks during “metal-on-metal” hip replacements, Ghassemi and Nsoesie write, “due in part to anatomic differences that aren’t taken into account in implant design.” Facts like these could be buried within the data fed to computer models whose output will be undermined as a result.

      Coming from computers, the product of machine-learning algorithms offers “the sheen of objectivity,” according to Ghassemi. But that can be deceptive and dangerous, because it’s harder to ferret out the faulty data supplied en masse to a computer than it is to discount the recommendations of a single possibly inept (and maybe even racist) doctor. “The problem is not machine learning itself,” she insists. “It’s people. Human caregivers generate bad data sometimes because they are not perfect.”

      Nevertheless, she still believes that machine learning can offer benefits in health care in terms of more efficient and fairer recommendations and practices. One key to realizing the promise of machine learning in health care is to improve the quality of data, which is no easy task. “Imagine if we could take data from doctors that have the best performance and share that with other doctors that have less training and experience,” Ghassemi says. “We really need to collect this data and audit it.”

      The challenge here is that the collection of data is not incentivized or rewarded, she notes. “It’s not easy to get a grant for that, or ask students to spend time on it. And data providers might say, ‘Why should I give my data out for free when I can sell it to a company for millions?’ But researchers should be able to access data without having to deal with questions like: ‘What paper will I get my name on in exchange for giving you access to data that sits at my institution?’

      “The only way to get better health care is to get better data,” Ghassemi says, “and the only way to get better data is to incentivize its release.”

      It’s not only a question of collecting data. There’s also the matter of who will collect it and vet it. Ghassemi recommends assembling diverse groups of researchers — clinicians, statisticians, medical ethicists, and computer scientists — to first gather diverse patient data and then “focus on developing fair and equitable improvements in health care that can be deployed in not just one advanced medical setting, but in a wide range of medical settings.”

      The objective of the Patterns paper is not to discourage technologists from bringing their expertise in machine learning to the medical world, she says. “They just need to be cognizant of the gaps that appear in treatment and other complexities that ought to be considered before giving their stamp of approval to a particular computer model.” More

    • 113 Shares99 Views

      in Data Management & Statistics

      Professors Elchanan Mossel and Rosalind Picard named 2021 ACM Fellows

      by

      The Association for Computing Machinery (ACM) has named MIT professors Elchanan Mossel and Rosalind Picard as fellows for outstanding accomplishments in computing and information technology.

      The ACM Fellows program recognizes wide-ranging and fundamental contributions in areas including algorithms, computer science education, cryptography, data security and privacy, medical informatics, and mobile and networked systems, among many other areas. The accomplishments of the 2021 ACM Fellows underpin important innovations that shape the technologies we use every day.

      Elchanan Mossel

      Mossel is a professor of mathematics and a member at the Statistics and Data Science Center of the MIT Institute for Data, Systems and Society. His research in discrete functional inequalities, isoperimetry, and hypercontractivity led to the proof that Majority is Stablest and confirmed the optimality of the Goemans-Williamson MAX-CUT algorithm under the unique games conjecture from computational complexity. His work on the reconstruction problem on trees provides optimal algorithms and bounds for phylogenetic reconstruction in molecular biology and has led to sharp results in the analysis of Gibbs samplers from statistical physics and inference problems on graphs. His research has resolved open problems in computational biology, machine learning, social choice theory, and economics.Mossel received a BS from the Open University in Israel in 1992, and MS (1997) and PhD (2000) degrees in mathematics from the Hebrew University of Jerusalem. He was a postdoc at the Microsoft Research Theory Group and a Miller Fellow at University of California at Berkeley. He joined the UC Berkeley faculty in 2003 as a professor of statistics and computer science, and spent leaves as a professor at the Weizmann Institute and at the Wharton School before joining MIT in 2016 as a full professor.

      In 2020, he received the Vannevar Bush Faculty Fellowship of the U.S. Department of Defense. Other distinctions include being named a Simons Investigator in Mathematics in 2019, being selected as a fellow of the AMS, and receiving a Sloan Research Fellowship, NSF CAREER Award, and the Bergmann Memorial Award from the U.S.-Israel Binational Science Foundation.

      “I am honored by this award,” says Mossel. “It makes me realize how fortunate I’ve been, working with creative and generous colleagues, and mentoring brilliant young minds.”

      Rosalind Picard

      Picard is a scientist, engineer, author, and professor of media arts and sciences at the MIT Media Lab. She is recognized as the founder of the field of affective computing, and has carried this research forward as head of the Media Lab’s Affective Computing research group. She is also a founding faculty chair of MIT’s MindHandHeart Initiative, and a faculty member of the MIT Center for Neurobiological Engineering. Picard is an IEEE fellow, and a member of the National Academy of Engineering. 

      Picard’s inventions are in use by thousands of research teams worldwide as well as in numerous products and services. She has co-founded two companies: Affectiva (now part of Smart Eye), providing emotion AI technologies now used by more than 25 percent of the Global Fortune 500, and Empatica, providing wearable sensors and analytics to improve health. Starting from inventions by Picard and her team, Empatica created the first AI-based smart watch cleared by the FDA (in neurology for monitoring seizures), which is now helping to bring potentially lifesaving help for people with epilepsy. 

      “This award makes me think of how blessed I am to work with so many amazing people here at MIT, especially at the Media Lab,” Picard notes. “Whenever any one of us has our contributions recognized, it is also a recognition of how special a place this is.” More

    • 100 Shares189 Views

      in Data Management & Statistics

      When should someone trust an AI assistant’s predictions?

      by

      In a busy hospital, a radiologist is using an artificial intelligence system to help her diagnose medical conditions based on patients’ X-ray images. Using the AI system can help her make faster diagnoses, but how does she know when to trust the AI’s predictions?

      She doesn’t. Instead, she may rely on her expertise, a confidence level provided by the system itself, or an explanation of how the algorithm made its prediction — which may look convincing but still be wrong — to make an estimation.

      To help people better understand when to trust an AI “teammate,” MIT researchers created an onboarding technique that guides humans to develop a more accurate understanding of those situations in which a machine makes correct predictions and those in which it makes incorrect predictions.

      By showing people how the AI complements their abilities, the training technique could help humans make better decisions or come to conclusions faster when working with AI agents.

      “We propose a teaching phase where we gradually introduce the human to this AI model so they can, for themselves, see its weaknesses and strengths,” says Hussein Mozannar, a graduate student in the Social and Engineering Systems doctoral program within the Institute for Data, Systems, and Society (IDSS) who is also a researcher with the Clinical Machine Learning Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Institute for Medical Engineering and Science. “We do this by mimicking the way the human will interact with the AI in practice, but we intervene to give them feedback to help them understand each interaction they are making with the AI.”

      Mozannar wrote the paper with Arvind Satyanarayan, an assistant professor of computer science who leads the Visualization Group in CSAIL; and senior author David Sontag, an associate professor of electrical engineering and computer science at MIT and leader of the Clinical Machine Learning Group. The research will be presented at the Association for the Advancement of Artificial Intelligence in February.

      Mental models

      This work focuses on the mental models humans build about others. If the radiologist is not sure about a case, she may ask a colleague who is an expert in a certain area. From past experience and her knowledge of this colleague, she has a mental model of his strengths and weaknesses that she uses to assess his advice.

      Humans build the same kinds of mental models when they interact with AI agents, so it is important those models are accurate, Mozannar says. Cognitive science suggests that humans make decisions for complex tasks by remembering past interactions and experiences. So, the researchers designed an onboarding process that provides representative examples of the human and AI working together, which serve as reference points the human can draw on in the future. They began by creating an algorithm that can identify examples that will best teach the human about the AI.

      “We first learn a human expert’s biases and strengths, using observations of their past decisions unguided by AI,” Mozannar says. “We combine our knowledge about the human with what we know about the AI to see where it will be helpful for the human to rely on the AI. Then we obtain cases where we know the human should rely on the AI and similar cases where the human should not rely on the AI.”

      The researchers tested their onboarding technique on a passage-based question answering task: The user receives a written passage and a question whose answer is contained in the passage. The user then has to answer the question and can click a button to “let the AI answer.” The user can’t see the AI answer in advance, however, requiring them to rely on their mental model of the AI. The onboarding process they developed begins by showing these examples to the user, who tries to make a prediction with the help of the AI system. The human may be right or wrong, and the AI may be right or wrong, but in either case, after solving the example, the user sees the correct answer and an explanation for why the AI chose its prediction. To help the user generalize from the example, two contrasting examples are shown that explain why the AI got it right or wrong.

      For instance, perhaps the training question asks which of two plants is native to more continents, based on a convoluted paragraph from a botany textbook. The human can answer on her own or let the AI system answer. Then, she sees two follow-up examples that help her get a better sense of the AI’s abilities. Perhaps the AI is wrong on a follow-up question about fruits but right on a question about geology. In each example, the words the system used to make its prediction are highlighted. Seeing the highlighted words helps the human understand the limits of the AI agent, explains Mozannar.

      To help the user retain what they have learned, the user then writes down the rule she infers from this teaching example, such as “This AI is not good at predicting flowers.” She can then refer to these rules later when working with the agent in practice. These rules also constitute a formalization of the user’s mental model of the AI.

      The impact of teaching

      The researchers tested this teaching technique with three groups of participants. One group went through the entire onboarding technique, another group did not receive the follow-up comparison examples, and the baseline group didn’t receive any teaching but could see the AI’s answer in advance.

      “The participants who received teaching did just as well as the participants who didn’t receive teaching but could see the AI’s answer. So, the conclusion there is they are able to simulate the AI’s answer as well as if they had seen it,” Mozannar says.

      The researchers dug deeper into the data to see the rules individual participants wrote. They found that almost 50 percent of the people who received training wrote accurate lessons of the AI’s abilities. Those who had accurate lessons were right on 63 percent of the examples, whereas those who didn’t have accurate lessons were right on 54 percent. And those who didn’t receive teaching but could see the AI answers were right on 57 percent of the questions.

      “When teaching is successful, it has a significant impact. That is the takeaway here. When we are able to teach participants effectively, they are able to do better than if you actually gave them the answer,” he says.

      But the results also show there is still a gap. Only 50 percent of those who were trained built accurate mental models of the AI, and even those who did were only right 63 percent of the time. Even though they learned accurate lessons, they didn’t always follow their own rules, Mozannar says.

      That is one question that leaves the researchers scratching their heads — even if people know the AI should be right, why won’t they listen to their own mental model? They want to explore this question in the future, as well as refine the onboarding process to reduce the amount of time it takes. They are also interested in running user studies with more complex AI models, particularly in health care settings.

      “When humans collaborate with other humans, we rely heavily on knowing what our collaborators’ strengths and weaknesses are — it helps us know when (and when not) to lean on the other person for assistance. I’m glad to see this research applying that principle to humans and AI,” says Carrie Cai, a staff research scientist in the People + AI Research and Responsible AI groups at Google, who was not involved with this research. “Teaching users about an AI’s strengths and weaknesses is essential to producing positive human-AI joint outcomes.” 

      This research was supported, in part, by the National Science Foundation. More