Artificial Intelligence

Scatterplots. You might not know them by name, but if you spend more than 10 minutes online you’ll find them everywhere. They’re popular in news articles, in the data science community, and, perhaps most crucially, for internet memes about the digestive quality of pancakes. 
By depicting data as a mass of points across two axes, scatterplots are effective in visualizing trends, correlations, and anomalies. But using them for large datasets often leads to overlapping dots that make them more or less unreadable.
Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) say they’ve solved this with a new open-source system that makes it possible to create interactive scatterplots based on large-scale datasets that have upwards of billions of distinct data points. 

Play video

Called “Kyrix-S,” the system has an interface that allows users to pan, zoom, and jump around a scatterplot as if they were looking at directions on Google Maps. Whereas other systems developed for large datasets often focus on very specific applications, Kyrix-S is generalizable enough to work for a wide range of visualization styles, including heat maps, pie charts, and radar-style graphics. (The team showed that the system allows users to create visualizations with 800 percent less code compared to a similar state-of-the-art authoring system.)
Users can produce a scatterplot by just writing a few dozen lines of JSON, a human-readable text format.
Lead developer Wenbo Tao, a PhD student at MIT CSAIL, gives the example of a static New York Times scatterplot (below) that he says would improve by being made interactive via a system like Kyrix-S.

A static New York Times scatterplot

Previous item Next item

“In these scatterplots, you are able to see overall trends and outliers, but the overplotting and the static nature of the plot limit the user’s ability to interact with the chart,” says Tao. 
In contrast, Kyrix-S can produce a version (below) that puts data in several zoom levels, enabling interaction with each county. To avoid overplotting, Kyrix-S’ scatterplot also shows only the most important examples, like the most populous counties.
Kyrix-S is currently being used by Data Civilizer 2.0, a data integration platform developed at MIT. An earlier version was also employed to help Massachusetts General Hospital analyze a massive brain activity dataset (EEG) that clocks in at 30 terabytes — the equivalent of more than 50,000 hours of digital music. (The goal of that study was to train a model that predicts seizures, given a series of 2-second EEG segments.)
Moving forward, the researchers will be adapting Kyrix-S to work as part of a graphical user interface. They also plan to add functionality so that the system can handle data that is being continuously updated.
Tao wrote a paper about Kyrix-S alongside MIT Adjunct Professor Mike Stonebraker, researchers Xinli Hou and Adam Sah, Leilani Battle SM ’13, PhD ’17, and Professor Remco Chang of Tufts University. It will be presented virtually at IEEE’s VIS data visualization conference Oct. 25. More

Imagine that one day you’re riding the train and decide to hop the turnstile to avoid paying the fare. It probably won’t have a big impact on the financial well-being of your local transportation system. But now ask yourself, “What if everyone did that?” The outcome is much different — the system would likely go bankrupt and no one would be able to ride the train anymore.
Moral philosophers have long believed this type of reasoning, known as universalization, is the best way to make moral decisions. But do ordinary people spontaneously use this kind of moral judgment in their everyday lives?
In a study of several hundred people, MIT and Harvard University researchers have confirmed that people do use this strategy in particular situations called “threshold problems.” These are social dilemmas in which harm can occur if everyone, or a large number of people, performs a certain action. The authors devised a mathematical model that quantitatively predicts the judgments they are likely to make. They also showed, for the first time, that children as young as 4 years old can use this type of reasoning to judge right and wrong.
“This mechanism seems to be a way that we spontaneously can figure out what are the kinds of actions that I can do that are sustainable in my community,” says Sydney Levine, a postdoc at MIT and Harvard and the lead author of the study.
Other authors of the study are Max Kleiman-Weiner, a postdoc at MIT and Harvard; Laura Schulz, an MIT professor of cognitive science; Joshua Tenenbaum, a professor of computational cognitive science at MIT and a member of MIT’s Center for Brains, Minds, and Machines and Computer Science and Artificial Intelligence Laboratory (CSAIL); and Fiery Cushman, an assistant professor of psychology at Harvard. The paper is appearing this week in the Proceedings of the National Academy of Sciences.
Judging morality
The concept of universalization has been included in philosophical theories since at least the 1700s. Universalization is one of several strategies that philosophers believe people use to make moral judgments, along with outcome-based reasoning and rule-based reasoning. However, there have been few psychological studies of universalization, and many questions remain regarding how often this strategy is used, and under what circumstances.
To explore those questions, the MIT/Harvard team asked participants in their study to evaluate the morality of actions taken in situations where harm could occur if too many people perform the action. In one hypothetical scenario, John, a fisherman, is trying to decide whether to start using a new, more efficient fishing hook that will allow him to catch more fish. However, if every fisherman in his village decided to use the new hook, there would soon be no fish left in the lake.
The researchers found that many subjects did use universalization to evaluate John’s actions, and that their judgments depended on a variety of factors, include the number of people who were interested in using the new hook and the number of people using it that would trigger a harmful outcome.
To tease out the impact of those factors, the researchers created several versions of the scenario. In one, no one else in the village was interested in using the new hook, and in that scenario, most participants deemed it acceptable for John to use it. However, if others in the village were interested but chose not to use it, then John’s decision to use it was judged to be morally wrong.
The researchers also found that they could use their data to create a mathematical model that explains how people take different factors into account, such as the number of people who want to do the action and the number of people doing it that would cause harm. The model accurately predicts how people’s judgments change when these factors change.
In their last set of studies, the researchers created scenarios that they used to test judgments made by children between the ages of 4 and 11. One story featured a child who wanted to take a rock from a path in a park for his rock collection. Children were asked to judge if that was OK, under two different circumstances: In one, only one child wanted a rock, and in the other, many other children also wanted to take rocks for their collections.
The researchers found that most of the children deemed it wrong to take a rock if everyone wanted to, but permissible if there was only one child who wanted to do it. However, the children were not able to specifically explain why they had made those judgments.
“What’s interesting about this is we discovered that if you set up this carefully controlled contrast, the kids seem to be using this computation, even though they can’t articulate it,” Levine says. “They can’t introspect on their cognition and know what they’re doing and why, but they seem to be deploying the mechanism anyway.”
In future studies, the researchers hope to explore how and when the ability to use this type of reasoning develops in children.
Collective action
In the real world, there are many instances where universalization could be a good strategy for making decisions, but it’s not necessary because rules are already in place governing those situations.
“There are a lot of collective action problems in our world that can be solved with universalization, but they’re already solved with governmental regulation,” Levine says. “We don’t rely on people to have to do that kind of reasoning, we just make it illegal to ride the bus without paying.”
However, universalization can still be useful in situations that arise suddenly, before any government regulations or guidelines have been put in place. For example, at the beginning of the Covid-19 pandemic, before many local governments began requiring masks in public places, people contemplating wearing masks might have asked themselves what would happen if everyone decided not to wear one.
The researchers now hope to explore the reasons why people sometimes don’t seem to use universalization in cases where it could be applicable, such as combating climate change. One possible explanation is that people don’t have enough information about the potential harm that can result from certain actions, Levine says.
The research was funded by the John Templeton Foundation, the Templeton World Charity Foundation, and the Center for Brains, Minds, and Machines. More

Researchers from Critical Analytics for Manufacturing Personalized-Medicine (CAMP), an interdisciplinary research group at Singapore-MIT Alliance for Research and Technology (SMART), MIT’s research enterprise in Singapore, have been awarded Intra-CREATE grants from the National Research Foundation (NRF) Singapore to help support research on retinal biometrics for glaucoma progression and neural cell implantation therapy for spinal cord injuries. The grants are part of the NRF’s initiative to bring together researchers from Campus for Research Excellence And Technological Enterprise (CREATE) partner institutions, in order to achieve greater impact from collaborative research efforts.
SMART CAMP was formed in 2019 to focus on ways to produce living cells as medicine delivered to humans to treat a range of illnesses and medical conditions, including tissue degenerative diseases, cancer, and autoimmune disorders.
“Singapore’s well-established biopharmaceutical ecosystem brings with it a thriving research ecosystem that is supported by skilled talents and strong manufacturing capabilities. We are excited to collaborate with our partners in Singapore, bringing together an interdisciplinary group of experts from MIT and Singapore, for new research areas at SMART. In addition to our existing research on our three flagship projects, we hope to develop breakthroughs in manufacturing other cell therapy platforms that will enable better medical treatments and outcomes for society,” says Krystyn Van Vliet, co-lead principal investigator at SMART CAMP, professor of materials science and engineering, and associate provost at MIT.
Understanding glaucoma progression for better-targeted treatments
Hosted by SMART CAMP, the first research project, Retinal Analytics via Machine learning aiding Physics (RAMP), brings together an interdisciplinary group of ophthalmologists, data scientists, and optical scientists from SMART, Singapore Eye Research Institute (SERI), Agency for Science, Technology and Research (A*STAR), Duke-NUS Medical School, MIT, and National University of Singapore (NUS). The team will seek to establish first principles-founded and statistically confident models of glaucoma progression in patients. Through retinal biomechanics, the models will enable rapid and reliable forecast of the rate and trajectory of glaucoma progression, leading to better-targeted treatments.
Glaucoma, an eye condition often caused by stress-induced damage over time at the optic nerve head, accounts for 5.1 million of the estimated 38 million blind in the world and 40 percent of blindness in Singapore. Currently, health practitioners face challenges forecasting glaucoma progression and its treatment strategies due to the lack of research and technology that accurately establish the relationship between its properties, such as the elasticity of the retina and optic nerve heads, blood flow, intraocular pressure and, ultimately, damage to the optic nerve head.
The research is co-led by George Barbastathis, principal investigator at SMART CAMP and professor of mechanical engineering at MIT, and Aung Tin, executive director at SERI and professor at the Department of Ophthalmology at NUS. The team includes CAMP principal investigators Nicholas Fang, also a professor of mechanical engineering at MIT; Lisa Tucker-Kellogg, assistant professor with the Cancer and Stem Biology program at Duke-NUS; and Hanry Yu, professor of physiology with the Yong Loo Lin School of Medicine, NUS and CAMP’s co-lead principal investigator.
“We look forward to leveraging the ideas fostered in SMART CAMP to build data analytics and optical imaging capabilities for this pressing medical challenge of glaucoma prediction,” says Barbastathis.
Cell transplantation to treat irreparable spinal cord injury
Engineering Scaffold-Mediated Neural Cell Therapy for Spinal Cord Injury Treatment (ScaNCellS), the second research project, gathers an interdisciplinary group of engineers, cell biologists, and clinician scientists from SMART, Nanyang Technological University (NTU), NUS, IMCB A*STAR, A*STAR, French National Centre for Scientific Research (CNRS), the University of Cambridge, and MIT. The team will seek to design a combined scaffold and neural cell implantation therapy for spinal cord injury treatment that is safe, efficacious, and reproducible, paving the way forward for similar neural cell therapies for other neurological disorders. The project, an intersection of engineering and health, will achieve its goals through an enhanced biological understanding of the regeneration process of nerve tissue and optimized engineering methods to prepare cells and biomaterials for treatment.
Spinal cord injury (SCI), affecting between 250,00 and 500,000 people yearly, is expected to incur higher societal costs as compared to other common conditions such as dementia, multiple sclerosis, and cerebral palsy. SCI can lead to temporary or permanent changes in spinal cord function, including numbness or paralysis. Currently, even with the best possible treatment, the injury generally results in some incurable impairment.
The research is co-led by Chew Sing Yian, principal investigator at SMART CAMP and associate professor of the School of Chemical and Biomedical Engineering and Lee Kong Chian School of Medicine at NTU, and Laurent David, professor at University of Lyon (France) and leader of the Polymers for Life Sciences group at CNRS Polymer Engineering Laboratory. The team includes CAMP principal investigators Ai Ye from Singapore University of Technology and Design; Jongyoon Han and Zhao Xuanhe, both professors at MIT; as well as Shi-Yan Ng and Jonathan Loh from Institute of Molecular and Cell Biology, A*STAR.
Chew says, “Our earlier SMART and NTU scientific collaborations on progenitor cells in the central nervous system are now being extended to cell therapy translation. This helps us address SCI in a new way, and connect to the methods of quality analysis for cells developed in SMART CAMP.”
“Cell therapy, one of the fastest-growing areas of research, will provide patients with access to more options that will prevent and treat illnesses, some of which are currently incurable. Glaucoma and spinal cord injuries affect many. Our research will seek to plug current gaps and deliver valuable impact to cell therapy research and medical treatments for both conditions. With a good foundation to work on, we will be able to pave the way for future exciting research for further breakthroughs that will benefit the health-care industry and society,” says Hanry Yu, co-lead principal investigator at SMART CAMP, professor of physiology with the Yong Loo Lin School of Medicine, NUS, and group leader of the Institute of Bioengineering and Nanotechnology at A*STAR.
The grants for both projects will commence on  Oct. 1, with RAMP expected to run until Sept. 30, 2022, and ScaNCellS expected to run until Sept. 30, 2023.
SMART was. established by the MIT in partnership with the NRF in 2007. SMART is the first entity in the CREATE developed by NRF. SMART serves as an intellectual and innovation hub for research interactions between MIT and Singapore, undertaking cutting-edge research projects in areas of interest to both Singapore and MIT. SMART currently comprises an Innovation Centre and five interdisciplinary research groups (IRGs): Antimicrobial Resistance, CAMP, Disruptive and Sustainable Technologies for Agricultural Precision, Future Urban Mobility, and Low Energy Electronic Systems.
CAMP is a SMART IRG launched in June 2019. It focuses on better ways to produce living cells as medicine, or cellular therapies, to provide more patients access to promising and approved therapies. The investigators at CAMP address two key bottlenecks facing the production of a range of potential cell therapies: critical quality attributes (CQA) and process analytic technologies (PAT). Leveraging deep collaborations within Singapore and MIT in the United States, CAMP invents and demonstrates CQA/PAT capabilities from stem to immune cells. Its work addresses ailments ranging from cancer to tissue degeneration, targeting adherent and suspended cells, with and without genetic engineering.
CAMP is the R&D core of a comprehensive national effort on cell therapy manufacturing in Singapore. More

Every year, roughly one out of eight U.S. deaths is caused at least in part by heart failure. One of acute heart failure’s most common warning signs is excess fluid in the lungs, a condition known as “pulmonary edema.” 
A patient’s exact level of excess fluid often dictates the doctor’s course of action, but making such determinations is difficult and requires clinicians to rely on subtle features in X-rays that sometimes lead to inconsistent diagnoses and treatment plans.
To better handle that kind of nuance, a group led by researchers at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) has developed a machine learning model that can look at an X-ray to quantify how severe the edema is, on a four-level scale ranging from 0 (healthy) to 3 (very, very bad). The system determined the right level more than half of the time, and correctly diagnosed level 3 cases 90 percent of the time.
Working with Beth Israel Deaconess Medical Center (BIDMC) and Philips, the team plans to integrate the model into BIDMC’s emergency-room workflow this fall.
“This project is meant to augment doctors’ workflow by providing additional information that can be used to inform their diagnoses as well as enable retrospective analyses,” says PhD student Ruizhi Liao, who was the co-lead author of a related paper with fellow PhD student Geeticka Chauhan and MIT professors Polina Golland and Peter Szolovits. 
The team says that better edema diagnosis would help doctors manage not only acute heart issues, but other conditions like sepsis and kidney failure that are strongly associated with edema. 
As part of a separate journal article, Liao and colleagues also took an existing public dataset of X-ray images and developed new annotations of severity labels that were agreed upon by a team of four radiologists. Liao’s hope is that these consensus labels can serve as a universal standard to benchmark future machine learning development.
An important aspect of the system is that it was trained not just on more than 300,000 X-ray images, but also on the corresponding text of reports about the X-rays that were written by radiologists. The team was pleasantly surprised that their system found such success using these reports, most of which didn’t have labels explaining the exact severity level of the edema.
“By learning the association between images and their corresponding reports, the method has the potential for a new way of automatic report generation from the detection of image-driven findings,” says Tanveer Syeda-Mahmood, a researcher not involved in the project who serves as chief scientist for IBM’s Medical Sieve Radiology Grand Challenge. “Of course, further experiments would have to be done for this to be broadly applicable to other findings and their fine-grained descriptors.”
Chauhan’s efforts focused on helping the system make sense of the text of the reports, which could often be as short as a sentence or two. Different radiologists write with varying tones and use a range of terminology, so the researchers had to develop a set of linguistic rules and substitutions to ensure that data could be analyzed consistently across reports. This was in addition to the technical challenge of designing a model that can jointly train the image and text representations in a meaningful manner.
“Our model can turn both images and text into compact numerical abstractions from which an interpretation can be derived,” says Chauhan. “We trained it to minimize the difference between the representations of the X-ray images and the text of the radiology reports, using the reports to improve the image interpretation.”
On top of that, the team’s system was also able to “explain” itself, by showing which parts of the reports and areas of X-ray images correspond to the model prediction. Chauhan is hopeful that future work in this area will provide more detailed lower-level image-text correlations, so that clinicians can build a taxonomy of images, reports, disease labels and relevant correlated regions. 
“These correlations will be valuable for improving search through a large database of X-ray images and reports, to make retrospective analysis even more effective,” Chauhan says.
Chauhan, Golland, Liao and Szolovits co-wrote the paper with MIT Assistant Professor Jacob Andreas, Professor William Wells of Brigham and Women’s Hospital, Xin Wang of Philips, and Seth Berkowitz and Steven Horng of BIDMC. The paper will be presented Oct. 5 (virtually) at the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). 
The work was supported in part by the MIT Deshpande Center for Technological Innovation, the MIT Lincoln Lab, the National Institutes of Health, Philips, Takeda, and the Wistron Corporation. More

Milo Phillips-Brown, a postdoc in MIT Philosophy, was recently named the inaugural recipient of the MAC3 Society and Ethics in Computing Research Award, which provides support to promising PhD candidates or postdocs conducting interdisciplinary research on the societal and ethical dimensions of computing.
Phillips-Brown, the Distinguished Postdoctoral Scholar in Ethics and Technology within the MIT Stephen A. Schwarzman College of Computing — a position that is supported, in part, by the MIT Quest for Intelligence — is being recognized for his work teaching responsible engineering practices to computer scientists. He teaches two courses, 24.131 (Ethics of Technology) and 24.133 (Experiential Ethics), and has been an active participant in the activities of the Social and Ethical Responsibilities of Computing (SERC), a new cross-cutting area in the MIT Stephen A. Schwarzman College of Computing that aims to actively weave social, ethical, and policy considerations into the teaching, research, and implementation of computing.
“We are delighted to be able to work so closely with Milo,” says Julie Shah, an associate professor in the Department of Aeronautics and Astronautics, who along with David Kaiser, the Germeshausen Professor of the History of Science and professor of physics, serves as associate dean of SERC. “Over this past spring semester, Milo was a great thought partner in the design of SERC-related materials, including original homework assignments and in-class demonstrations for instructors to embed into a wide variety of courses at MIT,” says Shah.
“We knew we had an exceptional colleague when we selected Milo as our inaugural postdoc. We look forward to collaborating with him and his continued contributions to SERC,” adds Kaiser.
In addition to active learning projects, Phillips-Brown has been working with Shah and Kaiser on preparing the first set of original case studies on social and ethical responsibilities of computing for release in the coming months. Commissioned and curated by SERC, each case study will be brief and appropriate for use in undergraduate instruction and will also be available to the public via MIT’s open access channels.
“I’m thrilled to be the inaugural recipient of the MAC3 Society and Ethics in Computing Research Award. This is a time when we need to be exploring all possible avenues for how to teach MIT students to build technologies ethically, and the award is enabling me to help just do that: work with professors and students across the Institute to develop new models for ethical engineering pedagogy,” says Phillips-Brown.
Phillips-Brown PhD ’19 received his doctorate in philosophy from MIT and his bachelor’s in philosophy from Reed College. He is a research fellow in digital ethics and governance at the Jain Family Institute and a member of the Society for Philosophy and Disability. From 2015 to 2018, he directed the Philosophy in an Inclusive Key (PIKSI) Boston, a summer program for undergraduates from underrepresented groups. In January 2021, he will begin an appointment at Oxford University as an associate professor of philosophy in the Faculty of Philosophy and the Department of Computer Science.
The MAC3 Society and Ethics in Computing Research Award was established through the MAC3 Impact Philanthropies which provides targeted support to organizations and initiatives that impact early childhood, health and education, as well as the environment and the oceans. More

The Standard Model of particle physics describes all the known elementary particles and three of the four fundamental forces governing the universe; everything except gravity. These three forces — electromagnetic, strong, and weak — govern how particles are formed, how they interact, and how the particles decay.
Studying particle and nuclear physics within this framework, however, is difficult, and relies on large-scale numerical studies. For example, many aspects of the strong force require numerically simulating the dynamics at the scale of 1/10th to 1/100th the size of a proton to answer fundamental questions about the properties of protons, neutrons, and nuclei.
“Ultimately, we are computationally limited in the study of proton and nuclear structure using lattice field theory,” says assistant professor of physics Phiala Shanahan. “There are a lot of interesting problems that we know how to address in principle, but we just don’t have enough compute, even though we run on the largest supercomputers in the world.”
To push past these limitations, Shanahan leads a group that combines theoretical physics with machine learning models. In their paper “Equivariant flow-based sampling for lattice gauge theory,” published this month in Physical Review Letters, they show how incorporating the symmetries of physics theories into machine learning and artificial intelligence architectures can provide much faster algorithms for theoretical physics. 
“We are using machine learning not to analyze large amounts of data, but to accelerate first-principles theory in a way which doesn’t compromise the rigor of the approach,” Shanahan says. “This particular work demonstrated that we can build machine learning architectures with some of the symmetries of the Standard Model of particle and nuclear physics built in, and accelerate the sampling problem we are targeting by orders of magnitude.” 
Shanahan launched the project with MIT graduate student Gurtej Kanwar and with Michael Albergo, who is now at NYU. The project expanded to include Center for Theoretical Physics postdocs Daniel Hackett and Denis Boyda, NYU Professor Kyle Cranmer, and physics-savvy machine-learning scientists at Google Deep Mind, Sébastien Racanière and Danilo Jimenez Rezende.
This month’s paper is one in a series aimed at enabling studies in theoretical physics that are currently computationally intractable. “Our aim is to develop new algorithms for a key component of numerical calculations in theoretical physics,” says Kanwar. “These calculations inform us about the inner workings of the Standard Model of particle physics, our most fundamental theory of matter. Such calculations are of vital importance to compare against results from particle physics experiments, such as the Large Hadron Collider at CERN, both to constrain the model more precisely and to discover where the model breaks down and must be extended to something even more fundamental.”
The only known systematically controllable method of studying the Standard Model of particle physics in the nonperturbative regime is based on a sampling of snapshots of quantum fluctuations in the vacuum. By measuring properties of these fluctuations, once can infer properties of the particles and collisions of interest.
This technique comes with challenges, Kanwar explains. “This sampling is expensive, and we are looking to use physics-inspired machine learning techniques to draw samples far more efficiently,” he says. “Machine learning has already made great strides on generating images, including, for example, recent work by NVIDIA to generate images of faces ‘dreamed up’ by neural networks. Thinking of these snapshots of the vacuum as images, we think it’s quite natural to turn to similar methods for our problem.”
Adds Shanahan, “In our approach to sampling these quantum snapshots, we optimize a model that takes us from a space that is easy to sample to the target space: given a trained model, sampling is then efficient since you just need to take independent samples in the easy-to-sample space, and transform them via the learned model.”
In particular, the group has introduced a framework for building machine-learning models that exactly respect a class of symmetries, called “gauge symmetries,” crucial for studying high-energy physics.
As a proof of principle, Shanahan and colleagues used their framework to train machine-learning models to simulate a theory in two dimensions, resulting in orders-of-magnitude efficiency gains over state-of-the-art techniques and more precise predictions from the theory. This paves the way for significantly accelerated research into the fundamental forces of nature using physics-informed machine learning.
The group’s first few papers as a collaboration discussed applying the machine-learning technique to a simple lattice field theory, and developed this class of approaches on compact, connected manifolds which describe the more complicated field theories of the Standard Model. Now they are working to scale the techniques to state-of-the-art calculations.
“I think we have shown over the past year that there is a lot of promise in combining physics knowledge with machine learning techniques,” says Kanwar. “We are actively thinking about how to tackle the remaining barriers in the way of performing full-scale simulations using our approach. I hope to see the first application of these methods to calculations at scale in the next couple of years. If we are able to overcome the last few obstacles, this promises to extend what we can do with limited resources, and I dream of performing calculations soon that give us novel insights into what lies beyond our best understanding of physics today.”This idea of physics-informed machine learning is also known by the team as “ab-initio AI,” a key theme of the recently launched MIT-based National Science Foundation Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), where Shanahan is research coordinator for physics theory.
Led by the Laboratory for Nuclear Science, the IAIFI is comprised of both physics and AI researchers at MIT and Harvard, Northeastern, and Tufts universities.
“Our collaboration is a great example of the spirit of IAIFI, with a team with diverse backgrounds coming together to advance AI and physics simultaneously” says Shanahan. As well as research like Shanahan’s targeting physics theory, IAIFI researchers are also working to use AI to enhance the scientific potential of various facilities, including the Large Hadron Collider and the Laser Interferometer Gravity Wave Observatory, and to advance AI itself.  More

Even in ordinary times, scientific process is stressful, with its demand for open-ended exploration and persistence in the face of failure. But the pandemic has added to the strain. In this new world of physical isolation, there are fewer opportunities for spontaneity and connection, and fewer distractions and events to mark the passage of time. Days pass in a numbing blur of sameness.
Working from home this summer, students participating in MIT’s Undergraduate Research Opportunities Program (UROP) did their best to overcome these challenges. Checking in with their advisors over Zoom and Slack, from as far west as Los Angeles, California and as far east as Skopje, North Macedonia, they completed two dozen projects sponsored by the MIT Quest for Intelligence. Four student projects are highlighted here.
Defending code-processing AI models against adversarial attacks 
Computer vision models have famously been fooled into classifying turtles as rifles, and planes as pigs, simply by making subtle changes to the objects and images the models are asked to interpret. But models that analyze computer code, which are a part of recent efforts to build automated tools to design programs efficiently, are also susceptible to so-called adversarial examples. 
The lab of Una-May O’Reilly, a principal research scientist at MIT, is focused on finding and fixing the weaknesses in code-processing models that can cause them to misbehave. As automated programming methods become more common, researchers are looking for ways to make this class of deep learning model more secure.
“Even small changes like giving a different name to a variable in a computer program can completely change how the model interprets the program,” says Tamara Mitrovska, a third-year student who worked on a UROP project this summer with Shashank Srikant, a graduate student in O’Reilly’s lab.
The lab is investigating two types of models used to summarize bits of a program as part of a broader effort to use machine learning to write new programs. One such model is Google’s seq2seq, originally developed for machine translation. A second is code2seq, which creates abstract representations of programs. Both are vulnerable to attacks due to a simple programming quirk: captions that let humans know what the code is doing, like assigning names to variables, give attackers an opening to exploit the model. By simply changing a variable name in a program or adding a print statement, the program may function normally, yet force the model processing it to give an incorrect answer.
This summer, from her home near Skopje, in North Macedonia, Mitrovska learned how to sift through a database of more than 100,000 programs in Java and Python and modify them algorithmically to try to fool seq2seq and code2seq. “These systems are challenging to implement,” she says. “Finding even the smallest bug can take a significant amount of time. But overall, I’ve been having fun and the project has been a very good learning experience for me.”
One exploit that she uncovered: Both models could be tricked by inserting “print” commands in the programs they process. That exploit, and others discovered by the lab, will be used to update the models to make them more robust.
What everyday adjectives can tell us about human reasoning
Embedded in the simplest of words are assumptions about the world that vary even among closely related languages. Take the word “biggest.” Like other superlatives in English, this adjective has no equivalent in French or Spanish. Speakers simply use the comparative form, “bigger” — plus grand in French or más grande in Spanish — to differentiate among objects of various sizes.
To understand what these words mean and how they are actually used, Helena Aparicio, formerly a postdoc at MIT and now a professor at Cornell University, devised a set of psychology experiments with MIT Associate Professor Roger Levy and Boston University Professor Elizabeth Coppock. Curtis Chen, a second-year student at MIT interested in the four topics that converge in Levy’s lab — computer science, psychology, linguistics, and cognitive science — joined on as a UROP student.
From his home in Hillsborough, New Jersey, Chen orchestrated experiments to identify why English speakers prefer superlatives in some cases and comparatives in others. He found that in scenes with more similarly sized objects, the more likely his human subjects were to prefer the word “biggest” to describe the largest object in the set. When objects appeared to fall within two clearly defined groups, subjects preferred the less-precise “bigger.” Chen also built an AI model to simulate the inferences made by his human subjects and found that it showed a similar preference for the superlative in ambiguous situations.
Designing a successful experiment can take several tries. To ensure consistency among the shapes that subjects were asked to describe, Chen generated them on the computer using HTML Canvas and JavaScript. “This way, the size differentials were exact, and we could simply report the formula used to make them,” he says.
After discovering that some subjects seemed confused by rectangle and line shapes, he replaced them with circles. He also removed the default option on his reporting scale after realizing that some subjects were using it to breeze through the tasks. Finally, he switched to the crowdsourcing platform Prolific after a number of participants on Amazon’s Mechanical Turk failed at tasks designed to ensure they were taking the experiments seriously.
“It was discouraging, but Curtis went through the process of exploring the data and figuring out what was going wrong,” says his mentor, Aparicio. 
In the end, he wound up with strong results and promising ideas for follow-up experiments this fall. “There’s still a lot to be done,” he says. “I had a lot of fun cooking up and tweaking the model, designing the experiment, and learning about this deceptively simple puzzle.”
Levy says he looks forward to the results. “Ultimately, this line of inquiry helps us understand how different vocabularies and grammatical resources of English and thousands of other languages support flexible communication by their native speakers,” he says.
Reconstructing real-world scenes from sensor data
AI systems that have become expert at sizing up scenes in photos and video may soon be able to do the same for real-world scenes. It’s a process that involves stitching together snapshots of a scene from varying viewpoints into a coherent picture. The brain performs these calculations effortlessly as we move through the world, but computers require sophisticated algorithms and extensive training. 
MIT Associate Professor Justin Solomon focuses on developing methods to help computers understand 3D environments. He and his lab look for new ways to take point cloud data gathered by sensors — essentially, reflections of infrared light bounced off the surfaces of objects — to create a holistic representation of a real-world scene. Three-dimensional scene analysis has many applications in computer graphics, but the one that drove second-year student Kevin Shao to join Solomon’s lab was its potential as a navigation tool for self-driving cars.
“Working on autonomous cars has been a childhood dream for me,” says Shao.
In the first phase of his UROP project, Shao downloaded the most important papers on 3D scene reconstruction and tried to reproduce their results. This improved his knowledge of PyTorch, the Python library that provides tools for training, testing, and evaluating models. It also gave him a deep understanding of the literature. In the second phase of the project, Shao worked with his mentor, PhD student Yue Wang, to improve on existing methods.
“Kevin implemented most of the ideas, and explained in detail why they would or wouldn’t work,” says Wang. “He didn’t give up on an idea until we had a comprehensive analysis of the problem.”
One idea they explored was the use of computer-drawn scenes to train a multi-view registration model. So far, the method works in simulation, but not on real-world scenes. Shao is now trying to incorporate real-world data to bridge the gap, and will continue the work this fall.
Wang is excited to see the results. “It sometimes takes PhD students a year to have a reasonable result,” he says. “Although we are still in the exploration phase, I think Kevin has made a successful transition from a smart student to a well-qualified researcher.”
When do infants become attuned to speech and music?
The ability to perceive speech and music has been traced to specialized parts of the brain, with infants as young as four months old showing sensitivity to speech-like sounds. MIT Professor Nancy Kanwisher and her lab are investigating how this special ear for speech and music arises in the infant brain.
Somaia Saba, a second-year student at MIT, was introduced to Kanwisher’s research last year in an intro to neuroscience class and immediately wanted to learn more. “The more I read up about cortical development, the more I realized how little we know about the development of the visual and auditory pathways,” she says. “I became very excited and met with [PhD student] Heather Kosakowski, who explained the details of her projects.”
Signing on for a project, Saba plunged into the “deep end” of cortical development research. Initially overwhelmed, she says she gained confidence through regular Zoom meetings with Kosakowski, who helped her to navigate MATLAB and other software for analyzing brain-imaging data. “Heather really helped motivate me to learn these programs quickly, which has also primed me to learn more easily in the future,” she says.
Before the pandemic shut down campus, Kanwisher’s lab collected functional magnetic resonance imaging (fMRI) data from two- to eight-week-old sleeping infants exposed to different sounds. This summer, from her home on Long Island, New York, Saba helped to analyze the data. She is now learning how to process fMRI data for awake infants, looking toward the study’s next phase. “This is a crucial and very challenging task that’s harder than processing child and adult fMRI data,” says Kosakowski. “Discovering how these specialized regions emerge in infants may be the key to unlocking mysteries about the origin of the mind.”
MIT Quest for Intelligence summer UROP projects were funded, in part, by the MIT-IBM Watson AI Lab and by Eric Schmidt, technical advisor to Alphabet Inc., and his wife, Wendy. More

For more than 100 years Nobel Prizes have been given out annually to recognize breakthrough achievements in chemistry, literature, medicine, peace, and physics. As these disciplines undoubtedly continue to impact society, newer fields like artificial intelligence (AI) and robotics have also begun to profoundly reshape the world.
In recognition of this, the world’s largest AI society — the Association for the Advancement of Artificial Intelligence (AAAI) — announced today the winner of their new Squirrel AI Award for Artificial Intelligence for the Benefit of Humanity, a $1 million award given to honor individuals whose work in the field has had a transformative impact on society.
The recipient, Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science at MIT and a member of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), is being recognized for her work developing machine learning models to develop antibiotics and other drugs, and to detect and diagnose breast cancer at early stages.
In February, AAAI will officially present Barzilay with the award, which comes with an associated prize of $1 million provided by the online education company Squirrel AI. 
“Only world-renowned recognitions, such as the Association of Computing Machinery’s A.M. Turing Award and the Nobel Prize, carry monetary rewards at the million-dollar level,” says AAAI awards committee chair Yolanda Gil. “This award aims to be unique in recognizing the positive impact of artificial intelligence for humanity.” 
Barzilay has conducted research on a range of topics in computer science, ranging from explainable machine learning to deciphering dead languages. Since surviving breast cancer in 2014, she has increasingly focused her efforts on health care. She created algorithms for early breast cancer diagnosis and risk assessment that have been tested at multiple hospitals around the globe, including in Sweden, Taiwan, and at Boston’s Massachusetts General Hospital. She is now working with breast cancer organizations such as Institute Protea in Brazil to make her diagnostic tools available for underprivileged populations around the world. (She realized from doing her work that, if a system like hers had existed at the time, her doctors actually could have detected her cancer two or three years earlier.) 
In parallel, she has been working on developing machine learning models for drug discovery: with collaborators she’s created models for selecting molecule candidates for therapeutics that have been able to speed up drug development, and last year helped discover a new antibiotic called Halicin that was shown to be able to kill many species of disease-causing bacteria that are antibiotic-resistant, including Acinetobacter baumannii and clostridium difficile (“c-diff”). 
“Through my own life experience, I came to realize that we can create technology that can alleviate human suffering and change our understanding of diseases,“ says Barzilay, who is also a member of the Koch Institute for Integrative Cancer Research. “I feel lucky to have found collaborators who share my passion and who have helped me realize this vision.”
Barzilay also serves as a member of MIT’s Institute for Medical Engineering and Science, and as faculty co-lead for MIT’s Abdul Latif Jameel Clinic for Machine Learning in Health. One of the J-Clinic’s most recent efforts is “AI Cures,” a cross-institutional initiative focused on developing affordable Covid-19 antivirals. 
“Regina has made truly-changing breakthroughs in imaging breast cancer and predicting the medicinal activity of novel chemicals,” says MIT professor of biology Phillip Sharp, a Nobel laureate who has served as director of both the McGovern Institute for Brain Research and the MIT Center for Cancer Research, predecessor to the Koch Institute. “I am honored to have as a colleague someone who is such a pioneer in using deeply creative machine learning methods to transform the fields of health care and biological science.”
Barzilay joined the MIT faculty in 2003 after earning her undergraduate at Ben-Gurion University of the Negev, Israel and her PhD at Columbia University. She is also the recipient of a MacArthur “genius grant”, the National Science Foundation Career Award, a Microsoft Faculty Fellowship, multiple “best paper” awards in her field, and MIT’s Jamieson Award for excellence in teaching.

Play video

“We believe AI advances will benefit a great many fields, from health care and education to smart cities and the environment,” says Derek Li, founder and chairman of Squirrel AI. “We believe that Dr. Barzilay and other future awardees will inspire the AI community to continue to contribute to and advance AI’s impact on the world.”
AAAI’s Gil says the organization was very excited to partner with Squirrel AI for this new award to recognize the positive impacts of artificial intelligence “to protect, enhance, and improve human life in meaningful ways.” With more than 300 elected fellows and 6,000 members from 50 countries across the globe, AAAI is the world’s largest scientific society devoted to artificial intelligence. Its officers have included many AI pioneers, including Allen Newell and John McCarthy. AAAI confers several influential AI awards including the Feigenbaum Prize, the Newell Award (jointly with ACM), and the Engelmore Award. 
“Regina has been a trailblazer in the field of health care AI by asking the important questions about how we can use machine learning to treat and diagnose diseases,” says Daniela Rus, director of CSAIL and the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science. “She has been both a brilliant researcher and a devoted educator, and all of us at CSAIL are so inspired by her work and proud to have her as a colleague.”  More