Arts

Subterms

    Latest story

    138 Shares139 Views

    AI generates high-quality images 30 times faster in a single step

    by

    In our current age of artificial intelligence, computers can generate their own “art” by way of diffusion models, iteratively adding structure to a noisy initial state until a clear image or video emerges. Diffusion models have suddenly grabbed a seat at everyone’s table: Enter a few words and experience instantaneous, dopamine-spiking dreamscapes at the intersection of reality and fantasy. Behind the scenes, it involves a complex, time-intensive process requiring numerous iterations for the algorithm to perfect the image.

    MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have introduced a new framework that simplifies the multi-step process of traditional diffusion models into a single step, addressing previous limitations. This is done through a type of teacher-student model: teaching a new computer model to mimic the behavior of more complicated, original models that generate images. The approach, known as distribution matching distillation (DMD), retains the quality of the generated images and allows for much faster generation. 

    “Our work is a novel method that accelerates current diffusion models such as Stable Diffusion and DALLE-3 by 30 times,” says Tianwei Yin, an MIT PhD student in electrical engineering and computer science, CSAIL affiliate, and the lead researcher on the DMD framework. “This advancement not only significantly reduces computational time but also retains, if not surpasses, the quality of the generated visual content. Theoretically, the approach marries the principles of generative adversarial networks (GANs) with those of diffusion models, achieving visual content generation in a single step — a stark contrast to the hundred steps of iterative refinement required by current diffusion models. It could potentially be a new generative modeling method that excels in speed and quality.”

    This single-step diffusion model could enhance design tools, enabling quicker content creation and potentially supporting advancements in drug discovery and 3D modeling, where promptness and efficacy are key.

    Distribution dreams

    DMD cleverly has two components. First, it uses a regression loss, which anchors the mapping to ensure a coarse organization of the space of images to make training more stable. Next, it uses a distribution matching loss, which ensures that the probability to generate a given image with the student model corresponds to its real-world occurrence frequency. To do this, it leverages two diffusion models that act as guides, helping the system understand the difference between real and generated images and making training the speedy one-step generator possible.

    The system achieves faster generation by training a new network to minimize the distribution divergence between its generated images and those from the training dataset used by traditional diffusion models. “Our key insight is to approximate gradients that guide the improvement of the new model using two diffusion models,” says Yin. “In this way, we distill the knowledge of the original, more complex model into the simpler, faster one, while bypassing the notorious instability and mode collapse issues in GANs.” 

    Yin and colleagues used pre-trained networks for the new student model, simplifying the process. By copying and fine-tuning parameters from the original models, the team achieved fast training convergence of the new model, which is capable of producing high-quality images with the same architectural foundation. “This enables combining with other system optimizations based on the original architecture to further accelerate the creation process,” adds Yin. 

    When put to the test against the usual methods, using a wide range of benchmarks, DMD showed consistent performance. On the popular benchmark of generating images based on specific classes on ImageNet, DMD is the first one-step diffusion technique that churns out pictures pretty much on par with those from the original, more complex models, rocking a super-close Fréchet inception distance (FID) score of just 0.3, which is impressive, since FID is all about judging the quality and diversity of generated images. Furthermore, DMD excels in industrial-scale text-to-image generation and achieves state-of-the-art one-step generation performance. There’s still a slight quality gap when tackling trickier text-to-image applications, suggesting there’s a bit of room for improvement down the line. 

    Additionally, the performance of the DMD-generated images is intrinsically linked to the capabilities of the teacher model used during the distillation process. In the current form, which uses Stable Diffusion v1.5 as the teacher model, the student inherits limitations such as rendering detailed depictions of text and small faces, suggesting that DMD-generated images could be further enhanced by more advanced teacher models. 

    “Decreasing the number of iterations has been the Holy Grail in diffusion models since their inception,” says Fredo Durand, MIT professor of electrical engineering and computer science, CSAIL principal investigator, and a lead author on the paper. “We are very excited to finally enable single-step image generation, which will dramatically reduce compute costs and accelerate the process.” 

    “Finally, a paper that successfully combines the versatility and high visual quality of diffusion models with the real-time performance of GANs,” says Alexei Efros, a professor of electrical engineering and computer science at the University of California at Berkeley who was not involved in this study. “I expect this work to open up fantastic possibilities for high-quality real-time visual editing.” 

    Yin and Durand’s fellow authors are MIT electrical engineering and computer science professor and CSAIL principal investigator William T. Freeman, as well as Adobe research scientists Michaël Gharbi SM ’15, PhD ’18; Richard Zhang; Eli Shechtman; and Taesung Park. Their work was supported, in part, by U.S. National Science Foundation grants (including one for the Institute for Artificial Intelligence and Fundamental Interactions), the Singapore Defense Science and Technology Agency, and by funding from Gwangju Institute of Science and Technology and Amazon. Their work will be presented at the Conference on Computer Vision and Pattern Recognition in June. More

    More stories

    • 113 Shares119 Views

      in Data Management & Statistics

      Co-creating climate futures with real-time data and spatial storytelling

      by

      Virtual story worlds and game engines aren’t just for video games anymore. They are now tools for scientists and storytellers to digitally twin existing physical spaces and then turn them into vessels to dream up speculative climate stories and build collective designs of the future. That’s the theory and practice behind the MIT WORLDING initiative.

      Twice this year, WORLDING matched world-class climate story teams working in XR (extended reality) with relevant labs and researchers across MIT. One global group returned for a virtual gathering online in partnership with Unity for Humanity, while another met for one weekend in person, hosted at the MIT Media Lab.

      “We are witnessing the birth of an emergent field that fuses climate science, urban planning, real-time 3D engines, nonfiction storytelling, and speculative fiction, and it is all fueled by the urgency of the climate crises,” says Katerina Cizek, lead designer of the WORLDING initiative at the Co-Creation Studio of MIT Open Documentary Lab. “Interdisciplinary teams are forming and blossoming around the planet to collectively imagine and tell stories of healthy, livable worlds in virtual 3D spaces and then finding direct ways to translate that back to earth, literally.”

      At this year’s virtual version of WORLDING, five multidisciplinary teams were selected from an open call. In a week-long series of research and development gatherings, the teams met with MIT scientists, staff, fellows, students, and graduates, as well as other leading figures in the field. Guests ranged from curators at film festivals such as Sundance and Venice, climate policy specialists, and award-winning media creators to software engineers and renowned Earth and atmosphere scientists. The teams heard from MIT scholars in diverse domains, including geomorphology, urban planning as acts of democracy, and climate researchers at MIT Media Lab.

      Mapping climate data

      “We are measuring the Earth’s environment in increasingly data-driven ways. Hundreds of terabytes of data are taken every day about our planet in order to study the Earth as a holistic system, so we can address key questions about global climate change,” explains Rachel Connolly, an MIT Media Lab research scientist focused in the “Future Worlds” research theme, in a talk to the group. “Why is this important for your work and storytelling in general? Having the capacity to understand and leverage this data is critical for those who wish to design for and successfully operate in the dynamic Earth environment.”

      Making sense of billions of data points was a key theme during this year’s sessions. In another talk, Taylor Perron, an MIT professor of Earth, atmospheric and planetary sciences, shared how his team uses computational modeling combined with many other scientific processes to better understand how geology, climate, and life intertwine to shape the surfaces of Earth and other planets. His work resonated with one WORLDING team in particular, one aiming to digitally reconstruct the pre-Hispanic Lake Texcoco — where current day Mexico City is now situated — as a way to contrast and examine the region’s current water crisis.

      Democratizing the future

      While WORLDING approaches rely on rigorous science and the interrogation of large datasets, they are also founded on democratizing community-led approaches.

      MIT Department of Urban Studies and Planning graduate Lafayette Cruise MCP ’19 met with the teams to discuss how he moved his own practice as a trained urban planner to include a futurist component involving participatory methods. “I felt we were asking the same limited questions in regards to the future we were wanting to produce. We’re very limited, very constrained, as to whose values and comforts are being centered. There are so many possibilities for how the future could be.”

      Scaling to reach billions

      This work scales from the very local to massive global populations. Climate policymakers are concerned with reaching billions of people in the line of fire. “We have a goal to reach 1 billion people with climate resilience solutions,” says Nidhi Upadhyaya, deputy director at Atlantic Council’s Adrienne Arsht-Rockefeller Foundation Resilience Center. To get that reach, Upadhyaya is turning to games. “There are 3.3 billion-plus people playing video games across the world. Half of these players are women. This industry is worth $300 billion. Africa is currently among the fastest-growing gaming markets in the world, and 55 percent of the global players are in the Asia Pacific region.” She reminded the group that this conversation is about policy and how formats of mass communication can be used for policymaking, bringing about change, changing behavior, and creating empathy within audiences.

      Socially engaged game development is also connected to education at Unity Technologies, a game engine company. “We brought together our education and social impact work because we really see it as a critical flywheel for our business,” said Jessica Lindl, vice president and global head of social impact/education at Unity Technologies, in the opening talk of WORLDING. “We upscale about 900,000 students, in university and high school programs around the world, and about 800,000 adults who are actively learning and reskilling and upskilling in Unity. Ultimately resulting in our mission of the ‘world is a better place with more creators in it,’ millions of creators who reach billions of consumers — telling the world stories, and fostering a more inclusive, sustainable, and equitable world.”

      Access to these technologies is key, especially the hardware. “Accessibility has been missing in XR,” explains Reginé Gilbert, who studies and teaches accessibility and disability in user experience design at New York University. “XR is being used in artificial intelligence, assistive technology, business, retail, communications, education, empathy, entertainment, recreation, events, gaming, health, rehabilitation meetings, navigation, therapy, training, video programming, virtual assistance wayfinding, and so many other uses. This is a fun fact for folks: 97.8 percent of the world hasn’t tried VR [virtual reality] yet, actually.”

      Meanwhile, new hardware is on its way. The WORLDING group got early insights into the highly anticipated Apple Vision Pro headset, which promises to integrate many forms of XR and personal computing in one device. “They’re really pushing this kind of pass-through or mixed reality,” said Dan Miller, a Unity engineer on the poly spatial team, collaborating with Apple, who described the experience of the device as “You are viewing the real world. You’re pulling up windows, you’re interacting with content. It’s a kind of spatial computing device where you have multiple apps open, whether it’s your email client next to your messaging client with a 3D game in the middle. You’re interacting with all these things in the same space and at different times.”

      “WORLDING combines our passion for social-impact storytelling and incredible innovative storytelling,” said Paisley Smith of the Unity for Humanity Program at Unity Technologies. She added, “This is an opportunity for creators to incubate their game-changing projects and connect with experts across climate, story, and technology.”

      Meeting at MIT

      In a new in-person iteration of WORLDING this year, organizers collaborated closely with Connolly at the MIT Media Lab to co-design an in-person weekend conference Oct. 25 – Nov. 7 with 45 scholars and professionals who visualize climate data at NASA, the National Oceanic and Atmospheric Administration, planetariums, and museums across the United States.

      A participant said of the event, “An incredible workshop that had had a profound effect on my understanding of climate data storytelling and how to combine different components together for a more [holistic] solution.”

      “With this gathering under our new Future Worlds banner,” says Dava Newman, director of the MIT Media Lab and Apollo Program Professor of Astronautics chair, “the Media Lab seeks to affect human behavior and help societies everywhere to improve life here on Earth and in worlds beyond, so that all — the sentient, natural, and cosmic — worlds may flourish.” 

      “WORLDING’s virtual-only component has been our biggest strength because it has enabled a true, international cohort to gather, build, and create together. But this year, an in-person version showed broader opportunities that spatial interactivity generates — informal Q&As, physical worksheets, and larger-scale ideation, all leading to deeper trust-building,” says WORLDING producer Srushti Kamat SM ’23.

      The future and potential of WORLDING lies in the ongoing dialogue between the virtual and physical, both in the work itself and in the format of the workshops. More

    • 75 Shares99 Views

      in Data Management & Statistics

      MIT at the 2023 Venice Biennale

      by

      The Venice Architecture Biennale, the world’s largest and most visited exhibition focusing on architecture, is once again featuring work by many MIT faculty, students, and alumni. On view through Nov. 26, the 2023 biennale, curated by Ghanaian-Scottish architect, academic, and novelist Lesley Lokko, is showcasing projects responding to the theme of “The Laboratory of Change.”

      Architecture and Planning and curator of the previous Venice Biennale. “Our students, faculty, and alumni have responded to the speculative theme with innovative projects at a range of scales and in varied media.”

      Below are descriptions of MIT-related projects and activities.

      MIT faculty participants

      Xavi Laida Aguirre, assistant professor of architecture

      Project: Everlasting Plastics

      Project description: SPACES, a nonprofit alternative art organization based in Cleveland, Ohio, and the U.S. Department of State’s Bureau of Educational and Cultural Affairs are behind the U.S. Pavilion’s exhibition at this year’s biennale. The theme, Everlasting Plastics, provides a platform for artists and designers to engage audiences in reframing the overabundance of plastic detritus in our waterways, landfills, and streets as a rich resource. Aguirre’s installation covers two rooms and holds a series of partial scenographies examining indoor proofing materials such as coatings, rubbers, gaskets, bent aluminum, silicone, foam, cement board, and beveled edges.

      Yolande Daniels, associate professor of architecture

      Project: The BLACK City Astrolabe: A Constellation of African Diasporic Women

      Project description: From the multiple displacements of race and gender, enter “The BLACK City Astrolabe,” a space-time field comprised of a 3D map and a 24-hour cycle of narratives that reorder the forces of subjugation, devaluation, and displacement through the spaces and events of African diasporic women. The diaspora map traces the flows of descendants of Africa (whether voluntary or forced) atop the visible tension between the mathematical regularity of meridians of longitude and the biases of international date lines.

      In this moment we are running out of time. The meridians and timeline decades are indexed to an infinite conical projection metered in decades. It structures both the diaspora map and timeline and serves as a threshold to project future structures and events. “The BLACK City Astrolabe” is a vehicle to proactively contemplate things that have happened, that are happening, and that will happen. Yesterday, a “Black” woman went to the future, and here she is.

      Mark Jarzombek, professor of architecture

      Project: Kishkindha NY

      Project description: “Kishkindha NY (Office of (Un)Certainty Research: Mark Jarzombek and Vikramaditya Parakash)” is inspired by an imagined forest-city as described in the ancient Indian text the Ramayana. It comes into being not through the limitations of human agency, but through a multi-species creature that destroys and rebuilds. It is exhibited as a video (Space, Time, Existence) and as a special dance performance.

      Ana Miljački, professor of architecture

      Team: Ana Miljački, professor of architecture and director of Critical Broadcasting Lab, MIT; Ous Abou Ras, MArch candidate; Julian Geltman, MArch; Recording and Design, faculty of Dramatic Arts, Belgrade; Calvin Zhong, MArch candidate. Sound design and production: Pavle Dinulović, assistant professor, Department of Sound Recording and Design, University of Arts in Belgrade.

      Collaborators: Melika Konjičanin, researcher, faculty of architecture, Sarajevo; Ana Martina Bakić, assistant professor, head of department of drawing and visual design, faculty of architecture, Zagreb; Jelica Jovanović, Grupa Arhitekata, Belgrade; Andrew Lawler, Belgrade; Sandro Đukić, CCN Images, Zagreb; Other Tomorrows, Boston.

      Project: The Pilgrimage/Pionirsko hodočašće

      Project description:  The artifacts that constitute Yugoslavia’s socialist architectural heritage, and especially those instrumental in the ideological wiring of several postwar generations for anti-fascism and inclusive living, have been subject to many forms of local and global political investment in forgetting their meaning, as well as to vandalism. The “Pilgrimage” synthesizes “memories” from Yugoslavian childhood visits to myriad postwar anti-fascist memorial monuments and offers them in a shifting and spatial multi-channel video presentation accompanied by a nonlinear documentary soundscape, presenting thus anti-fascism and unity as political and activist positions available (and necessary) today, for the sake of the future. Supported by: MIT Center for Art, Science, and Technology (CAST) Mellon Faculty Grant.

      Adèle Naudé Santos, professor of architecture, planning, and urban design; and Mohamad Nahleh, lecturer in architecture and urbanism; in collaboration with the Beirut Urban Lab at the American University of Beirut

      MIT research team: Ghida El Bsat, Joude Mabsout, Sarin Gacia Vosgerichian, Lasse Rau

      Project: Housing as Infrastructure

      Project description: On Aug. 4, 2020, an estimated 2,750 tons of ammonium nitrate stored at the Port of Beirut exploded, resulting in the deaths of more than 200 people and the devastation of port-adjacent neighborhoods. With over 200,000 housing units in disrepair, exploitative real estate ventures, and the lack of equitable housing policies, we viewed the port blast as a potential escalation of the mechanisms that have produced the ongoing affordable housing crisis across the city. 

      The Dar Group requested proposals to rethink the affected part of the city, through MIT’s Norman B. Leventhal Center for Advanced Urbanism. To best ground our design proposal, we invited the Beirut Urban Lab at the American University of Beirut to join us. We chose to work on the heavily impacted low-rise and high-density neighborhood of Mar Mikhael. Our resultant urban strategy anchors housing within a corridor of shared open spaces. Housing is inscribed within this network and sustained through an adaptive system defined by energy-efficiency and climate responsiveness. Cross-ventilation sweeps through the project on all sides, with solar panel lined roofs integrated to always provide adequate levels of electricity for habitation. These strategies are coupled with an array of modular units designed to echo the neighborhood’s intimate quality — all accessible through shared ramps and staircases. Within this context, housing itself becomes the infrastructure, guiding circulation, managing slopes, integrating green spaces, and providing solar energy across the community. 

      Rafi Segal, associate professor of architecture and urbanism, director of the Future Urban Collectives Lab, director of the SMArchS program; and Susannah Drake.

      Contributors: Olivia Serra, William Minghao Du

      Project:  From Redlining to Blue Zoning: Equity and Environmental Risk, Miami 2100 (2021)

      Project description: As part of Susannah Drake and Rafi Segal’s ongoing work on “Coastal Urbanism,” this project examines the legacy of racial segregation in South Florida and the existential threat that climate change poses to communities in Miami. Through models of coops and community-owned urban blocks, this project seeks to empower formerly disenfranchised communities with new methods of equity capture, allowing residents whose parents and grandparents suffered from racial discrimination to build wealth and benefit from increased real estate value and development.

      Nomeda Urbonas, Art, Culture, and Technology research affiliate; and Gediminas Urbonas, ACT associate professor

      Project: The Swamp Observatory

      Project description: “The Swamp Observatory” augmented reality app is a result of two-year collaboration with a school in Gotland Island in the Baltic Sea, arguably the most polluted sea in the world. Developed as a conceptual playground and a digital tool to augment reality with imaginaries for new climate commons, the app offers new perspective to the planning process, suggesting eco-monsters as emergent ecology for the planned stormwater ponds in the new sustainable city. 

      Sarah Williams, associate professor, technology and urban planning

      Team members: listed here.

      Project: DISTANCE UNKNOWN: RISKS AND OPPORTUNITIES OF MIGRATION IN THE AMERICAS 

      Project description: On view are visualizations made by the MIT Civic Data Design Lab and the United Nations World Food Program that helped to shape U.S. migration policy. The exhibition is built from a unique dataset collected from 4,998 households surveyed in El Salvador, Guatemala, and Honduras. A tapestry woven out of money and constructed by the hands of Central America migrants illustrates that migrants spent $2.2 billion to migrate from Central America in 2021.

      MIT student curators

      Carmelo Ignaccolo, PhD candidate, Department of Urban Studies and Planning (DUSP)

      Curator: Carmelo Ignaccolo; advisor: Sarah Williams; researchers: Emily Levenson (DUSP), Melody Phu (MIT), Leo Saenger (Harvard University), Yuke Zheng (Harvard); digital animation designer: Ting Zhang

      Exhibition Design Assistant: Dila Ozberkman (architecture and DUSP)

      Project: The Consumed City 

      Project description: “The Consumed City” narrates a spatial investigation of “overtourism” in the historic city of Venice by harnessing granular data on lodging, dining, and shopping. The exhibition presents two large maps and digital animations to showcase the complexity of urban tourism and to reveal the spatial interplay between urban tourism and urban features, such as landmarks, bridges, and street patterns. By leveraging by-product geospatial datasets and advancing visualization techniques, “The Consumed City” acts as a prototype to call for novel policymaking tools in cities “consumed” by “overtourism.”

      MIT-affiliated auxiliary events

      Rania Ghosn, associate professor of architecture and urbanism, El Hadi Jazairy, Anhong Li, and Emma Jurczynski, with initial contributions from Marco Nieto and Zhifei Xu. Graphic design: Office of Luke Bulman.

      Project: Climate Inheritance

      Project description: “Climate Inheritance” is a speculative design research publication that reckons with the complexity of “heritage” and “world” in the Anthropocene Epoch. The impacts of climate change on heritage sites — from Venice flooding to extinction in the Galapagos Islands — have garnered empathetic attention in a media landscape that has otherwise mostly failed to communicate the urgency of the climate crisis. In a strategic subversion of the media aura of heritage, the project casts World Heritage sites as narrative figures to visualize pervasive climate risks all while situating the present emergency within the wreckage of other ends of worlds, replete with the salvages of extractivism, racism, and settler colonialism.   

      Rebuilding Beirut: Using Data to Co-Design a New Future

      SA+P faculty, researchers, and students are participating in the sixth biennial architecture exhibition “Time Space Existence,” presented by the European Cultural Center. The exhibit showcases three collaborative research and design proposals that support the rebuilding efforts in Beirut following the catastrophic explosion at the Port of Beirut in August 2020.

      “Living Heritage Atlas” captures the significance and vulnerability of Beirut’s cultural heritage. 

      “City Scanner” tracks the environmental impacts of the explosion and the subsequent rebuilding efforts. “Community Streets” supports the redesign of streets and public space. 

      The work is supported by the Dar Group Urban Seed Grant Fund at MIT’s Norman B. Leventhal Center for Advanced Urbanism.

      Team members:Living Heritage AtlasCivic Data Design Lab and Future Heritage Lab at MITAssociate Professor Sarah Williams, co-principal investigator (PI)Associate Professor Azra Aksamija, co-PICity Scanner Senseable City Lab at MIT with the American University of Beirut and FAE Technology Professor Carlo Ratti, co-PIFábio Duarte, co-PISimone Mora, research and project leadCommunity Streets City Form Lab at MIT with the American University of BeirutAssociate Professor Andres Sevtsuk, co-PIProfessor Maya Abou-Zeid, co-PISchool of Architecture and Planning alumni participants   Rodrigo Escandón Cesarman SMArchS Design ’20 (co-curator, Mexican Pavilion)Felecia Davis PhD ’17 Design and Computation, SOFTLAB@PSU (Penn State University)Jaekyung Jung SM ’10, (with the team for the Korean pavilion)Vijay Rajkumar MArch ’22 (with the team for the Bahrain Pavilion)

      Other MIT alumni participants

      Basis with GKZ

      Team: Emily Mackevicius PhD ’18, brain and cognitive sciences, with Zenna Tavares, Kibwe Tavares, Gaika Tavares, and Eli Bingham

      Project description: The nonprofit research group works on rethinking AI as a “reasoning machine.” Their two goals are to develop advanced technological models and to make society able to tackle “intractable problems.” Their approach to technology is founded less on pattern elaboration than on the Bayes’ hypothesis, the ability of machines to work on abductive reasoning, which is the same used by the human mind. Two city-making projects model cities after interaction between experts and stakeholders, and representation is at the heart of the dialogue. More

    • 100 Shares119 Views

      in Data Management & Statistics

      Architectural heritage like you haven’t seen it before

      by

      The shrine of Khwaja Abu Nasr Parsa is a spectacular mosque in Balkh, Afghanistan. Also known as the “Green Mosque” due to the brilliant color of its tiled and painted dome, the intricately decorated building dates to the 16th century.

      If it were more accessible, the Green Mosque would attract many visitors. But Balkh is located in northern Afghanistan, roughly 50 miles from the border with Uzbekistan, and few outsiders will ever reach it. Still, anyone can now get a vivid sense of the mosque thanks to MIT’s new “Ways of Seeing” project, an innovative form of historic preservation.

      Play video

      PHD student Nikolaos Vlavianos created the following Extended Reality sequences for the “Ways of Seeing” project.

      “Ways of Seeing” uses multiple modes of imagery to produce a rich visual record of four historic building sites in Afghanistan — including colorful 3D still images, virtual reality imagery that takes viewers around and in some cases inside the structures, and exquisite hand-drawn architectural renderings of the buildings. The project’s imagery will be made available for viewing through the MIT Libraries by the end of June, with open access for the public. A subset of curated project materials will also be available through Archnet, an open access resource on the built environment of Muslim societies, which is a collaboration between the Aga Khan Documentation Center of the MIT Libraries and the Aga Khan Trust for Culture.

      “After the U.S. withdrawal from Afghanistan in August 2021, Associate Provost Richard Lester convened a set of MIT faculty in a working group to think of what we as a community of scholars could be doing that would be meaningful to people in Afghanistan at this point in time,” says Fotini Christia, an MIT political science professor who led the project. “‘Ways of Seeing’ is a project that I conceived after discussions with that group of colleagues and which is truly in the MIT tradition: It combines field data, technology, and art to protect heritage and serve the world.”

      Christia, the Ford International Professor of the Social Sciences and director of the Sociotechnical Systems Research Center at the MIT Schwarzman College of Computing, has worked extensively in Afghanistan conducting field research about civil society. She viewed this project as a unique opportunity to construct a detailed, accessible record of remarkable heritage sites — through sophisticated digital elements as well as finely wrought ink drawings.

      “The idea is these drawings would inspire interest and pride in this heritage, a kind of amazement and motivation to preserve this for as long as humanly possible,” says Jelena Pejkovic MArch ’06, a practicing architect who made the large-scale renderings by hand over a period of months.

      Pejkovic adds: “These drawings are extremely time-consuming, and for me this is part of the motivation. They ask you to slow down and pay attention. What can you take in from all this material that we have collected? How do you take time to look, to interpret, to understand what is in front of you?”

      The project’s “digital transformation strategy” was led by Nikolaos Vlavianos, a PhD candidate in the Department of Architecture’s Design and Computation group. The group uses cutting-edge technologies and drones to make three-dimensional digital reconstructions of large-scale architectural sites and create immersive experiences in extended reality (XR). Vlavianos also conducts studies of the psychological and physiological responses of humans experiencing such spaces in XR and in person. 

      “I regard this project as an effort toward a broader architectural metaverse consisting of immersive experiences in XR of physical spaces around the world that are difficult or impossible to access due to political, social, and even cultural constraints,” says Vlavianos. “These spaces in the metaverse are information hubs promoting an embodied experiential approach of living, sensing, seeing, hearing, and touching.”

      Nasser Rabbat, the Aga Khan Professor and director of the Aga Khan Program for Islamic Architecture at MIT, also offered advice and guidance on the early stages of the project.

      The project — formally titled “Ways of Seeing: Documenting Endangered Built Heritage in Afghanistan” — encompasses imaging of four quite varied historical sites in Afghanistan.

      These are the Green Mosque in Balkh; the Parwan Stupa, a Buddhist dome south of Kabul; the tomb of Gawhar Saad, in Herat, in honor of the queen of the emperor of the Timurid, who was herself a highly influential figure in the 14th and 15th centuries; and the Minaret of Jam, a remarkable 200-foot tall tower dating to the 12th century, next to the Hari River in a distant spot in western Afghanistan.

      The sites thus encompass multiple religions and a diversity of building types. Many are in remote locations within Afghanistan that cannot readily be accessed by visitors — including scholars.

      “Ways of Seeing” is supported by a Mellon Faculty Grant from the MIT Center for Art, Science, and Technology (CAST), and by faculty funding from the MIT School of Humanities, Arts, and Social Sciences (SHASS). It is co-presented with the Institute for Data, Systems, and Society (IDSS), the Sociotechnical Systems Research Center (SSRC) at the MIT Schwarzman College of Computing, the MIT Department of Political Science, and SHASS.

      Two students from Wellesley College participating in MIT’s Undergraduate Research Opportunities Program (UROP), juniors Meng Lu and Muzi Fang, also worked on the project under the guidance of Vlavianos to create a video game for children involving the Gawhar Saad heritage site. 

      To generate the imagery, the MIT team worked with an Afghan digital production team that was on the ground in the country; they went to the four sites and took thousands of pictures, having been trained remotely by Vlavianos to perform a 3D scanning aerial operation. They were led by Shafic Gawhari, the managing director for Afghanistan at the Moby Group, an international media enterprise; others involved were Mohammad Jan Kamal, Nazifullah Benaam, Warekzai Ghayoor, Rahm Ali Mohebzada, Mohammad Harif Ghobar, and Abdul Musawer Anwari.

      The journalists documented the sites by collecting 15,000 to 30,000 images, while Vlavianos computationally generated point clouds and mesh geometry with detailed texture mapping. The outcome of those models consisted of still images,  immersive experiences in XR, and data for Pejkovic.  

      “‘Ways of Seeing’ proposes a hybrid model of remote data collection,” says Vlavianos, who in his time at MIT has also led similar projects at Machu Picchu in Peru, and the Simonos Petra monastery at Mount Athos, Greece. To produce similar imagery even more easily, he says, “The next step — which I am working on — is to utilize autonomous drones deployed simultaneously in various locations on the world for rapid production and advanced neural network algorithms to generate models from lower number of images.”  

      In the future, Vlavianos envisions documenting and reconstructing other sites around the world using crowdsourcing data, historical images, satellite imagery, or even by having local communities learn XR techniques. 

      Pejkovic produced her drawings based on the digital models assembled by Vlavianos, carefully using a traditional rendering technique in which she would first outline the measurements of each structure, at scale, and then gradually ink in the drawings to give the buildings texture. The inking technique she used is based on VERNADOC, a method of documenting vernacular architecture developed by the Finnish architect Markku Mattila.

      “I wanted to rediscover the most traditional possible kind of documentation — measuring directly by hand, and drawing by hand,” says Pejkovic. She has been active in conservation of cultural heritage for over 10 years.

      The first time Pejkovic ever saw this type of hand-drawn renderings in person, she recalls thinking, “This is not possible, a human being cannot make drawings like this.” However, she wryly adds, “You know the motto at MIT is ‘mens et manus,’ mind and hand.” And so she embarked on hand drawing these renderings herself, at a large scale — her image of the Minaret of Jam has been printed in a crisp 8-foot version by the MIT team.

      “The ultimate intent of this project has been to make all these outputs, which are co-owned with the Afghans who carried out the data collection on the ground, available to Afghan refugees displaced around the world but also accessible to anyone keen to witness them,” Christia says. “The digital twins [representations] of these sites are also meant to work as repositories of information for any future preservation efforts. This model can be replicated and scaled for other heritage sites at risk from wars, environmental disaster, or cultural appropriation.” More

    • 38 Shares109 Views

      in Data Management & Statistics

      Visualizing migration stories

      by

      On July 27, 2020, 51 people migrating to the United States were found dead in an overheated trailer near the Mexican border. Understanding why migrants willingly take such risks is the topic of a recent exhibition and report, co-authored by researchers at MIT’s Civic Data Design Lab (CDDL). The research has been used by the U.S. Senate and the United Nations to develop new policies to address the challenges, dangers, and opportunities presented by migration in the Americas.

      To illustrate these motivations and risks, researchers at CDDL have designed an exhibition featuring digital and physical visualizations that encourage visitors to engage with migrants’ experiences more fully. “Distance Unknown” made its debut at the United Nations World Food Program (WFP) executive board meeting in Rome earlier this summer, with plans for additional exhibition stops over the next year.

      The exhibition is inspired by the 2021 report about migration, co-authored by CDDL, that highlighted economic distress as the main factor pushing migrants from Central America to the United States. The report’s findings were cited in a January 2022 letter from 35 U.S. senators to Homeland Security Secretary Alejandro Mayorkas and Secretary of State Antony Blinken (who leads the Biden administration’s migration task force) that advocated for addressing humanitarian needs in Central America. In June, the United States joined 20 countries in issuing the Los Angeles Declaration on Migration and Protection, which proposed expanded legal avenues to migration.

      “This exhibition takes a unique approach to visualizing migration stories by humanizing the data. Visitors to the exhibition can see the data in aggregate, but then they can dive deeper and learn migrants’ individual motivations,” says Sarah Williams, associate professor of technology and urban planning, director of the Civic Data Design Lab and the Norman B. Leventhal Center for Advanced Urbanism, and the lead designer of the exhibition.

      The data for the exhibition were taken from a survey of over 5,000 people in El Salvador, Guatemala, and Honduras conducted by the WFP and analyzed in the subsequent report. The report showed that approximately 43 percent of people surveyed in 2021 were considering migrating in the prior year, compared to 8 percent in 2019 — a change that comes after nearly two years of impacts from a global pandemic and as food insecurity dramatically increased in that region. Survey respondents cited low wages, unemployment, and minimal income levels as factors increasing their desire to migrate — ahead of reasons such as violence or natural disasters. 

      On the wall of the exhibition is a vibrant tapestry made of paper currency woven by 13 Latin American immigrants. Approximately 15-by-8 feet, this physical data visualization explains the root causes of migration from Central America documented by CDDL research. Each bill in the tapestry represents one migrant; visitors are invited to take a piece of the tapestry and scan it at a touch-screen station, where the story of that migrant appears. This allows visitors to dive deeper into the causes of migration by learning more about why an individual migrant family in the study left home, their household circumstances, and their personal stories.

      Another feature of the exhibition is an interactive map that allows visitors to explore the journeys and barriers that migrants face along the way. Created from a unique dataset collected by researchers from internet hotspots along the migration trail, the data showed that migrants from 43 countries (some as distant as China and Afghanistan) used this Latin American trail. The map highlights the Darien Gap region of Central America, one of the most dangerous and costly migration routes. The area is remote, without roads, and consists of swamps and dense jungle.

      The “Distance Unknown” exhibition represented data taken from internet hotspots on the migration pathway from the Darien Gap in Colombia to the Mexican border. This image shows migrant routes from 43 countries.

      Image courtesy of the Civic Data Design Lab.

      Previous item
      Next item

      The intense multimedia exhibition demonstrates the approach that Williams takes with her research. “One of the exciting features of the exhibition is that it shows that artistic forms of data visualization start new conversations, which create the dialogue necessary for policy change. We couldn’t be more thrilled with the way the exhibition helped influence the hearts and minds of people who have the political will to impact policy,” says Williams.

      In his opening remarks to the exhibition, David Beasley, executive director of WFP, explained that “when people have to migrate because they have no choice, it creates political problems on all sides,” and emphasized the importance of proposing solutions. Citing the 2021 report, Beasley noted that migrants from El Salvador, Guatemala, and Honduras collectively spent $2.2 billion to migrate to the United States in 2021, which is comparable to what their respective governments spend on primary education.

      The WFP hopes to bring the exhibition to other locations, including Washington, Geneva, New York, Madrid, Buenos Aires, and Panama. More