Invention

Subterms

    Latest story

    163 Shares129 Views

    To excel at engineering design, generative AI must learn to innovate, study finds

    by

    ChatGPT and other deep generative models are proving to be uncanny mimics. These AI supermodels can churn out poems, finish symphonies, and create new videos and images by automatically learning from millions of examples of previous works. These enormously powerful and versatile tools excel at generating new content that resembles everything they’ve seen before.

    But as MIT engineers say in a new study, similarity isn’t enough if you want to truly innovate in engineering tasks.

    “Deep generative models (DGMs) are very promising, but also inherently flawed,” says study author Lyle Regenwetter, a mechanical engineering graduate student at MIT. “The objective of these models is to mimic a dataset. But as engineers and designers, we often don’t want to create a design that’s already out there.”

    He and his colleagues make the case that if mechanical engineers want help from AI to generate novel ideas and designs, they will have to first refocus those models beyond “statistical similarity.”

    “The performance of a lot of these models is explicitly tied to how statistically similar a generated sample is to what the model has already seen,” says co-author Faez Ahmed, assistant professor of mechanical engineering at MIT. “But in design, being different could be important if you want to innovate.”

    In their study, Ahmed and Regenwetter reveal the pitfalls of deep generative models when they are tasked with solving engineering design problems. In a case study of bicycle frame design, the team shows that these models end up generating new frames that mimic previous designs but falter on engineering performance and requirements.

    When the researchers presented the same bicycle frame problem to DGMs that they specifically designed with engineering-focused objectives, rather than only statistical similarity, these models produced more innovative, higher-performing frames.

    The team’s results show that similarity-focused AI models don’t quite translate when applied to engineering problems. But, as the researchers also highlight in their study, with some careful planning of task-appropriate metrics, AI models could be an effective design “co-pilot.”

    “This is about how AI can help engineers be better and faster at creating innovative products,” Ahmed says. “To do that, we have to first understand the requirements. This is one step in that direction.”

    The team’s new study appeared recently online, and will be in the December print edition of the journal Computer Aided Design. The research is a collaboration between computer scientists at MIT-IBM Watson AI Lab and mechanical engineers in MIT’s DeCoDe Lab. The study’s co-authors include Akash Srivastava and Dan Gutreund at the MIT-IBM Watson AI Lab.

    Framing a problem

    As Ahmed and Regenwetter write, DGMs are “powerful learners, boasting unparalleled ability” to process huge amounts of data. DGM is a broad term for any machine-learning model that is trained to learn distribution of data and then use that to generate new, statistically similar content. The enormously popular ChatGPT is one type of deep generative model known as a large language model, or LLM, which incorporates natural language processing capabilities into the model to enable the app to generate realistic imagery and speech in response to conversational queries. Other popular models for image generation include DALL-E and Stable Diffusion.

    Because of their ability to learn from data and generate realistic samples, DGMs have been increasingly applied in multiple engineering domains. Designers have used deep generative models to draft new aircraft frames, metamaterial designs, and optimal geometries for bridges and cars. But for the most part, the models have mimicked existing designs, without improving the performance on existing designs.

    “Designers who are working with DGMs are sort of missing this cherry on top, which is adjusting the model’s training objective to focus on the design requirements,” Regenwetter says. “So, people end up generating designs that are very similar to the dataset.”

    In the new study, he outlines the main pitfalls in applying DGMs to engineering tasks, and shows that the fundamental objective of standard DGMs does not take into account specific design requirements. To illustrate this, the team invokes a simple case of bicycle frame design and demonstrates that problems can crop up as early as the initial learning phase. As a model learns from thousands of existing bike frames of various sizes and shapes, it might consider two frames of similar dimensions to have similar performance, when in fact a small disconnect in one frame — too small to register as a significant difference in statistical similarity metrics — makes the frame much weaker than the other, visually similar frame.

    Beyond “vanilla”
    An animation depicting transformations across common bicycle designs. Credit: Courtesy of the researchers

    The researchers carried the bicycle example forward to see what designs a DGM would actually generate after having learned from existing designs. They first tested a conventional “vanilla” generative adversarial network, or GAN — a model that has widely been used in image and text synthesis, and is tuned simply to generate statistically similar content. They trained the model on a dataset of thousands of bicycle frames, including commercially manufactured designs and less conventional, one-off frames designed by hobbyists.

    Once the model learned from the data, the researchers asked it to generate hundreds of new bike frames. The model produced realistic designs that resembled existing frames. But none of the designs showed significant improvement in performance, and some were even a bit inferior, with heavier, less structurally sound frames.

    The team then carried out the same test with two other DGMs that were specifically designed for engineering tasks. The first model is one that Ahmed previously developed to generate high-performing airfoil designs. He built this model to prioritize statistical similarity as well as functional performance. When applied to the bike frame task, this model generated realistic designs that also were lighter and stronger than existing designs. But it also produced physically “invalid” frames, with components that didn’t quite fit or overlapped in physically impossible ways.

    “We saw designs that were significantly better than the dataset, but also designs that were geometrically incompatible because the model wasn’t focused on meeting design constraints,” Regenwetter says.

    The last model the team tested was one that Regenwetter built to generate new geometric structures. This model was designed with the same priorities as the previous models, with the added ingredient of design constraints, and prioritizing physically viable frames, for instance, with no disconnections or overlapping bars. This last model produced the highest-performing designs, that were also physically feasible.

    “We found that when a model goes beyond statistical similarity, it can come up with designs that are better than the ones that are already out there,” Ahmed says. “It’s a proof of what AI can do, if it is explicitly trained on a design task.”

    For instance, if DGMs can be built with other priorities, such as performance, design constraints, and novelty, Ahmed foresees “numerous engineering fields, such as molecular design and civil infrastructure, would greatly benefit. By shedding light on the potential pitfalls of relying solely on statistical similarity, we hope to inspire new pathways and strategies in generative AI applications outside multimedia.” More

    More stories

    • 113 Shares159 Views

      in Data Management & Statistics

      A new way to integrate data with physical objects

      by

      To get a sense of what StructCode is all about, says Mustafa Doğa Doğan, think of Superman. Not the “faster than a speeding bullet” and “more powerful than a locomotive” version, but a Superman, or Superwoman, who sees the world differently from ordinary mortals — someone who can look around a room and glean all kinds of information about ordinary objects that is not apparent to people with less penetrating faculties.

      That, in a nutshell, is “the high-level idea behind StructCode,” explains Doğan, a PhD student in electrical engineering and computer science at MIT and an affiliate of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “The goal is to change the way we interact with objects” — to make those interactions more meaningful and more meaning-laden — “by embedding information into objects in ways that can be readily accessed.”

      StructCode grew out of an effort called InfraredTags, which Doğan and other colleagues introduced in 2022. That work, as well as the current project, was carried out in the laboratory of MIT Associate Professor Stefanie Mueller — Doğan’s advisor, who has taken part in both projects. In last year’s approach, “invisible” tags — that can only be seen with cameras capable of detecting infrared light — were used to reveal information about physical objects. The drawback there was that many cameras cannot perceive infrared light. Moreover, the method for fabricating these objects and affixing the tags to their surfaces relied on 3D printers, which tend to be very slow and often can only make objects that are small.

      StructCode, at least in its original version, relies on objects produced with laser-cutting techniques that can be manufactured within minutes, rather than the hours it might take on a 3D printer. Information can be extracted from these objects, moreover, with the RGB cameras that are commonly found in smartphones; the ability to operate in the infrared range of the spectrum is not required.

      In their initial demonstrations of the idea, the MIT-led team decided to construct their objects out of wood, making pieces such as furniture, picture frames, flowerpots, or toys that are well suited to laser-cut fabrication. A key question that had to be resolved was this: How can information be stored in a way that is unobtrusive and durable, as compared to externally-attached bar codes and QR codes, and also will not undermine an object’s structural integrity?

      The solution that the team has come up with, for now, is to rely on joints, which are ubiquitous in wooden objects made out of more than one component. Perhaps the most familiar is the finger joint, which has a kind of zigzag pattern whereby two wooden pieces adjoin at right angles such that every protruding “finger” along the joint of the first piece fits into a corresponding “gap” in the joint of the second piece and, similarly, every gap in the joint of the first piece is filled with a finger from the second.

      “Joints have these repeating features, which are like repeating bits,” Dogan says. To create a code, the researchers slightly vary the length of the gaps or fingers. A standard size length is accorded a 1. A slightly shorter length is assigned a 0, and a slightly longer length is assigned a 2. The encoding scheme is based on the sequence of these numbers, or bits, that can be observed along a joint. For every string of four bits, there are 81 (34) possible variations.

      The team also demonstrated ways of encoding messages in “living hinges” — a kind of joint that is made by taking a flat, rigid piece of material and making it bendable by cutting a series of parallel, vertical lines. As with the finger joints, the distance between these lines can be varied: 1 being the standard length, 0 being a slightly shorter length, and 2 being slightly longer. And in this way, a code can be assembled from an object that contains a living hinge.

      The idea is described in a paper, “StructCode: Leveraging Fabrication Artifacts to Store Data in Laser-Cut Objects,” that was presented this month at the 2023 ACM Symposium on Computational Fabrication in New York City. Doğan, the paper’s first author, is joined by Mueller and four coauthors — recent MIT alumna Grace Tang ’23, MNG ’23; MIT undergraduate Richard Qi; University of California at Berkeley graduate student Vivian Hsinyueh Chan; and Cornell University Assistant Professor Thijs Roumen.

      “In the realm of materials and design, there is often an inclination to associate novelty and innovation with entirely new materials or manufacturing techniques,” notes Elvin Karana, a professor of materials innovation and design at the Delft University of Technology. One of the things that impresses Karana most about StructCode is that it provides a novel means of storing data by “applying a commonly used technique like laser cutting and a material as ubiquitous as wood.”

      The idea for StructCode, adds University of Colorado computer scientist Ellen Yi-Luen Do, “is “simple, elegant, and totally makes sense. It’s like having the Rosetta Stone to help decipher Egyptian hieroglyphs.”

      Patrick Baudisch, a computer scientist at the Hasso Plattner Institute in Germany, views StructCode as “a great step forward for personal fabrication. It takes a key piece of functionality that’s only offered today for mass-produced goods and brings it to custom objects.”

      Here, in brief, is how it works: First, a laser cutter — guided by a model created via StructCode — fabricates an object into which encoded information has been embedded. After downloading a StructCode app, an user can decode the hidden message by pointing a cellphone camera at the object, which can (aided by StructCode software) detect subtle variations in length found in an object’s outward-facing joints or living hinges.

      The process is even easier if the user is equipped with augmented reality glasses, Doğan says. “In that case, you don’t need to point a camera. The information comes up automatically.” And that can give people more of the “superpowers” that the designers of StructCode hope to confer.

      “The object doesn’t need to contain a lot of information,” Doğan adds. “Just enough — in the form of, say, URLs — to direct people to places they can find out what they need to know.”

      Users might be sent to a website where they can obtain information about the object — how to care for it, and perhaps eventually how to disassemble it and recycle (or safely dispose of) its contents. A flowerpot that was made with living hinges might inform a user, based on records that are maintained online, as to when the plant inside the pot was last watered and when it needs to be watered again. Children examining a toy crocodile could, through StructCode, learn scientific details about various parts of the animal’s anatomy. A picture frame made with finger joints modified by StructCode could help people find out about the painting inside the frame and about the person (or persons) who created the artwork — perhaps linking to a video of an artist talking about this work directly.

      “This technique could pave the way for new applications, such as interactive museum exhibits,” says Raf Ramakers, a computer scientist at Hasselt University in Belgium. “It holds the potential for broadening the scope of how we perceive and interact with everyday objects” — which is precisely the goal that motivates the work of Doğan and his colleagues.

      But StructCode is not the end of the line, as far as Doğan and his collaborators are concerned. The same general approach could be adapted to other manufacturing techniques besides laser cutting, and information storage doesn’t have to be confined to the joints of wooden objects. Data could be represented, for instance, in the texture of leather, within the pattern of woven or knitted pieces, or concealed by other means within an image. Doğan is excited by the breadth of available options and by the fact that their “explorations into this new realm of possibilities, designed to make objects and our world more interactive, are just beginning.” More

    • 113 Shares149 Views

      in Data Management & Statistics

      Communications system achieves fastest laser link from space yet

      by

      In May 2022, the TeraByte InfraRed Delivery (TBIRD) payload onboard a small CubeSat satellite was launched into orbit 300 miles above Earth’s surface. Since then, TBIRD has delivered terabytes of data at record-breaking rates of up to 100 gigabits per second — 100 times faster than the fastest internet speeds in most cities — via an optical communication link to a ground-based receiver in California. This data rate is more than 1,000 times higher than that of the radio-frequency links traditionally used for satellite communication and the highest ever achieved by a laser link from space to ground. And these record-setting speeds were all made possible by a communications payload roughly the size of a tissue box.

      MIT Lincoln Laboratory conceptualized the TBIRD mission in 2014 as a means of providing unprecedented capability to science missions at low cost. Science instruments in space today routinely generate more data than can be returned to Earth over typical space-to-ground communications links. With small, low-cost space and ground terminals, TBIRD can enable scientists from around the world to fully take advantage of laser communications to downlink all the data they could ever dream of.

      Designed and built at Lincoln Laboratory, the TBIRD communications payload was integrated onto a CubeSat manufactured by Terran Orbital as part of NASA’s Pathfinder Technology Demonstrator program. NASA Ames Research Center established this program to develop a CubeSat bus (the “vehicle” that powers and steers the payload) for bringing science and technology demonstrators into orbit more quickly and inexpensively. Weighing approximately 25 pounds and the size of two stacked cereal boxes, the CubeSat was launched into low-Earth orbit (LEO) aboard Space X’s Transporter-5 rideshare mission from Cape Canaveral Space Force Station in Florida in May 2022. The optical ground station is located in Table Mountain, California, where most weather takes place below the mountain’s summit, making this part of the sky relatively clear for laser communication. This ground station leverages the one-meter telescope and adaptive optics (to correct for distortions caused by atmospheric turbulence) at the NASA Jet Propulsion Laboratory Optical Communications Telescope Laboratory, with Lincoln Laboratory providing the TBIRD-specific ground communications hardware.

      “We’ve demonstrated a higher data rate than ever before in a smaller package than ever before,” says Jade Wang, the laboratory’s program manager for the TBIRD payload and ground communications and assistant leader of the Optical and Quantum Communications Technology Group. “While sending data from space using lasers may sound futuristic, the same technical concept is behind the fiber-optic internet we use every day. The difference is that the laser transmissions are taking place in the open atmosphere, rather than in contained fibers.”

      From radio waves to laser light

      Whether video conferencing, gaming, or streaming movies in high definition, you are using high-data-rate links that run across optical fibers made of glass (or sometimes plastic). About the diameter of a strand of human hair, these fibers are bundled into cables, which transmit data via fast-traveling pulses of light from a laser or other source. Fiber-optic communications are paramount to the internet age, in which large amounts of data must be quickly and reliably distributed across the globe every day.

      For satellites, however, a high-speed internet based on laser communications does not yet exist. Since the beginning of spaceflight in the 1950s, missions have relied on radio frequencies to send data to and from space. Compared to radio waves, the infrared light employed in laser communications has a much higher frequency (or shorter wavelength), which allows more data to be packed into each transmission. Laser communications will enable scientists to send 100 to 1,000 times more data than today’s radio-frequency systems — akin to our terrestrial switch from dial-up to high-speed internet.

      From Earth observation to space exploration, many science missions will benefit from this speedup, especially as instrument capabilities advance to capture larger troves of high-resolution data, experiments involve more remote control, and spacecraft voyage further from Earth into deep space.  

      However, laser-based space communication comes with several engineering challenges. Unlike radio waves, laser light forms a narrow beam. For successful data transmission, this narrow beam must be pointed precisely toward a receiver (e.g., telescope) located on the ground. And though laser light can travel long distances in space, laser beams can be distorted because of atmospheric effects and weather conditions. This distortion causes the beam to experience power loss, which can result in data loss.

      For the past 40 years, Lincoln Laboratory been tackling these and related challenges through various programs. At this point, these challenges have been reliably solved, and laser communications is rapidly becoming widely adopted. Industry has begun a proliferation of LEO cross-links using laser communications, with the intent to enhance the existing terrestrial backbone, as well as to provide a potential internet backbone to serve users in rural locations. Last year, NASA launched the Laser Communications Relay Demonstration (LCRD), a two-way optical communications system based on a laboratory design. In upcoming missions, a laboratory-developed laser communications terminal will be launched to the International Space Station, where the terminal will “talk” to LCRD, and support Artemis II, a crewed program that will fly by the moon in advance of a future crewed lunar landing.

      “With the expanding interest and development in space-based laser communications, Lincoln Laboratory continues to push the envelope of what is possible,” says Wang. “TBIRD heralds a new approach with the potential to further increase data rate capabilities; shrink size, weight, and power; and reduce lasercom mission costs.”

      One way that TBIRD aims to reduce these costs is by utilizing commercial off-the-shelf components originally developed for terrestrial fiber-optic networks. However, terrestrial components are not designed to survive the rigors of space, and their operation can be impacted by atmospheric effects. With TBIRD, the laboratory developed solutions to both challenges.

      Commercial components adapted for space

      The TBIRD payload integrates three key commercial off-the-shelf components: a high-rate optical modem, a large high-speed storage drive, and an optical signal amplifier.

      All these hardware components underwent shock and vibration, thermal-vacuum, and radiation testing to inform how the hardware might fare in space, where it would be subject to powerful forces, extreme temperatures, and high radiation levels. When the team first tested the amplifier through a thermal test simulating the space environment, the fibers melted. As Wang explains, in vacuum, no atmosphere exists, so heat gets trapped and cannot be released by convection. The team worked with the vendor to modify the amplifier to release heat through conduction instead.

      To deal with data loss from atmospheric effects, the laboratory developed its own version of Automatic Repeat Request (ARQ), a protocol for controlling errors in data transmission over a communications link. With ARQ, the receiver (in this case, the ground terminal) alerts the sender (satellite) through a low-rate uplink signal to re-transmit any block of data (frame) that has been lost or damaged.

      “If the signal drops out, data can be re-transmitted, but if done inefficiently — meaning you spend all your time sending repeat data instead of new data — you can lose a lot of throughput,” explains TBIRD system engineer Curt Schieler, a technical staff member in Wang’s group. “With our ARQ protocol, the receiver tells the payload which frames it received correctly, so the payload knows which ones to re-transmit.”

      Another aspect of TBIRD that is new is its lack of a gimbal, a mechanism for pointing the narrow laser beam. Instead, TBIRD relies on a laboratory-developed error-signaling concept for precision body pointing of the spacecraft. Error signals are provided to the CubeSat bus so it knows how exactly to point the body of the entire satellite toward the ground station. Without a gimbal, the payload can be even further miniaturized.

      “We intended to demonstrate a low-cost technology capable of quickly downlinking a large volume of data from LEO to Earth, in support of science missions,” says Wang. “In just a few weeks of operations, we have already accomplished this goal, achieving unprecedented transmission rates of up to 100 gigabits per second. Next, we plan to exercise additional features of the TBIRD system, including increasing rates to 200 gigabits per second, enabling the downlink of more than 2 terabytes of data — equivalent to 1,000 high-definition movies — in a single five-minute pass over a ground station.”

      Lincoln Laboratory developed the TBIRD mission and technology in partnership with NASA Goddard Space Flight Center. More

    • 75 Shares179 Views

      in Data Management & Statistics

      Unlocking new doors to artificial intelligence

      by

      Artificial intelligence research is constantly developing new hypotheses that have the potential to benefit society and industry; however, sometimes these benefits are not fully realized due to a lack of engineering tools. To help bridge this gap, graduate students in the MIT Department of Electrical Engineering and Computer Science’s 6-A Master of Engineering (MEng) Thesis Program work with some of the most innovative companies in the world and collaborate on cutting-edge projects, while contributing to and completing their MEng thesis.

      During a portion of the last year, four 6-A MEng students teamed up and completed an internship with IBM Research’s advanced prototyping team through the MIT-IBM Watson AI Lab on AI projects, often developing web applications to solve a real-world issue or business use cases. Here, the students worked alongside AI engineers, user experience engineers, full-stack researchers, and generalists to accommodate project requests and receive thesis advice, says Lee Martie, IBM research staff member and 6-A manager. The students’ projects ranged from generating synthetic data to allow for privacy-sensitive data analysis to using computer vision to identify actions in video that allows for monitoring human safety and tracking build progress on a construction site.

      “I appreciated all of the expertise from the team and the feedback,” says 6-A graduate Violetta Jusiega ’21, who participated in the program. “I think that working in industry gives the lens of making sure that the project’s needs are satisfied and [provides the opportunity] to ground research and make sure that it is helpful for some use case in the future.”

      Jusiega’s research intersected the fields of computer vision and design to focus on data visualization and user interfaces for the medical field. Working with IBM, she built an application programming interface (API) that let clinicians interact with a medical treatment strategy AI model, which was deployed in the cloud. Her interface provided a medical decision tree, as well as some prescribed treatment plans. After receiving feedback on her design from physicians at a local hospital, Jusiega developed iterations of the API and how the results where displayed, visually, so that it would be user-friendly and understandable for clinicians, who don’t usually code. She says that, “these tools are often not acquired into the field because they lack some of these API principles which become more important in an industry where everything is already very fast paced, so there’s little time to incorporate a new technology.” But this project might eventually allow for industry deployment. “I think this application has a bunch of potential, whether it does get picked up by clinicians or whether it’s simply used in research. It’s very promising and very exciting to see how technology can help us modify, or I can improve, the health-care field to be even more custom-tailored towards patients and giving them the best care possible,” she says.

      Another 6-A graduate student, Spencer Compton, was also considering aiding professionals to make more informed decisions, for use in settings including health care, but he was tackling it from a causal perspective. When given a set of related variables, Compton was investigating if there was a way to determine not just correlation, but the cause-and-effect relationship between them (the direction of the interaction) from the data alone. For this, he and his collaborators from IBM Research and Purdue University turned to a field of math called information theory. With the goal of designing an algorithm to learn complex networks of causal relationships, Compton used ideas relating to entropy, the randomness in a system, to help determine if a causal relationship is present and how variables might be interacting. “When judging an explanation, people often default to Occam’s razor” says Compton. “We’re more inclined to believe a simpler explanation than a more complex one.” In many cases, he says, it seemed to perform well. For instance, they were able to consider variables such as lung cancer, pollution, and X-ray findings. He was pleased that his research allowed him to help create a framework of “entropic causal inference” that could aid in safe and smart decisions in the future, in a satisfying way. “The math is really surprisingly deep, interesting, and complex,” says Compton. “We’re basically asking, ‘when is the simplest explanation correct?’ but as a math question.”

      Determining relationships within data can sometimes require large volumes of it to suss out patterns, but for data that may contain sensitive information, this may not be available. For her master’s work, Ivy Huang worked with IBM Research to generate synthetic tabular data using a natural language processing tool called a transformer model, which can learn and predict future values from past values. Trained on real data, the model can produce new data with similar patterns, properties, and relationships without restrictions like privacy, availability, and access that might come with real data in financial transactions and electronic medical records. Further, she created an API and deployed the model in an IBM cluster, which allowed users increased access to the model and abilities to query it without compromising the original data.

      Working with the advanced prototyping team, MEng candidate Brandon Perez also considered how to gather and investigate data with restrictions, but in his case it was to use computer vision frameworks, centered on an action recognition model, to identify construction site happenings. The team based their work on the Moments in Time dataset, which contains over a million three-second video clips with about 300 attached classification labels, and has performed well during AI training. However, the group needed more construction-based video data. For this, they used YouTube-8M. Perez built a framework for testing and fine-tuning existing object detection models and action recognition models that could plug into an automatic spatial and temporal localization tool — how they would identify and label particular actions in a video timeline. “I was satisfied that I was able to explore what made me curious, and I was grateful for the autonomy that I was given with this project,” says Perez. “I felt like I was always supported, and my mentor was a great support to the project.”

      “The kind of collaborations that we have seen between our MEng students and IBM researchers are exactly what the 6-A MEng Thesis program at MIT is all about,” says Tomas Palacios, professor of electrical engineering and faculty director of the MIT 6-A MEng Thesis program. “For more than 100 years, 6-A has been connecting MIT students with industry to solve together some of the most important problems in the world.” More