Robotics

Miso Robotics’ CookRight Coffee system. 
Miso Robotics
Coffee is one of those habits around which much ritual has cropped up. You go to the cafe, if you’re lucky your barista knows your order and gives you a warm smile, and you savor that first delightful sip. So are we ready for a robot to take charge of our brew?That’s the bet that Miso Robotics, whose dexterous robot fry cooks are already slinging burgers and chicken wings at fast food restaurants, is making. Miso is launching a new coffee system and the first partner is Panera Bread.

To be sure, technology and coffee are no strangers. From pods to Aeropress, the hunt for a perfect cup has gone on as long as beans have been roasted. Panera is betting the efficiency and quality of a system that continually monitors coffee status using AI and customized metrics will appeal to patrons. The subplot here is the rapid automation of the quick serve restaurant industry, which is reeling from labor shortages and struggling to keep up with high demand coming out of the worst lockdowns of the pandemic. There’s now a real sense that momentum is shifting toward robotic systems to add greater efficiency to human-led, front-of-house operations.”Panera has a long history of tech innovation in service of meeting the needs of our guests and associates when they walk through our doors each day,” said George Hanson, SVP and Chief Digital Officer of Panera. “CookRight Coffee is a game changer when it comes to convenience and operational efficiency, and we are extremely excited to take our coffee station into the future with Miso Robotics.”Miso is on a bit of a tear, having followed up its burger robot, Flippy 2, with a tortilla chip making robot in partnership with Chipotle. Automation seems well paired with rising takeout demand during a pandemic-influenced tight labor market. Delivery, takeout, and drive-thru orders in particular have increased the need for speed just as demand is booming, and restaurants are having trouble keeping pace. As I’ve written, one of the big draws for national brands at this early adoption stage is Miso’s strategy is the speed and efficiency of a robotic system paired with unseen levels of customization. For Panera, Miso’s CookRight Coffee system monitors coffee volume and temperature to brew at precisely the right time. Panera in turn can support a club-style membership program that gives members unlimited coffee and tea for $8.99 per month.Miso previously partnered with Lancer Worldwide, a global beverage dispenser manufacturer, to roll out what’s described as an intelligence-backed, automated beverage dispenser.All of this has made Miso a popular investment with the crowdfund crowd. The company is a crowdfunding success story with over 18,000 shareholders and $50MM in crowdfunding to date and a Series E market valuation of $500 million.  More

Peloton Guide
Peloton
AI is driving the future of fitness, and companies like Peloton are leveraging the technology to enhance products and improve experiences for users. But what role will AI and data play in the future of at-home and connected fitness, and how it will increasingly shape the landscape?

Peloton Guide (Peloton’s first connected strength device) is a good case study. It uses computer vision and machine learning technology to create focused and well-rounded training experiences from home. Guide’s Movement Tracker can recognize a user’s activity, encouraging and keeping them motivated to keep up with the Instructor’s cues.AI is now a core tenant of Peloton and many other major home exercise brands. For insights into the future of smart connected home fitness (and some deeper understanding of just how embarrassed we’re going to feel at a machine’s consoling prods), I connected with Sanjay Nichani, Peloton’s VP of Artificial Intelligence and Computer Vision.GN: Across the market, where are we seeing AI intersect with fitness?Sanjay Nichani: This is a great question because we’re seeing AI intersect with fitness more and more with consumers and their experiences. AI is something we’re continuing to tap into at Peloton, and as we continue to conduct research, test products and speak with more people, including our Members — we’re able to unlock additional ways that AI can improve the at-home fitness experience. Specifically, we see that AI can be used to drive convenience, accountability, motivation, education, gamification, competition, collaboration and social connection within the fitness market.GN: Can you describe Peloton’s development history with AI? When did it first become a priority, and how has it grown in importance?Sanjay Nichani: Working with the latest technology is always top of mind for us because we’re always looking to enhance the Peloton experience. We’ve been working with AI for a few years now, starting, of course, with our class recommender system. You’ll really see AI take center stage with Peloton Guide since it is our first connected strength product. AI drives the experiences such as the Self Mode so that you can see yourself on the screen next to the instructor, Movement Tracker that gives your credit for following along with an instructor, and Body Activity that powers class recommendations to ensure you are working all muscle groups evenly. GN: How is machine vision aiding Peloton’s offerings? Can you explain what the Guide product is and how CV and ML help shape the user experience?Sanjay Nichani: Peloton Guide connects to any TV to transform the biggest screen in any home into an interactive personal training studio. Once it’s connected, Members have access to Peloton’s world-class instructors who lead a wide range of fun-yet-intense classes and programs that use dumbbells and bodyweight. Since Members and experts told us that they derive motivation from their metrics, we’re using AI for Guide’s Movement Tracker.It’s really cool to see Guide’s Movement Tracker using Computer Vision activity recognition technology to recognize a Member’s activity as they follow along with the Instructors and complete each move throughout the class. For example, during a class, an Instructor will have a plan where they’ll be coaching Members through different movements like bicep curls for 30 seconds or squats for 45 seconds. Guide recognizes the activity and metric-driven accountability to our members to keep them motivated to keep up with the Instructor’s cues. Additionally, with Self Mode, Guide’s smart frame technology where the camera automatically pans and zooms on the member working out, you can see yourself on screen and compare your form to the Instructor’s. Peloton Guide also shows members’ muscle groups they have recently worked on with a new feature called Body Activity. With this feature, Peloton will then recommend classes focused on the muscle groups that haven’t been trained in awhile to help Members have a more well-rounded training experience.GN: Have there been any interesting learnings or insights from customer reactions? Have you changed course in any way based on unexpected findings regarding user experience?Sanjay Nichani: One interesting insight in various user and field testing trials was the value of having a bounding box around the person detected; this established a strong connection of the member to the Guide (proof that the Guide had detected the member, “seen them” and they were now locked. This simple visual feedback was far more valuable than perhaps displaying a skeletal pose that was too busy and distracting, taking away from their exercise experience, or otherwise swinging to the other end, where nothing was displayed, which made users feel disconnected.) From the very beginning, data-driven insights have been baked into not only our company culture but into the products we produce. For example, our strength Members who are creating a gym experience from the comforts of their home without a human coach may not hold themselves accountable. The Members and experts we talk to often tell us that little feedback and motivation they get from the metrics is what keeps them going, e.g., you did x number last week, and this week your number went up. This is exclusively a result of our cutting edge AI technology, Guide’s Movement Tracker. Our AI teams ensure that customer needs and feedback are woven into our product planning and assessment. We work with a number of other departments — Systems Engineering, UI/UX Design, User Research, QA, Field Testing — to ensure that the way AI is implemented within our offerings is directly addressing the need of our consumers. GN: What does the future of home fitness look like (for Peloton and beyond)? How are AI and MV helping shape that experience?Sanjay Nichani: Honestly, we’re just scratching the surface of how AI  technology can impact fitness. Our AI teams ensure that customer needs and feedback are woven into our product planning and assessment. We work with a number of other departments — Systems Engineering, UI/UX Design, User Research, QA, Field Testing — to ensure that the way AI is implemented within our offerings is directly addressing the need of our consumers. We have a top-notch cross-functional team optimizing and diversifying our CV and ML tools to usher in new, safe and fun ways to practice fitness. You can also see a future where CV and ML can help create more personalized content or offer real-time feedback. There’s a lot of potential with the technology, and for Peloton, we’re going to continue experimenting. On day one Guide is going to provide a really different and motivating strength experience. But because Guide is because it’s built on CV and ML, we have an opportunity to keep iterating and making the product stronger with more features, exercises and disciplines. We’re continuing to conduct field testing and have plans to keep updating Guide. 

Artificial Intelligence More

Worker inspects sustainable packaging.
Zume
A fleet of robots and some fancy sustainable material may point the way to solving a vexing problem: How to reduce harmful single-use packaging materials, which include not just plastics and polystyrene but also chemicals used to line cardboard packaging.

The company in question is Zume, and you might remember the name from its first life as a pizza-making robotics firm once valued in the billions. After trying to scale its end to end automated pizza business too quickly, the Softbank-backed brand pivoted went through major layoffs and then pivoted to sustainable packaging. With a breakthrough partnership with global robotics leader ABB, as well as a new partnership with Solenis, a leading global producer of speciality chemicals, Zume is launching a line of 100% PFAS-free compostable packaging for the food packaging industry, capping off a major brand reinvention.Why is this important? Ever looked closely at the inside of a cardboard takeout box? That sheen keeping your lo mein from soaking into the cardboard is most likely PFAS (per- and polyfluoroalkyl substances), widely used chemicals commonly found in sustainable food packaging. These toxic chemicals bind to proteins in the soil, transmit up the food chain and have been linked to a slew of health risks.”Until now, an economically viable solution for brands to transition from plastic and foam packaging hasn’t been available,” said Zume CEO and Chairman Alex Garden. “Our patented molded fiber manufacturing equipment system and technology enable us to offer sustainable packaging at the same price or less than plastic. This partnership with Solenis advances efforts to eliminate single-use plastic and enables brands to keep commitments to stop using PFAS.”Also: Robot fry cook gets job at 100 White Castle locationsZume and Solenis have been working together to innovate molded fiber solutions to replace entire categories of single-use plastic without harmful chemicals. Part of the breakthrough here is a robotics-enabled manufacturing process, something that’s given single-use plastics manufacturers a leg up when it comes to pricing single-use packaging. Under the terms of the new partnership, Zume will provide its advanced molded fiber manufacturing capabilities and professional services, and Solenis will supply expertise in functional additives and surface coatings to increase the strength and functionality of Zume’s fiber recipes making them adequate for a broad array of commercial uses.Initial products will include cups, bowls, plates, premium egg cartons, coffee cup lids, protein trays, and yogurt cups. ABB’s robots will automate production and enable the scale and speed needed to make Zume’s sustainable packaging. More

DroneUp
Swinging back against the Bezos behemoth and searching for every advantage in an increasingly digital world, Walmart is looking to the skies. Drone delivery is coming, and Walmart is using its vast geographical footprint in a modern game of retail Risk.As the FAA increasingly embraces commercial drone operations, drone delivery in the United States is closer than ever to reality. In concert with company DroneUp, Walmart has rolled out the first of its drone “Hubs,” which are co-located with Walmart stores. Walmarts have 90 percent of the U.S. population within 10 miles of their stores, and each store is optimized for that particular area’s audience.

Walmart invested in DroneUp last June, signaling the retailer’s commitment to drone services for the long haul. This isn’t just about retail. In a bid to integrate Hubs with the local communities they serve, drone services are also being delivered, including community resources for law enforcement, insurance inspections, infrastructure inspections, and other use cases that businesses in those areas need from a drone. This is a big move for Walmart and a coup for DroneUp, which has been expanding its drone services offerings.”First, you have to understand the role of the Hub and why positioning those relative to Walmarts is so valuable,” Tom Walker, DroneUp’s CEO, told me recently. “The goal is to be able to successfully deliver products that people want store-to-door consistently, and in order to do that, it’s important to have access to an inventory of products that are readily and consistently available despite supply chain issues.”Here’s where Walmart’s baked-in regional fluency is a big advantage. Walmart stocks in-market products for the audience in and around the Hub locations. Positioning these Hubs next to Walmarts gives DroneUp direct access to tens of thousands of SKUs that are specifically designed for the demographic audience in and around that location.”After the order is made,” said Walker, “the product is brought from the store to the Hub, loaded onto the drone then flown in an optimized route to the home. We deliver goods from as low as 80 feet to as high as 100 feet, and gently set the product on the ground at the customer’s location. We are doing that consistently today in under 30 minutes from the time the product(s) is ordered until the product is at the customer’s home.”One of the interesting subplots here is that there’s not much of a labor force for a commercial drone sector, and the challenges of creating one from scratch are profound in a historically tight labor market. DroneUp has launched a job training initiative where they fly new employees into one of their hubs for training on the drones, the delivery operations, and how to maintain the drones themselves. It’s a compelling illustration of what the industry has been promising for years: jobs, delivery, services, and safety.”We believe that one of the key components to being able to scale these operations is talent,” says Walker. “We invested specifically into our recruiting engine to bring on the right people. We understand that not only is recruitment important, but retention is important, too. Turnover can create challenges when you’re trying to scale operations as quickly as we are. So we are investing significantly into training programs. We’re actually building a facility right now in Virginia where we’ll be able to bring the operators in for training. There’ll be a couple of weeks of online training, and then a couple of weeks of in-person training covering around-the-clock data, daylight operations, nighttime operations, different ways of dealing with potential issues that arise, and more, and then they’ll go out in the field.”Underscoring all this is an evolution in the FAA’s view of beyond visual line of sight (BVLOS) flying. “We’re excited about the FAA leaning into this and recognizing that in order for this industry to scale, we’re going to have to fly beyond the visual line of sight. One of the challenges has always been that we’ve held the unmanned systems world to the same level of safety [standards] that we’ve held the manned aviation community, and that’s a very difficult thing to do in terms of where and how we’re operating.”Walker points out that the recent FAA Aviation Rulemaking Committee (ARC) recommendations were very specific when it comes to risk: how do we appropriately manage and measure risk to enable us to go beyond our visual line of sight? “The important thing I say to the community is we don’t need to immediately go from visual line of sight to unlimited BVLOS. We need to make incremental changes. How can we go from here to a little bit further, then a bit further than that, and maintain safety while determining an acceptable level of risk? Does it have to be equivalent to standard general aviation rules that are carrying personnel and so forth?”Last November the companies announced plans to open three drone Hubs at Arkansas Walmarts, with more locations pending.

ZDNet Recommends More

A drone collecting data in the forest using Treeswift technology.
Treeswift
One of the major challenges of managing forest lands against threats like fire and deforestation is the daunting logistical hurdle of collecting data. Drones darting through forests under the canopy may help.

The precarious-sounding premise has won investors a big vote of confidence to the tune of $6.4 million, including a recently closed $4.8 million round. Treeswift, a startup that’s developed a drone=based solution to navigate under the forest canopy to collect vast quantities of data, including information on individual trees, announced the seed round led by Pathbreaker Ventures.The problem is profound, and it’s not hard to spot the market need. “Our mission is to build the data ecosystem for the natural world, and we are accomplishing that by capturing important data from below the forest canopy,” said Steven Chen, Treeswift’s CEO. “We anticipate that Treeswift technology will help to obtain a more transparent, verifiable, and accurate view of the planet from the ground up.”Also: Drone delivery nearer to take-off following latest FAA recommendationsNot surprisingly, Treeswift is iterating on a previously performed process with a mishmash of technologies and boots on the ground measurement. Previous approaches used remote sensing methods such as satellites or planes, but this top-down method is insufficient for detailed insights about what’s going on beneath the canopy. That job has traditionally been accomplished by human workers called timber cruisers, who take to the forest floor with journals and tape measures. Drones that employ a variety of cameras, sensors, and advanced machine learning algorithms can simply cover more ground faster while doing a more comprehensive job of collecting data sets.That’s important for a variety of industries and use cases. Forestry data can help provide a detailed picture of fire risk, for example, by identifying highly combustible dead and damaged trees. Climate models depend on accurate measures of carbon capture, which is directly related to the health and size of trees in forests. Also: Butterfly propulsion for electric aviation commences testingTreeswift is advised by Dr. Harold Burkhart of Virginia Tech’s Department of Forest Resources and Environmental Conservation and Dr. Vijay Kumar, the Nemirovsky Family Dean of Engineering at the University of Pennsylvania’s GRASP Laboratory.Of course, extractive industries can also use tools that can be used for conservation can also be used by extractive industries; one of the primary customers for Treeswift is likely to be the commercial logging sector. “The Molpus team has been particularly impressed with Treeswift’s ability to allocate trees into product classes accurately and to identify stem quality issues impacting value,” said Randy Taylor, Senior Director, Resource Planning of Molpus Woodlands Group. “Understanding and accurately maintaining timber inventories play an important part in managing our clients’ forests to their potential.”The hope is that equipping the commercial forestry sector with tools to better identify and target which trees it cuts may help avoid the indiscriminate cutting methods that are destructive to habitats and soil health and can actually increase fire risk. More

Striped hover fly.
Peakpx
Bio-inspired design has been a hallmark of technological advancement, and that’s still true in the age of flying robots. The latest proof comes out of Australia, where researchers have mapped the visual systems of hovering insects as a means of detecting the acoustic signatures of drones up to 2.5 miles away.Anthony Finn, University of South Australia Professor of Autonomous Systems, says that insect vision systems have been mapped for some time now to improve camera-based detections. But applying the same method to acoustic data represents a major innovation. 

“Bio-vision processing has been shown to greatly increase the detection range of drones in both visual and infrared data. However, we have now shown we can pick up clear and crisp acoustic signatures of drones, including very small and quiet ones, using an algorithm based on the hover fly’s visual system,” Finn says.The potential applications of the research, of course, include military and defense uses. In addition to the University of South Australia and Flinders University, defense company Midspar Systems participated in trials using bio-inspired signal processing techniques. Such techniques, according to the researchers, show up to a 50% better detection rate than existing methods.The hover fly, which can hover above plants to collect nectar, was chosen because of its superior visual and tracking skills. Dark lit regions are visually very noisy, but insects such as the hover fly can process and capture visual signals with remarkable effectiveness. Mapping this same processing technique to acoustic detection resulted in a substantial increase in detection capabilities, including in noisy environments. “Unauthorised drones pose distinctive threats to airports, individuals, and military bases,” says Finn. “It is therefore becoming ever-more critical for us to be able to detect specific locations of drones at long distances, using techniques that can pick up even the weakest signals. Our trials using the hoverfly-based algorithms show we can now do this.”The researchers specifically looked for patterns (narrowband) and/or general signals (broadband) to pick up drone acoustics at short to medium distances. The new bio-inspired processing technique improved detection ranges by between 30% and 49%.The findings have been reported in The Journal of the Acoustical Society of America. More

OTTO Lifter smart autonomous forklift.
OTTO Motors
A new robot forklift with some serious smarts is debuting at MODEX, the largest supply chain conference in the Americas. The conference is getting a lot more attention than usual amid ongoing global logistics pressures, and so is the case for a major automation overhaul in the logistics sector.

Innovation

Into that fray enters OTTO Motors, one in a growing number of robotics firms specializing in materials handling for a majorly strained global logistics paradigm. OTTO’s newest robot forklift, named Lifter, is billed as the smartest forklift in the market and adds to the company’s lineup of autonomous mobile robots (AMRs) that, taken together, can do much of the materials handling work of a major shipping and receiving warehouse.That’s a fairly astounding development given how little automation penetration there was in the sector until the 2010s.”Over the last decade, our AMRs have solved material handling challenges for some of the largest companies in the world, but our customers have made it clear that there was a missing member of the team — one that could pick up a pallet on its own,” said Chief Executive Officer and founder of OTTO Motors Matt Rendall. “We’ve answered that call with OTTO Lifter — your new forklift driver. We put years of autonomous driving experience into OTTO Lifter, making it the smartest forklift on the market.”What makes the new generation of AMRs like Lifter so useful is their ability to navigate in crowded and dynamic semi-structured environments, a feat made possible by a host of sensing technologies that have rapidly fallen in price over the past few years and intelligence such as dynamic path planning, lane tending, and intelligent pallet detection. Also: Giant 180-ton robot trucks are mining goldAccording to the company, all that adds up to a safer forklift, which is no small thing. Traditional forklifts account for 10% of all physical injuries in workplaces where they’re used, an average of 85 deaths annually, and nearly 35,000 serious injuries in the U.S. alone, according to OSHA statistics. 70% of all industrial accidents are caused by operator error, according to the National Safety Council, and 69% are working through fatigue. Remarkably, OTTO boasts three million hours of material handling driving without a single safety incident.Of course, if safety is a major concern, it’s probably trumped in the market by financial incentives. OTTO estimates that its forklift costs about $9 per working hour. That’s an important figure as it falls substantially below the federal minimum wage. OTTO, like most automation vendors, emphasizes that its robots are meant to work alongside people and augment human workforces — and that’s substantially true of most AMRs on the market today — but it’s a sure bet that operations managers will weigh that $9 per hour against the projected cost to run a manned forklift.All of this paints a pretty clear picture of how automation is beginning to tip the balance in a variety of industries, from logistics and manufacturing to spaces like construction. The blended era of humans and autonomous machines collaborating in the same spaces is very much here, and the next few years will see AMRs crop up in a variety of more generally visible spaces, including hospitals and restaurants.For now, if you want a pallet lifted, there’s a smart forklift ready and willing to work for cheap. More

Chipotle
We love our robots, and the quirkier the better. Doesn’t get much more smile-inducing than a new model from food service robotics pioneer Miso Robotics, which is designed to cook and freshly season tortilla chips to order.Miso is a growing tech company to watch, an early leader in the push to automate fast food, at least when it comes to the actual cooking part. Miso’s burger and chicken wing preparing robots (Flippy is the best known) tend a griddle just like a human chef, making them easy to integrate into existing kitchens, and have scored big votes of confidence from national chains like White Castle and Buffalo Wild Wings.Chipotle is the latest brand to dip a toe in automation. The chain is partnering with Miso on a robot named Chippy, an autonomous kitchen assistant that integrates culinary traditions with artificial intelligence to make tortilla chips. “We are always exploring opportunities to enhance our employee and guest experience. Our goal is to drive efficiencies through collaborative robotics that will enable Chipotle’s crew members to focus on other tasks in the restaurant,” said Curt Garner, Chief Technology Officer, Chipotle.One of the big draws for national brands at this early adoption stage is Miso’s strategy of customization. Chipotle’s culinary team guided Miso in tailoring its technology to replicate Chipotle’s exact recipe – using corn masa flour, water and sunflower oil – to cook chips that are indistinguishable from human made counterparts. Chipotle’s chips are finished with a dusting of seasoning and a hint of fresh lime juice.”Everyone loves finding a chip with a little more salt or an extra hint of lime,” said Nevielle Panthaky, Vice President of Culinary, Chipotle. “To ensure we didn’t lose the humanity behind our culinary experience, we trained Chippy extensively to ensure the output mirrored our current product, delivering some subtle variations in flavor that our guests expect.”That’s an interesting window into one of the pitfalls (and possible opportunities) of automation. Much like the unplanned artifacts and saturated colors of vintage film, something can be lost in the pursuit of technologically abetted perfection. Miso’s robot, then, was trained to embrace some measure of inconsistency.Chippy is currently being tested at the Chipotle Cultivate Center, Chipotle’s innovation hub in Irvine, Calif., and will be integrated into a Chipotle restaurant in Southern California later this year. More