A new AI technology from MIT called VideoCAD aims to turn back-of-the-envelope sketches into real products faster by making computer aided design (CAD) more accessible to everyone. CAD is a cornerstone of modern product development and is a core technology in high growth areas like the creation of digital twins, electric vehicles, airplane construction and renewable energy projects.
VideoCAD is likely to have a major impact on an $11 billion CAD industry. Roughly 438,000 people in the workplace are graduates of CAD drafting and design programs in the U.S as of 2023, according to Data USA.
CAD is known as a fiendishly complicated tech that takes years to master. VideoCAD drastically lowers the learning curve by using AI to quickly turn a sketch into a 3D version. Making this possible is a dataset developed by MIT that contains 41,000 annotated examples of how 3D models are built into CAD software. By learning from these videos, which illustrate how different shapes and objects are constructed step-by-step, the AI system can now operate CAD software much like a human user. The videos are basically real-time walk-throughs of CAD actions originally carried out by humans.
MIT frames VideoCAD as an AI-enabled “CAD co-pilot” that not only creates 3D versions of a design but also suggests next steps or automatically carries out build sequences that would be tedious and time-consuming to manually click through. The MIT team says this is the first time that user interface (UI) interactions have been modeled for precision engineering. Behavior cloning from video, where a UI agent learns from human demonstrations, has shown promise in robotics and gaming but its application to UI software had been largely unexplored due to the lack of large-scale video-annotated datasets. VideoCAD also has implications for computer vision due to its 3D reasoning capability.
VideoCAD “is significant because it lowers the barrier to entry for design, helping people without years of CAD training to create 2D models more easily and tap into their creativity,” says Faez Ahmed, associate professor of mechanical engineering at MIT who also is part of the team that created VideoCAD.
The VideoCAD dataset includes everything from simple brackets to house design. The MIT team is training VideoCAD on more complex shapes and envisions its use in a wide range of fields. VideoCAD currently is limited to use on a single platform called Onshape so its ability to work on other platforms remains to be seen.
“VideoCAD is a valuable first step toward AI assistants that help onboard new users automate the repetitive modeling work that follows familiar patterns,” says Medhi Ataei, a senior research fellow at software developer Autodesk Research, who was not involved in the MIT project. “This is an early foundation and I would be excited to see successors that span multiple CAD systems, richer operations like assemblies and constraints and more realistic, messy human workflows.”
