“3D Sketch” Your Ideas and Bring Them to Life, Instantly!

A novel 3D sketching system for rapidly designing articulated 3D concepts with intuitive pen and multi-touch gestures bridges the gap between 2D sketching and 3D modeling and is generally applicable in film, animation, game, and product design

Article | Spring 2023

Foldable drones, transforming vehicles, and multi-legged robots from sci-fi movies are now becoming commonplace thanks to technological progress. However, designing them remains a difficult challenge even for skilled experts, because complex design decisions must be made regarding not only their form, but also the structure, poses, and motions, which are interdependent on one another.

Creating a 3D concept comprising of multiple moving parts connected by different types of joints using a traditional 3D CAD tool can be a painstaking and time-consuming process. This presents a major source of friction during the early stage of design, in which the designers try out and discard many ideas quickly in order to explore a wide range of possibilities in a short amount of time.

A research team led by Professor Seok-Hyung Bae in the Department of Industrial Design at KAIST has focused on designers’ freehand sketches drawn with pen on paper that serve as the starting point for virtually all design projects. This led them to develop “3D sketching” systems for generating desired 3D curves from rough but expressive 2D strokes drawn with a stylus pen on a digital tablet.

Their latest 3D sketching system helps designers bring their 3D sketches to life almost instantly. The team successfully designed and implemented an intuitive set of multi-touch gestures allowing designers to manipulate the 3D sketches they are working on with their fingers as if they were playing with toys (Figure 1) and animate the 3D sketches in no time (Figure 2).

 

 

The major findings of their work were published under the title “Rapid Design of Articulated Objects” in ACM Transactions on Graphics (impact factor: 7.403), the top international journal in the field of computer graphics, and presented at ACM SIGGRAPH 2022 (h5-index: 103), the world’s largest international academic conference in the field, which was held in Vancouver, Canada with Joon Hyub Lee, a Ph.D. student of the Department of Industrial Design as the first author.

The ACM SIGGRAPH 2022 conference was reportedly attended by over 10,000 participants including researchers, artists, and developers from world-renowned universities; film, animation, and game studios, such as Marvel, Pixar, and Blizzard; high-tech manufacturers, such as Lockheed Martin and Boston Dynamics; and metaverse platform companies, such as Meta and Roblox.

 

The team was also invited to present their technical paper as a demo and a special talk at the Emerging Technologies program at ACM SIGGRAPH 2022 as one of the top-three impactful technologies (Figure 3, 4). The live performance, in which Hanbit Kim, a Ph.D. student of the Department of Industrial Design at KAIST and a co-author, sketched and animated a sophisticated animal-shaped robot from scratch in a matter of a few minutes (Figure 3), wowed the audience and won the Honorable Mention Award from the jury.

Edwin Catmull, the co-founder of Pixar and a keynote speaker at the SIGGRAPH conference, praised the team’s research on 3D sketching as “really excellent work” and “a kind of tool that would be useful to Pixar’s creative model designers.”

This technology, which became virally popular in Japan after being featured in an online IT media outlet and attracting more than 600K views, received a special award from the Digital Content Association of Japan (DCAJ) and was invited and exhibited at Tokyo as a part of Inter BEE 2022, the largest broadcasting and media expo in Japan.

“The more we come to understand how designers think and work, the more effective design tools can be built around that understanding,” said Professor Bae, explaining that “the key is to integrate various algorithms into a harmonious system as intuitive interactions.” He added that “this work wouldn’t have been possible if it weren’t for the convergent research environment cultivated by the Department of Industrial Design at KAIST, in which all students see themselves not only as aspiring creative designers, but also as practical engineers.”

By enabling designers to produce highly expressive animated 3D concepts far more quickly and easily in comparison to using existing methods, this new tool is expected to revolutionize design practices and processes in the content creation, manufacturing, and metaverse-related industries.

This research was funded by the Ministry of Science and ICT, and the National Research Foundation of Korea.

More info: https://sketch.kaist.ac.kr/publications/2022_siggraph_rapid_design
Video: https://www.youtube.com/watch?v=rsBl0QvSDqI