Sketch2Anim: Towards Transferring Sketch Storyboards into 3D Animation

Lei Zhong1, Chuan Guo2, Yiming Xie3, Jiawei Wang1, Changjian Li1
1University of Edinburgh, 2Snap Research, 3Northeastern University
ACM Transactions on Graphics (Proc. SIGGRAPH 2025)
paper Code

Sketch2Anim can transfer sketch storyboards into 3D animation.

Abstract

Storyboarding is a common practice in 3D animation, where animators use 2D sketches as references to manually create motions—a process that is both time-consuming and requires significant expertise. To automate this workflow, we introduce Sketch2Anim, a novel framework that translates 2D storyboard sketches into 3D animations through conditional motion synthesis. Sketch2Anim features a multi-conditional motion generator, leveraging 3D keyposes, joint trajectories, and action words for precise motion control. To bridge the 2D-3D domain gap, we also propose a neural mapper that aligns 2D sketches with 3D motion constraints in a shared embedding space, enabling direct 2D-to-3D animation control. Our method produces high-quality 3D motions and naturally supports interactive editing. Extensive experiments and user studies confirm the effectiveness of our approach.

Method

Our pipeline consists of two core modules - the multi-conditional motion generator and the 2D-3D neural mapper. Instead of directly lifting the 2D keypose and trajectory into their 3D counterparts, we train a neural mapper dedicated to aligning the two domains in the embedding space. Because of this shared embedding, it enables the employment of more informative and precise 3D keyposes and trajectories as the motion conditions in the motion generator, while exploiting the 2D keypose and trajectory detected from the sketch storyboard at inference time.



Traditional Animation Work


Animators first envision key poses to represent the intended motion. These key poses are imported into Blender as visual references, where predefined 3D joints are manually adjusted to match them and align with the desired trajectory and action description.

.




Gallery Results




Comparing to Other Baseline Methods


Since no existing method directly addresses our conditional motion generation task, we developed three baseline approaches for comparison:




Real-world Storyboard


Future Works

Speed Control

Storyboard artists often use speed lines to express motion intensity. We plan to incorporate a module to detect and interpret these over-sketched lines to control animation timing more precisely.

Scene Recovery

Industrial storyboards often include scene elements such as props, furniture, or environments. We plan to integrate scene reconstruction and enforce spatial constraints between the environment and the character animation.

Concurrent Works

  • (CVPR 2025) StickMotion: Generating 3D Human Motions by Drawing a Stickman
  • (TVCG 2025) Doodle Your Motion: Sketch-Guided Human Motion Generation

    • BibTeX

    
      @Article{Zhong:2025:Sketch2Anim, 
          Title = {Sketch2Anim: Towards Transferring Sketch Storyboards into 3D
          Animation}, 
          Author = {Lei Zhong, Chuan Guo, Yiming Xie, Jiawei Wang and Changjian Li}, 
          Journal = {ACM Transaction on Graphics (TOG)},
          volume={44},
          number={4},
          pages={1--15},
          Year = {2025}, 
          Publisher = {ACM New York, NY, USA} 
      }