cadrille: Multi-modal CAD Reconstruction with Online Reinforcement
  Learning
Computer-Aided Design (CAD) plays a central role in engineering and
manufacturing, making it possible to create precise and editable 3D models.
Using a variety of sensor or user-provided data as inputs for CAD
reconstruction can democratize access to design applications. However, existing
methods typically focus on a single input modality, such as point clouds,
images, or text, which limits their generalizability and robustness. Leveraging
recent advances in vision-language models (VLM), we propose a multi-modal CAD
reconstruction model that simultaneously processes all three input modalities.
Inspired by large language model (LLM) training paradigms, we adopt a two-stage
pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated
data, followed by reinforcement learning (RL) fine-tuning using online
feedback, obtained programatically. Furthermore, we are the first to explore RL
fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such
as Group Relative Preference Optimization (GRPO) outperform offline
alternatives. In the DeepCAD benchmark, our SFT model outperforms existing
single-modal approaches in all three input modalities simultaneously. More
importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three
challenging datasets, including a real-world one.
6839533ae7922379b361b3cb
Maksim Kolodiazhnyi
claimed_verified
2025-06-02T07:46:40.766Z
665b10fb270e47e678f2ddf1
/avatars/1bc7a9211acf767f7bfca998c24315a0.svg
max
user
maksimko123
6839533ae7922379b361b3cc
Denis Tarasov
claimed_verified
2025-06-02T07:46:43.447Z
63f613cab29015adc3393d60
/avatars/d8402762bf8f96270a729b048f4bc557.svg
Denis Tarasov
user
Adagrad
6839533ae7922379b361b3cd
Dmitrii Zhemchuzhnikov
extracted_pending
2025-05-30T06:42:03.218Z
67d5a331eab66ce9cb01bae4
/avatars/3ed437c874889e8a6db66c3ef88a60c0.svg
DMITRII ZHEMCHUZHNIKOV
user
zhemchuzhnikov
6839533ae7922379b361b3ce
Alexander Nikulin
claimed_verified
2025-06-02T07:46:46.258Z
636824b2856a92b278936457
https://cdn-avatars.huggingface.co/v1/production/uploads/636824b2856a92b278936457/fzQgPTUjPKg1XZ73L3gzw.jpeg
Alexander Nikulin
user
Howuhh
6839533ae7922379b361b3cf
Ilya Zisman
6839533ae7922379b361b3d0
Anna Vorontsova
6839533ae7922379b361b3d1
Anton Konushin
6839533ae7922379b361b3d2
Vladislav Kurenkov
6839533ae7922379b361b3d3
Danila Rukhovich
claimed_verified
2025-07-15T19:12:12.152Z
6394434e872e49c02f5934c3
/avatars/d77228347a45868e7984607b18a98c29.svg
Danila Rukhovich
user
filapro
cadrille: Multi-modal CAD Reconstruction with Online Reinforcement
  Learning
Computer-Aided Design (CAD) plays a central role in engineering and
manufacturing, making it possible to create precise and editable 3D models.
Using a variety of sensor or user-provided data as inputs for CAD
reconstruction can democratize access to design applications. However, existing
methods typically focus on a single input modality, such as point clouds,
images, or text, which limits their generalizability and robustness. Leveraging
recent advances in vision-language models (VLM), we propose a multi-modal CAD
reconstruction model that simultaneously processes all three input modalities.
Inspired by large language model (LLM) training paradigms, we adopt a two-stage
pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated
data, followed by reinforcement learning (RL) fine-tuning using online
feedback, obtained programatically. Furthermore, we are the first to explore RL
fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such
as Group Relative Preference Optimization (GRPO) outperform offline
alternatives. In the DeepCAD benchmark, our SFT model outperforms existing
single-modal approaches in all three input modalities simultaneously. More
importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three
challenging datasets, including a real-world one.
A multi-modal CAD reconstruction model leveraging vision-language models and reinforcement learning achieves state-of-the-art performance across various datasets, including real-world ones.
vision-language models multi-modal large language model supervised fine-tuning reinforcement learning Group Relative Preference Optimization DeepCAD benchmark