Graph Diffusion Transformers are In-Context Molecular Designers
In-context learning allows large models to adapt to new tasks from a few
demonstrations, but it has shown limited success in molecular design. Existing
databases such as ChEMBL contain molecular properties spanning millions of
biological assays, yet labeled data for each property remain scarce. To address
this limitation, we introduce demonstration-conditioned diffusion models
(DemoDiff), which define task contexts using a small set of molecule-score
examples instead of text descriptions. These demonstrations guide a denoising
Transformer to generate molecules aligned with target properties. For scalable
pretraining, we develop a new molecular tokenizer with Node Pair Encoding that
represents molecules at the motif level, requiring 5.5times fewer nodes. We
curate a dataset containing millions of context tasks from multiple sources
covering both drugs and materials, and pretrain a 0.7-billion-parameter model
on it. Across 33 design tasks in six categories, DemoDiff matches or surpasses
language models 100-1000times larger and achieves an average rank of 3.63
compared to 5.25-10.20 for domain-specific approaches. These results position
DemoDiff as a molecular foundation model for in-context molecular design. Our
code is available at https://github.com/liugangcode/DemoDiff.
68edad48de1fee572713a720
Gang Liu
68edad48de1fee572713a721
Jie Chen
68edad48de1fee572713a722
Yihan Zhu
68edad48de1fee572713a723
Michael Sun
68edad48de1fee572713a724
Tengfei Luo
68edad48de1fee572713a725
Nitesh V Chawla
68edad48de1fee572713a726
Meng Jiang
Graph Diffusion Transformers are In-Context Molecular Designers
In-context learning allows large models to adapt to new tasks from a few
demonstrations, but it has shown limited success in molecular design. Existing
databases such as ChEMBL contain molecular properties spanning millions of
biological assays, yet labeled data for each property remain scarce. To address
this limitation, we introduce demonstration-conditioned diffusion models
(DemoDiff), which define task contexts using a small set of molecule-score
examples instead of text descriptions. These demonstrations guide a denoising
Transformer to generate molecules aligned with target properties. For scalable
pretraining, we develop a new molecular tokenizer with Node Pair Encoding that
represents molecules at the motif level, requiring 5.5times fewer nodes. We
curate a dataset containing millions of context tasks from multiple sources
covering both drugs and materials, and pretrain a 0.7-billion-parameter model
on it. Across 33 design tasks in six categories, DemoDiff matches or surpasses
language models 100-1000times larger and achieves an average rank of 3.63
compared to 5.25-10.20 for domain-specific approaches. These results position
DemoDiff as a molecular foundation model for in-context molecular design. Our
code is available at https://github.com/liugangcode/DemoDiff.
DemoDiff, a demonstration-conditioned diffusion model, uses molecule-score examples to guide a denoising Transformer for molecular design, outperforming larger language models and domain-specific approaches.
in-context learning diffusion models denoising Transformer molecular tokenizer Node Pair Encoding molecular design pretraining foundation model