SimpleFold: Folding Proteins is Simpler than You Think
Protein folding models have achieved groundbreaking results typically via a
combination of integrating domain knowledge into the architectural blocks and
training pipelines. Nonetheless, given the success of generative models across
different but related problems, it is natural to question whether these
architectural designs are a necessary condition to build performant models. In
this paper, we introduce SimpleFold, the first flow-matching based protein
folding model that solely uses general purpose transformer blocks. Protein
folding models typically employ computationally expensive modules involving
triangular updates, explicit pair representations or multiple training
objectives curated for this specific domain. Instead, SimpleFold employs
standard transformer blocks with adaptive layers and is trained via a
generative flow-matching objective with an additional structural term. We scale
SimpleFold to 3B parameters and train it on approximately 9M distilled protein
structures together with experimental PDB data. On standard folding benchmarks,
SimpleFold-3B achieves competitive performance compared to state-of-the-art
baselines, in addition SimpleFold demonstrates strong performance in ensemble
prediction which is typically difficult for models trained via deterministic
reconstruction objectives. Due to its general-purpose architecture, SimpleFold
shows efficiency in deployment and inference on consumer-level hardware.
SimpleFold challenges the reliance on complex domain-specific architectures
designs in protein folding, opening up an alternative design space for future
progress.
68d43e7536950a9dff156787
Yuyang Wang
68d43e7536950a9dff156788
Jiarui Lu
68d43e7536950a9dff156789
Navdeep Jaitly
68d43e7536950a9dff15678a
Josh Susskind
68d43e7536950a9dff15678b
Miguel Angel Bautista
SimpleFold: Folding Proteins is Simpler than You Think
Protein folding models have achieved groundbreaking results typically via a
combination of integrating domain knowledge into the architectural blocks and
training pipelines. Nonetheless, given the success of generative models across
different but related problems, it is natural to question whether these
architectural designs are a necessary condition to build performant models. In
this paper, we introduce SimpleFold, the first flow-matching based protein
folding model that solely uses general purpose transformer blocks. Protein
folding models typically employ computationally expensive modules involving
triangular updates, explicit pair representations or multiple training
objectives curated for this specific domain. Instead, SimpleFold employs
standard transformer blocks with adaptive layers and is trained via a
generative flow-matching objective with an additional structural term. We scale
SimpleFold to 3B parameters and train it on approximately 9M distilled protein
structures together with experimental PDB data. On standard folding benchmarks,
SimpleFold-3B achieves competitive performance compared to state-of-the-art
baselines, in addition SimpleFold demonstrates strong performance in ensemble
prediction which is typically difficult for models trained via deterministic
reconstruction objectives. Due to its general-purpose architecture, SimpleFold
shows efficiency in deployment and inference on consumer-level hardware.
SimpleFold challenges the reliance on complex domain-specific architectures
designs in protein folding, opening up an alternative design space for future
progress.
SimpleFold, a flow-matching based protein folding model using general-purpose transformer blocks, achieves competitive performance with reduced complexity and improved efficiency.
flow-matching transformer blocks adaptive layers generative flow-matching objective structural term protein folding triangular updates pair representations multiple training objectives ensemble prediction consumer-level hardware