Combinatorial Creativity: A New Frontier in Generalization Abilities
Artificial intelligence (AI) systems, and Large Language Models (LLMs) in
particular, are increasingly employed for creative tasks like scientific idea
generation, constituting a form of generalization from training data
unaddressed by existing conceptual frameworks. Despite its similarities to
compositional generalization (CG), combinatorial creativity (CC) is an
open-ended ability. Instead of evaluating for accuracy or correctness against
fixed targets, which would contradict the open-ended nature of CC, we propose a
theoretical framework and algorithmic task for evaluating outputs by their
degrees of novelty and utility. From here, we make several important empirical
contributions: (1) We obtain the first insights into the scaling behavior of
creativity for LLMs. (2) We discover that, for fixed compute budgets, there
exist optimal model depths and widths for creative ability. (3) We find that
the ideation-execution gap, whereby LLMs excel at generating novel scientific
ideas but struggle to ensure their practical feasibility, may be explained by a
more fundamental novelty-utility tradeoff characteristic of creativity
algorithms in general. Importantly, this tradeoff remains persistent even at
scale, casting doubt on the long-term creative potential of LLMs in their
current form. Together, our conceptual framework and empirical findings provide
a foundation for understanding and improving creativity in modern AI models,
bridging the gap between human and machine intelligence.
68dbf51f4159d1f2418f9815
Samuel Schapiro
68dbf51f4159d1f2418f9816
Sumuk Shashidhar
68dbf51f4159d1f2418f9817
Alexi Gladstone
68dbf51f4159d1f2418f9818
Jonah Black
68dbf51f4159d1f2418f9819
Royce Moon
claimed_verified
2025-10-01T10:23:35.149Z
66085f22972496838af1e0c6
https://cdn-avatars.huggingface.co/v1/production/uploads/66085f22972496838af1e0c6/Q4acHo7ajM1yXyi16qYJt.jpeg
Royce Moon
user
compulsi0n
68dbf51f4159d1f2418f981a
Dilek Hakkani-Tur
68dbf51f4159d1f2418f981b
Lav R. Varshney
Combinatorial Creativity: A New Frontier in Generalization Abilities
Artificial intelligence (AI) systems, and Large Language Models (LLMs) in
particular, are increasingly employed for creative tasks like scientific idea
generation, constituting a form of generalization from training data
unaddressed by existing conceptual frameworks. Despite its similarities to
compositional generalization (CG), combinatorial creativity (CC) is an
open-ended ability. Instead of evaluating for accuracy or correctness against
fixed targets, which would contradict the open-ended nature of CC, we propose a
theoretical framework and algorithmic task for evaluating outputs by their
degrees of novelty and utility. From here, we make several important empirical
contributions: (1) We obtain the first insights into the scaling behavior of
creativity for LLMs. (2) We discover that, for fixed compute budgets, there
exist optimal model depths and widths for creative ability. (3) We find that
the ideation-execution gap, whereby LLMs excel at generating novel scientific
ideas but struggle to ensure their practical feasibility, may be explained by a
more fundamental novelty-utility tradeoff characteristic of creativity
algorithms in general. Importantly, this tradeoff remains persistent even at
scale, casting doubt on the long-term creative potential of LLMs in their
current form. Together, our conceptual framework and empirical findings provide
a foundation for understanding and improving creativity in modern AI models,
bridging the gap between human and machine intelligence.
Research explores creativity in LLMs, revealing scaling behaviors, optimal model dimensions, and a persistent novelty-utility tradeoff affecting their creative potential.
Large Language Models compositional generalization combinatorial creativity novelty utility model depths model widths ideation-execution gap creativity algorithms