Large Language Models as Planning Domain Generators
Developing domain models is one of the few remaining places that require
manual human labor in AI planning. Thus, in order to make planning more
accessible, it is desirable to automate the process of domain model generation.
To this end, we investigate if large language models (LLMs) can be used to
generate planning domain models from simple textual descriptions. Specifically,
we introduce a framework for automated evaluation of LLM-generated domains by
comparing the sets of plans for domain instances. Finally, we perform an
empirical analysis of 7 large language models, including coding and chat models
across 9 different planning domains, and under three classes of natural
language domain descriptions. Our results indicate that LLMs, particularly
those with high parameter counts, exhibit a moderate level of proficiency in
generating correct planning domains from natural language descriptions. Our
code is available at https://github.com/IBM/NL2PDDL.
6642e7a724555ed542ec83fd
James Oswald
extracted_confirmed
2025-02-16T22:58:59.792Z
648b152a4cb9bdfc9fe144aa
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/jaiMWXW9_6AEiDhgbIb7T.png
James Oswald
user
James-Oswald
6642e7a724555ed542ec83fe
Kavitha Srinivas
6642e7a724555ed542ec83ff
Harsha Kokel
claimed_verified
2025-04-02T08:57:30.109Z
648b67b2355113a5fba0ff31
/avatars/8ea58a2e3f261d20b33c033992bc7f52.svg
Harsha Kokel
user
harshakokel
6642e7a724555ed542ec8400
Junkyu Lee
6642e7a724555ed542ec8401
Michael Katz
claimed_verified
2025-03-31T15:15:06.023Z
65b2d595b389ca2de1945d59
/avatars/768bb50d7fd0d3cb416fe63645b3a658.svg
Michael Katz
user
ctpelok
6642e7a724555ed542ec8402
Shirin Sohrabi
Large Language Models as Planning Domain Generators
Developing domain models is one of the few remaining places that require
manual human labor in AI planning. Thus, in order to make planning more
accessible, it is desirable to automate the process of domain model generation.
To this end, we investigate if large language models (LLMs) can be used to
generate planning domain models from simple textual descriptions. Specifically,
we introduce a framework for automated evaluation of LLM-generated domains by
comparing the sets of plans for domain instances. Finally, we perform an
empirical analysis of 7 large language models, including coding and chat models
across 9 different planning domains, and under three classes of natural
language domain descriptions. Our results indicate that LLMs, particularly
those with high parameter counts, exhibit a moderate level of proficiency in
generating correct planning domains from natural language descriptions. Our
code is available at https://github.com/IBM/NL2PDDL.
Large language models show moderate proficiency in generating automated planning domain models from natural language descriptions.
domain models AI planning large language models (LLMs) automated evaluation planning domain models natural language descriptions