Agent Lightning: Train ANY AI Agents with Reinforcement Learning
We present Agent Lightning, a flexible and extensible framework that enables
Reinforcement Learning (RL)-based training of Large Language Models (LLMs) for
any AI agent. Unlike existing methods that tightly couple RL training with
agent or rely on sequence concatenation with masking, Agent Lightning achieves
complete decoupling between agent execution and training, allowing seamless
integration with existing agents developed via diverse ways (e.g., using
frameworks like LangChain, OpenAI Agents SDK, AutoGen, and building from
scratch) with almost ZERO code modifications. By formulating agent execution as
Markov decision process, we define an unified data interface and propose a
hierarchical RL algorithm, LightningRL, which contains a credit assignment
module, allowing us to decompose trajectories generated by ANY agents into
training transition. This enables RL to handle complex interaction logic, such
as multi-agent scenarios and dynamic workflows. For the system design, we
introduce a Training-Agent Disaggregation architecture, and brings agent
observability frameworks into agent runtime, providing a standardized agent
finetuning interface. Experiments across text-to-SQL, retrieval-augmented
generation, and math tool-use tasks demonstrate stable, continuous
improvements, showcasing the framework's potential for real-world agent
training and deployment.
68940ca8741a16f544fbce6b
Xufang Luo
68940ca8741a16f544fbce6c
Yuge Zhang
claimed_verified
2025-08-07T10:39:35.704Z
6466d323ac657f60661d2778
/avatars/62f70630cdf1c252b80b4d5eaa5a4150.svg
Yuge Zhang
user
ultmaster
68940ca8741a16f544fbce6d
Zhiyuan He
claimed_verified
2025-08-07T10:39:32.667Z
6455f5cababbbbd3486d6ee3
/avatars/b6c8f65fd2bef8a00aa3269856ea238e.svg
Zhiyuan He
user
hzy46
68940ca8741a16f544fbce6e
Zilong Wang
claimed_verified
2025-09-09T13:50:33.615Z
60a14314fc0249ded29992c2
/avatars/0c05eaea3362854f7af309b1edca949c.svg
WangZilong
user
dddwzl3703
68940ca8741a16f544fbce6f
Siyun Zhao
claimed_verified
2025-08-07T10:39:29.935Z
64c20f8264e3e59137d88742
https://cdn-avatars.huggingface.co/v1/production/uploads/64c20f8264e3e59137d88742/CB1Mi95QUhzkp8GlrDoln.jpeg
Siyun
user
SiyunZhao
68940ca8741a16f544fbce70
Dongsheng Li
68940ca8741a16f544fbce71
Luna K. Qiu
claimed_verified
2025-08-11T06:49:39.373Z
6892e3a533ebcd3f251f1a7f
/avatars/5389db68c2d83a3f20f322b52b123e32.svg
Luna Qiu
user
lunaqiu66
68940ca8741a16f544fbce72
Yuqing Yang
Agent Lightning: Train ANY AI Agents with Reinforcement Learning
We present Agent Lightning, a flexible and extensible framework that enables
Reinforcement Learning (RL)-based training of Large Language Models (LLMs) for
any AI agent. Unlike existing methods that tightly couple RL training with
agent or rely on sequence concatenation with masking, Agent Lightning achieves
complete decoupling between agent execution and training, allowing seamless
integration with existing agents developed via diverse ways (e.g., using
frameworks like LangChain, OpenAI Agents SDK, AutoGen, and building from
scratch) with almost ZERO code modifications. By formulating agent execution as
Markov decision process, we define an unified data interface and propose a
hierarchical RL algorithm, LightningRL, which contains a credit assignment
module, allowing us to decompose trajectories generated by ANY agents into
training transition. This enables RL to handle complex interaction logic, such
as multi-agent scenarios and dynamic workflows. For the system design, we
introduce a Training-Agent Disaggregation architecture, and brings agent
observability frameworks into agent runtime, providing a standardized agent
finetuning interface. Experiments across text-to-SQL, retrieval-augmented
generation, and math tool-use tasks demonstrate stable, continuous
improvements, showcasing the framework's potential for real-world agent
training and deployment.
Agent Lightning is a flexible RL framework for training LLMs in various agents, using a hierarchical RL algorithm and decoupling execution from training to handle complex interactions.
Reinforcement Learning Large Language Models Markov decision process hierarchical RL algorithm credit assignment module Training-Agent Disaggregation architecture agent observability frameworks text-to-SQL retrieval-augmented generation math tool-use tasks