GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong
  Prompt Optimizers
The effectiveness of large language models (LLMs) is closely tied to the
design of prompts, making prompt optimization essential for enhancing their
performance across a wide range of tasks. Many existing approaches to
automating prompt engineering rely exclusively on textual feedback, refining
prompts based solely on inference errors identified by large, computationally
expensive LLMs. Unfortunately, smaller models struggle to generate high-quality
feedback, resulting in complete dependence on large LLM judgment. Moreover,
these methods fail to leverage more direct and finer-grained information, such
as gradients, due to operating purely in text space. To this end, we introduce
GReaTer, a novel prompt optimization technique that directly incorporates
gradient information over task-specific reasoning. By utilizing task loss
gradients, GReaTer enables self-optimization of prompts for open-source,
lightweight language models without the need for costly closed-source LLMs.
This allows high-performance prompt optimization without dependence on massive
LLMs, closing the gap between smaller models and the sophisticated reasoning
often needed for prompt refinement. Extensive evaluations across diverse
reasoning tasks including BBH, GSM8k, and FOLIO demonstrate that GReaTer
consistently outperforms previous state-of-the-art prompt optimization methods,
even those reliant on powerful LLMs. Additionally, GReaTer-optimized prompts
frequently exhibit better transferability and, in some cases, boost task
performance to levels comparable to or surpassing those achieved by larger
language models, highlighting the effectiveness of prompt optimization guided
by gradients over reasoning. Code of GReaTer is available at
https://github.com/psunlpgroup/GreaTer.
67607fea899f4731eb7dc446
Sarkar Snigdha Sarathi Das
67607fea899f4731eb7dc447
Ryo Kamoi
claimed_verified
2024-12-17T08:04:07.183Z
60591896102f61b42f65ae37
https://cdn-avatars.huggingface.co/v1/production/uploads/60591896102f61b42f65ae37/PSHcLoEeNnwvzAPOd6-6i.jpeg
Ryo Kamoi
user
ryokamoi
67607fea899f4731eb7dc448
Bo Pang
67607fea899f4731eb7dc449
Yusen Zhang
67607fea899f4731eb7dc44a
Caiming Xiong
67607fea899f4731eb7dc44b
Rui Zhang
GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong
  Prompt Optimizers
The effectiveness of large language models (LLMs) is closely tied to the
design of prompts, making prompt optimization essential for enhancing their
performance across a wide range of tasks. Many existing approaches to
automating prompt engineering rely exclusively on textual feedback, refining
prompts based solely on inference errors identified by large, computationally
expensive LLMs. Unfortunately, smaller models struggle to generate high-quality
feedback, resulting in complete dependence on large LLM judgment. Moreover,
these methods fail to leverage more direct and finer-grained information, such
as gradients, due to operating purely in text space. To this end, we introduce
GReaTer, a novel prompt optimization technique that directly incorporates
gradient information over task-specific reasoning. By utilizing task loss
gradients, GReaTer enables self-optimization of prompts for open-source,
lightweight language models without the need for costly closed-source LLMs.
This allows high-performance prompt optimization without dependence on massive
LLMs, closing the gap between smaller models and the sophisticated reasoning
often needed for prompt refinement. Extensive evaluations across diverse
reasoning tasks including BBH, GSM8k, and FOLIO demonstrate that GReaTer
consistently outperforms previous state-of-the-art prompt optimization methods,
even those reliant on powerful LLMs. Additionally, GReaTer-optimized prompts
frequently exhibit better transferability and, in some cases, boost task
performance to levels comparable to or surpassing those achieved by larger
language models, highlighting the effectiveness of prompt optimization guided
by gradients over reasoning. Code of GReaTer is available at
https://github.com/psunlpgroup/GreaTer.
GReaTer optimizes prompts for lightweight language models using task-specific gradient information, achieving performance comparable to larger models.
large language models prompt optimization textual feedback inference errors gradients task loss gradients open-source language models prompt self-optimization BBH GSM8k FOLIO task performance transferability