Task-Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation
Yang Zhao, Pu Wang, Hao Frank Yang
Task-adaptive prompting and reasoning for Large Language Models (LLMs) are necessity in real-world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework to designing optimal prompts, efficient reasoning processes and providing enhanced casual process. EGO-Prompt begins with an general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that expert-defined SCGs may be partial or imperfect and that their optimal integration varies across LLMs, EGO-Prompt integrates a novel casual-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks and EGO-Prompt demonstrates clear improvements, achieving increases of 8% to 12% in F1 score compared to state-of-the-art methods, together with a refined domain SCG.