Retrieval-augmented reasoning with lean language models
This technical report details a novel approach to combining reasoning and
retrieval augmented generation (RAG) within a single, lean language model
architecture. While existing RAG systems typically rely on large-scale models
and external APIs, our work addresses the increasing demand for performant and
privacy-preserving solutions deployable in resource-constrained or secure
environments. Building on recent developments in test-time scaling and
small-scale reasoning models, we develop a retrieval augmented conversational
agent capable of interpreting complex, domain-specific queries using a
lightweight backbone model. Our system integrates a dense retriever with
fine-tuned Qwen2.5-Instruct models, using synthetic query generation and
reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a
curated corpus, in this case, the NHS A-to-Z condition pages. We explore the
impact of summarisation-based document compression, synthetic data design, and
reasoning-aware fine-tuning on model performance. Evaluation against both
non-reasoning and general-purpose lean models demonstrates that our
domain-specific fine-tuning approach yields substantial gains in answer
accuracy and consistency, approaching frontier-level performance while
remaining feasible for local deployment. All implementation details and code
are publicly released to support reproducibility and adaptation across domains.
68a622599e4b49496aac68ab
Ryan Sze-Yin Chan
claimed_verified
2025-08-20T19:39:13.251Z
639080fed2cf01fdfe322a8e
https://cdn-avatars.huggingface.co/v1/production/uploads/1671645871849-639080fed2cf01fdfe322a8e.jpeg
Ryan Chan
user
rchan26
68a622599e4b49496aac68ac
Federico Nanni
68a622599e4b49496aac68ad
Tomas Lazauskas
68a622599e4b49496aac68ae
Rosie Wood
68a622599e4b49496aac68af
Penelope Yong
68a622599e4b49496aac68b0
Lionel Tarassenko
68a622599e4b49496aac68b1
Mark Girolami
68a622599e4b49496aac68b2
James Geddes
68a622599e4b49496aac68b3
Andrew Duncan
Retrieval-augmented reasoning with lean language models
This technical report details a novel approach to combining reasoning and
retrieval augmented generation (RAG) within a single, lean language model
architecture. While existing RAG systems typically rely on large-scale models
and external APIs, our work addresses the increasing demand for performant and
privacy-preserving solutions deployable in resource-constrained or secure
environments. Building on recent developments in test-time scaling and
small-scale reasoning models, we develop a retrieval augmented conversational
agent capable of interpreting complex, domain-specific queries using a
lightweight backbone model. Our system integrates a dense retriever with
fine-tuned Qwen2.5-Instruct models, using synthetic query generation and
reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a
curated corpus, in this case, the NHS A-to-Z condition pages. We explore the
impact of summarisation-based document compression, synthetic data design, and
reasoning-aware fine-tuning on model performance. Evaluation against both
non-reasoning and general-purpose lean models demonstrates that our
domain-specific fine-tuning approach yields substantial gains in answer
accuracy and consistency, approaching frontier-level performance while
remaining feasible for local deployment. All implementation details and code
are publicly released to support reproducibility and adaptation across domains.
A retrieval-augmented conversational agent using a lightweight model achieves high accuracy and consistency in domain-specific queries through fine-tuning and document compression.
reasoning and retrieval augmented generation (RAG) test-time scaling small-scale reasoning models dense retriever Qwen2.5-Instruct synthetic query generation reasoning traces summarisation-based document compression synthetic data design reasoning-aware fine-tuning domain-specific fine-tuning