Select to Know: An Internal-External Knowledge Self-Selection Framework
  for Domain-Specific Question Answering
Large Language Models (LLMs) perform well in general QA but often struggle in
domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces
external knowledge but suffers from hallucinations and latency due to noisy
retrievals. Continued pretraining internalizes domain knowledge but is costly
and lacks cross-domain flexibility. We attribute this challenge to the
long-tail distribution of domain knowledge, which leaves partial yet useful
internal knowledge underutilized. We further argue that knowledge acquisition
should be progressive, mirroring human learning: first understanding concepts,
then applying them to complex reasoning. To address this, we propose Selct2Know
(S2K), a cost-effective framework that internalizes domain knowledge through an
internal-external knowledge self-selection strategy and selective supervised
fine-tuning. We also introduce a structured reasoning data generation pipeline
and integrate GRPO to enhance reasoning ability. Experiments on medical, legal,
and financial QA benchmarks show that S2K consistently outperforms existing
methods and matches domain-pretrained LLMs with significantly lower cost.
68aecc0f5f5ef03314d8c432
Bolei He
claimed_verified
2025-08-27T14:59:41.480Z
68aefd6482a3030535876c27
/avatars/8df724044e3087eea3628a8927b4da69.svg
HeBolei
user
HeBolei
68aecc0f5f5ef03314d8c433
Xinran He
68aecc0f5f5ef03314d8c434
Run Shao
claimed_verified
2025-08-28T08:55:54.208Z
630cc0ab8e3ff0c72329c364
/avatars/c3e0e071a9efa6061388ad4c66136030.svg
RunShao
user
ShaoRun
68aecc0f5f5ef03314d8c435
Shanfu Shu
68aecc0f5f5ef03314d8c436
Xianwei Xue
68aecc0f5f5ef03314d8c437
Mingquan Cheng
68aecc0f5f5ef03314d8c438
Haifeng Li
68aecc0f5f5ef03314d8c439
Zhenhua Ling
Select to Know: An Internal-External Knowledge Self-Selection Framework
  for Domain-Specific Question Answering
Large Language Models (LLMs) perform well in general QA but often struggle in
domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces
external knowledge but suffers from hallucinations and latency due to noisy
retrievals. Continued pretraining internalizes domain knowledge but is costly
and lacks cross-domain flexibility. We attribute this challenge to the
long-tail distribution of domain knowledge, which leaves partial yet useful
internal knowledge underutilized. We further argue that knowledge acquisition
should be progressive, mirroring human learning: first understanding concepts,
then applying them to complex reasoning. To address this, we propose Selct2Know
(S2K), a cost-effective framework that internalizes domain knowledge through an
internal-external knowledge self-selection strategy and selective supervised
fine-tuning. We also introduce a structured reasoning data generation pipeline
and integrate GRPO to enhance reasoning ability. Experiments on medical, legal,
and financial QA benchmarks show that S2K consistently outperforms existing
methods and matches domain-pretrained LLMs with significantly lower cost.
Selct2Know (S2K) enhances domain-specific QA by progressively internalizing and selecting knowledge through a cost-effective framework, outperforming existing methods with lower costs.
Large Language Models Retrieval-Augmented Generation RAG domain knowledge long-tail distribution internal-external knowledge self-selection selective supervised fine-tuning structured reasoning data generation GRPO medical QA legal QA financial QA domain-pretrained LLMs