Empower Large Language Model to Perform Better on Industrial
  Domain-Specific Question Answering
Large Language Model (LLM) has gained popularity and achieved remarkable
results in open-domain tasks, but its performance in real industrial
domain-specific scenarios is average since there is no specific knowledge in
it. This issue has attracted widespread attention, but there are few relevant
benchmarks available. In this paper, we provide a benchmark Question Answering
(QA) dataset named MSQA, which is about Microsoft products and IT technical
problems encountered by customers. This dataset contains industry
cloud-specific QA knowledge, which is not available for general LLM, so it is
well suited for evaluating methods aimed at improving domain-specific
capabilities of LLM. In addition, we propose a new model interaction paradigm
that can empower LLM to achieve better performance on domain-specific tasks
where it is not proficient. Extensive experiments demonstrate that the approach
following our model fusion framework outperforms the commonly used LLM with
retrieval methods.
646ad1a8c37ca1e1230c8a23
Zezhong Wang
admin_assigned
2023-05-26T09:00:36.444Z
642d6b792320338df27dd884
/avatars/f76290e059a0e15cee1d755bc1e8f0ba.svg
Zezhong Wnag
user
AwesomeW
646ad1a8c37ca1e1230c8a24
Fangkai Yang
claimed_verified
2025-02-27T09:19:06.996Z
669dcf6200970c3b27aafa5d
/avatars/bb9ed5ff86326fdaeb184c6b0e40f74f.svg
kaikai yang
user
keanudicap
646ad1a8c37ca1e1230c8a25
Pu Zhao
646ad1a8c37ca1e1230c8a26
Lu Wang
646ad1a8c37ca1e1230c8a27
Jue Zhang
646ad1a8c37ca1e1230c8a28
Mohit Garg
admin_assigned
2023-05-26T09:01:17.361Z
643425c8ea46c00990480f60
/avatars/012e77ac7e5d755050ea065b723a6b60.svg
Mohit Garg
user
mohitgargai
646ad1a8c37ca1e1230c8a29
Qingwei Lin
646ad1a8c37ca1e1230c8a2a
Dongmei Zhang
Empower Large Language Model to Perform Better on Industrial
  Domain-Specific Question Answering
Large Language Model (LLM) has gained popularity and achieved remarkable
results in open-domain tasks, but its performance in real industrial
domain-specific scenarios is average since there is no specific knowledge in
it. This issue has attracted widespread attention, but there are few relevant
benchmarks available. In this paper, we provide a benchmark Question Answering
(QA) dataset named MSQA, which is about Microsoft products and IT technical
problems encountered by customers. This dataset contains industry
cloud-specific QA knowledge, which is not available for general LLM, so it is
well suited for evaluating methods aimed at improving domain-specific
capabilities of LLM. In addition, we propose a new model interaction paradigm
that can empower LLM to achieve better performance on domain-specific tasks
where it is not proficient. Extensive experiments demonstrate that the approach
following our model fusion framework outperforms the commonly used LLM with
retrieval methods.
Large Language Model (LLM) Question Answering (QA) domain-specific scenarios benchmark MSQA cloud-specific QA knowledge model fusion framework retrieval methods