Empower Large Language Model to Perform Better on Industrial
  Domain-Specific Question Answering
Large Language Model (LLM) has gained popularity and achieved remarkable
results in open-domain tasks, but its performance in real industrial
domain-specific scenarios is average since there is no specific knowledge in
it. This issue has attracted widespread attention, but there are few relevant
benchmarks available. In this paper, we provide a benchmark Question Answering
(QA) dataset named MSQA, which is about Microsoft products and IT technical
problems encountered by customers. This dataset contains industry
cloud-specific QA knowledge, which is not available for general LLM, so it is
well suited for evaluating methods aimed at improving domain-specific
capabilities of LLM. In addition, we propose a new model interaction paradigm
that can empower LLM to achieve better performance on domain-specific tasks
where it is not proficient. Extensive experiments demonstrate that the approach
following our model fusion framework outperforms the commonly used LLM with
retrieval methods.
646ad1a8c37ca1e1230c8a23
Zezhong Wang
admin_assigned
2023-05-26T09:00:36.444Z
642d6b792320338df27dd884
/avatars/f76290e059a0e15cee1d755bc1e8f0ba.svg
Zezhong Wnag
user
AwesomeW
646ad1a8c37ca1e1230c8a24
Fangkai Yang
claimed_verified
2025-02-27T09:19:06.996Z
669dcf6200970c3b27aafa5d
/avatars/bb9ed5ff86326fdaeb184c6b0e40f74f.svg
kaikai yang
user
keanudicap
646ad1a8c37ca1e1230c8a25
Pu Zhao
646ad1a8c37ca1e1230c8a26
Lu Wang
646ad1a8c37ca1e1230c8a27
Jue Zhang
646ad1a8c37ca1e1230c8a28
Mohit Garg
admin_assigned
2023-05-26T09:01:17.361Z
643425c8ea46c00990480f60
/avatars/012e77ac7e5d755050ea065b723a6b60.svg
Mohit Garg
user
mohitgargai
646ad1a8c37ca1e1230c8a29
Qingwei Lin
646ad1a8c37ca1e1230c8a2a
Dongmei Zhang
Empower Large Language Model to Perform Better on Industrial
  Domain-Specific Question Answering
Large Language Model (LLM) has gained popularity and achieved remarkable
results in open-domain tasks, but its performance in real industrial
domain-specific scenarios is average since there is no specific knowledge in
it. This issue has attracted widespread attention, but there are few relevant
benchmarks available. In this paper, we provide a benchmark Question Answering
(QA) dataset named MSQA, which is about Microsoft products and IT technical
problems encountered by customers. This dataset contains industry
cloud-specific QA knowledge, which is not available for general LLM, so it is
well suited for evaluating methods aimed at improving domain-specific
capabilities of LLM. In addition, we propose a new model interaction paradigm
that can empower LLM to achieve better performance on domain-specific tasks
where it is not proficient. Extensive experiments demonstrate that the approach
following our model fusion framework outperforms the commonly used LLM with
retrieval methods.
A benchmark QA dataset for Microsoft products and IT issues is introduced, along with a model fusion framework enhancing domain-specific LLM performance.
Large Language Model (LLM) Question Answering (QA) industry cloud-specific QA knowledge model fusion framework