Pu Zhao, Dongmei Zhang, Qingwei Lin, Lu Wang, Saravan Rajmohan, Fangkai Yang, Zezhong Wang, Jue Zhang, Mohit Garg
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Model (LLM) has gained popularity and achieved remarkable results in open-domain tasks, but its performance in real industrial domain-specific scenarios is average due to its lack of specific domain knowledge. This issue has attracted widespread attention, but there are few relevant benchmarks available. In this paper, we provide a benchmark Question Answering (QA) dataset named MSQA, centered around Microsoft products and IT technical problems encountered by customers. This dataset contains industry cloudspecific QA knowledge, an area not extensively covered in general LLMs, making it well-suited for evaluating methods aiming to enhance LLMs' domain-specific capabilities. In addition, we propose a new model interaction paradigm that can empower LLM to achieve better performance on domain-specific tasks where it is not proficient. Extensive experiments demonstrate that the approach following our method outperforms the commonly used LLM with retrieval methods. We make our source code and sample data available at: https://aka.ms/Microsoft_QA.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.11541")
get_code_for_paper("2305.11541")
have("2305.11541")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2305.11541.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2305.11541)
Connect an agent — have() is free.