We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Intent Detection systems in the real world are exposed to complexities of imbalanced datasets containing varying perception of intent, unintended correlations and domain-specific aberrations. To facilitate benchmarking which can reflect near real-world scenarios, we introduce 3 new datasets created from live chatbots in diverse domains. Unlike most existing datasets that are crowdsourced, our datasets contain real user queries received by the chatbots and facilitates penalising unwanted correlations grasped during the training process. We evaluate 4 NLU platforms and a BERT based classifier and find that performance saturates at inadequate levels on test sets because all systems latch on to unintended patterns in training data.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2009.13833")
get_code_for_paper("2009.13833")
have("2009.13833")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2009.13833.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2009.13833)
Connect an agent — have() is free.