Xiang Ren, Bill Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Question: I have five fingers but I am not alive. What am I? Answer: a glove. Answering such a riddle-style question is a challenging cognitive process, in that it requires complex commonsense reasoning abilities, an understanding of figurative language, and counterfactual reasoning skills, which are all important abilities for advanced natural language understanding (NLU). However, there is currently no dataset aiming to test these abilities. In this paper, we present RIDDLE-SENSE 1 , a new multiple-choice question answering task, which comes with the first large dataset (5.7k examples) for answering riddlestyle commonsense questions. We systematically evaluate a wide range of models over the RIDDLESENSE challenge, and point out that there is a large gap between the bestsupervised model and human performancesuggesting intriguing future research in the direction of higher-order commonsense reasoning and linguistic creativity towards building advanced NLU systems.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2101.00376")
get_code_for_paper("2101.00376")
have("2101.00376")
Connect an agent — have() is free.