Brendan O'connor, Katherine Keith, Andrew Halterman, Muhammad Sarwar
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Automated event extraction in social science applications often requires corpus-level evaluations: for example, aggregating text predictions across metadata and unbiased estimates of recall. We combine corpus-level evaluation requirements with a real-world, social science setting and introduce the INDIAPO-LICEEVENTS corpus-all 21,391 sentences from 1,257 English-language Times of India articles about events in the state of Gujarat during March 2002. Our trained annotators read and label every document for mentions of police activity events, allowing for unbiased recall evaluations. In contrast to other datasets with structured event representations, we gather annotations by posing natural questions, and evaluate off-the-shelf models for three different tasks: sentence classification, document ranking, and temporal aggregation of target events. We present baseline results from zero-shot BERT-based models fine-tuned on natural language inference and passage retrieval tasks. Our novel corpus-level evaluations and annotation approach can guide creation of similar social-science-oriented resources in the future.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2105.12936")
get_code_for_paper("2105.12936")
have("2105.12936")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2105.12936.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2105.12936)
Connect an agent — have() is free.