SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1911.03347 · 2019

Macro F1 and Macro F1

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1911.03347")
get_code_for_paper("1911.03347")
have("1911.03347")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1911.03347.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1911.03347.svg)](https://syntology.ai/paper/1911.03347)

Connect an agent — have() is free.