Hrayr Harutyunyan, Avinash Ravichandran, Rahul Bhotika, Stefano Soatto, Alessandro Achille, Amazon Services, Giovanni Paolini, Orchid Majumder
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final weights and how much it informs the function computed by the weights. Though related, we show that these quantities have a qualitatively different behavior. We give efficient approximations of these quantities using a linearized network and demonstrate empirically that the approximation is accurate for real-world architectures, such as pre-trained ResNets. We apply these measures to several problems, such as dataset summarization, analysis of under-sampled classes, comparison of informativeness of different data sources, and detection of adversarial and corrupted examples. Our work generalizes existing frameworks but enjoys better computational properties for heavily overparametrized models, which makes it possible to apply it to real-world networks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2101.06640")
get_code_for_paper("2101.06640")
have("2101.06640")
Connect an agent — have() is free.