SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2405.18347 · ECCV · 2024

Dataset Growth

Radu Timofte, Hao Tang, Kai Wang, Yang You, Lei Shang, Baigui Sun, Zangwei Zheng, Zebang Cheng, Xiaojiang Peng, Yukun Zhou, Ziheng Qin, Zhaopan Xu, and 1 more

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Deep learning benefits from the growing abundance of available data. Meanwhile, efficiently dealing with the growing data scale has become a challenge. Data publicly available are from different sources with various qualities, and it is impractical to do manual cleaning against noise and redundancy given today's data scale. There are existing techniques for cleaning/selecting the collected data. However, these methods are mainly proposed for offline settings that target one of the cleanness and redundancy problems. In practice, data are growing exponentially with both problems. This leads to repeated data curation with sub-optimal efficiency. To tackle this challenge, we propose InfoGrowth, an efficient online algorithm for data cleaning and selection, resulting in a growing dataset that keeps up to date with awareness of cleanliness and diversity. InfoGrowth can improve data quality/efficiency on both singlemodal and multi-modal tasks, with an efficient and scalable design. Its framework makes it practical for real-world data engines.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2405.18347")
get_code_for_paper("2405.18347")
have("2405.18347")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2405.18347.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2405.18347.svg)](https://syntology.ai/paper/2405.18347)

Connect an agent — have() is free.