Shimon Whiteson, M Pawan Kumar, Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent multi-task learning research argues against unitary scalarization, where training simply minimizes the sum of the task losses. Several ad-hoc multi-task optimization algorithms have instead been proposed, inspired by various hypotheses about what makes multi-task settings difficult. The majority of these optimizers require per-task gradients, and introduce significant memory, runtime, and implementation overhead. We show that unitary scalarization, coupled with standard regularization and stabilization techniques from single-task learning, matches or improves upon the performance of complex multi-task optimizers in popular supervised and reinforcement learning settings. We then present an analysis suggesting that many specialized multi-task optimizers can be partly interpreted as forms of regularization, potentially explaining our surprising results. We believe our results call for a critical reevaluation of recent research in the area.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2201.04122")
get_code_for_paper("2201.04122")
have("2201.04122")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2201.04122.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2201.04122)
Connect an agent — have() is free.