SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1606.02206 · 2016

A Minimax Approach to Supervised Learning

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Given a task of predicting $Y$ from $X$, a loss function $L$, and a set of probability distributions $Γ$ on $(X,Y)$, what is the optimal decision rule minimizing the worst-case expected loss over $Γ$? In this paper, we address this question by introducing a generalization of the principle of maximum entropy. Applying this principle to sets of distributions with marginal on $X$ constrained to be the empirical marginal from the data, we develop a general minimax approach for supervised learning problems. While for some loss functions such as squared-error and log loss, the minimax approach rederives well-knwon regression models, for the 0-1 loss it results in a new linear classifier which we call the maximum entropy machine. The maximum entropy machine minimizes the worst-case 0-1 loss over the structured set of distribution, and by our numerical experiments can outperform other well-known linear classifiers such as SVM. We also prove a bound on the generalization worst-case error in the minimax approach.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1606.02206")
get_code_for_paper("1606.02206")
have("1606.02206")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1606.02206.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1606.02206.svg)](https://syntology.ai/paper/1606.02206)

Connect an agent — have() is free.