SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1901.11503 · 2019

Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space exploration methods and those in which they are superior. Through simple theoretical analyses, we prove that complexity of exploration in parameter space depends on the dimensionality of parameter space, while complexity of exploration in action space depends on both the dimensionality of action space and horizon length. This is also demonstrated empirically by comparing simple exploration methods on several model problems, including Contextual Bandit, Linear Regression and Reinforcement Learning in continuous control.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1901.11503")
get_code_for_paper("1901.11503")
have("1901.11503")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1901.11503.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1901.11503.svg)](https://syntology.ai/paper/1901.11503)

Connect an agent — have() is free.