Foresight Optimization for Strategic Reasoning in Large Language Models

Jian Wang, Wenjie Li, Kaitao Song, Jiashuo Wang, Chunpu Xu, Fenggang Yu, Jiawen Duan, Johan Hoorn, Johnny Ho

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 11 functions out of this paper's own repositories and ran 10 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• wangjs9/ForesightOptim

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Reasoning capabilities in large language models (LLMs) have generally advanced significantly. However, it is still challenging for existing reasoning-based LLMs to perform effective decision-making abilities in multiagent environments, due to the absence of explicit foresight modeling. To this end, strategic reasoning, the most fundamental capability to anticipate the counterpart's behaviors and foresee its possible future actions, has been introduced to alleviate the above issues. Strategic reasoning is fundamental to effective decision-making in multi-agent environments, yet existing reasoning enhancement methods for LLMs do not explicitly capture its foresight nature. In this work, we introduce Foresight Policy Optimization (FoPO) to enhance strategic reasoning in LLMs, which integrates opponent modeling principles into policy optimization, thereby enabling explicit consideration of both self-interest and counterpart influence. Specifically, we construct two curated datasets, namely Cooperative RSA and Competitive Taboo, equipped with well-designed rules and moderate difficulty to facilitate a systematic investigation of FoPO in a self-play framework. Our experiments demonstrate that FoPO significantly enhances strategic reasoning across LLMs of varying sizes and origins. Moreover, models trained with FoPO exhibit strong generalization to out-of-domain strategic scenarios, substantially outperforming standard LLM reasoning optimization baselines. 1 * Equal contribution. 1 https://github.com/wangjs9/ForesightOptim.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2604.13592")
get_code_for_paper("2604.13592")
have("2604.13592")

Connect an agent — have() is free.