Jie Li, Jingjing Liu, Shengbo Li, Xianyuan Zhan, Jingliang Duan, Keqiang Li, Yujie Yang, Yao Lyu, Wenjun Zou, Yaqin Zhang
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Safe reinforcement learning (RL) offers advanced solutions to constrained optimal control problems. Existing studies in safe RL implicitly assume continuity in policy functions, where policies map states to actions in a smooth, uninterrupted manner. However, our research finds that in some scenarios, the feasible policy should be discontinuous or multi-valued, and a continuous policy can inevitably lead to constraint violations. We are the first to identify the generating mechanism of such a phenomenon, and employ topological analysis to rigorously prove the existence of policy bifurcation in safe RL, which corresponds to the contractibility of the reachable tuple. Our theorem reveals that in scenarios where the obstacle-free state space is non-simply connected, a feasible policy is required to be bifurcated, meaning its output action needs to change abruptly in response to the varying state. To train such a bifurcated policy, we propose a safe RL algorithm called multimodal policy optimization (MUPO), which utilizes a Gaussian mixture distribution as the policy output. The bifurcated behavior can be achieved by selecting the Gaussian component with the highest mixing coefficient. Experiments with vehicle control tasks show that our algorithm successfully learns the bifurcated policy and ensures satisfying safety, while a continuous policy suffers from inevitable constraint violations.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2403.12847")
get_code_for_paper("2403.12847")
have("2403.12847")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2403.12847.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2403.12847)
Connect an agent — have() is free.