MORL-A2C: Multi-Objective Reinforcement Learning Reranker for Optimizing Healthiness in MOPI-HFRS

Chuxu Zhang, Aarya Vasantlal, Joshua Zolla

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 15 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• JoshZ411/MOPI-HFRS-SLC

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Unhealthy dietary behavior continues to be a persistent and escalating public health issue in the United States, exacerbated by the prevalence of recommendation systems that prioritize user preference without adequately considering nutritional health. The Multi-Objective Personalized Interpretable Health-aware Food Recommendation System (MOPI-HFRS) [9], from which this work extends, addresses this limitation by jointly optimizing user preference, personalized health, and nutritional diversity through a Pareto-based optimization framework. However, this approach relies on static, per-step tradeoff solutions which fail to capture the sequential and cumulative nature of dietary decision-making, in which the value of recommending an item depends on what has already been recommended. In this work, we introduce the Multi-Objective Reinforcement Learning Reranker (MORL-A2C), a sequential decisionmaking extension to MOPI-HFRS that targets the health-preference axis of the multi-objective problem. Leveraging frozen graph neural network embeddings from MOPI-HFRS, MORL-A2C formulates recommendation as a 𝐾-step sequential reranking problem, employing an Advantage Actor-Critic (A2C) algorithm to learn a policy balancing relevance and health objectives through a scalarized reward signal. To accelerate convergence and avoid degenerate exploration, the policy is warm-started using behavior cloning pretraining against a strong dot-product ranker derived from the frozen embeddings. In the course of this work, we identify and correct a non-trivial bug in the original MOPI-HFRS evaluation pipeline that understated baseline ranking performance; all RL-based metrics are reported against the corrected baseline. Experimental results on both the macro-nutrient and full-nutrient benchmarks indicate that MORL-A2C achieves a modest reduction in ranking quality relative to the corrected baseline (Recall@20: 25.64% → 23.61%, NDCG@20: 23.52% → 20.64% on the macro benchmark) in exchange for a substantial improvement in personalized health alignment (H-Score@20: 46.05% → 69.57%), with consistent trends on the full-nutrient benchmark. These findings validate that policydriven sequential optimization can effectively navigate the healthpreference trade-off in multi-objective food recommendation. Our code is publicly available at https://github.com/JoshZ411/MOPI-HFRS-SLC.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2606.23603")
get_code_for_paper("2606.23603")
have("2606.23603")

Connect an agent — have() is free.