SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2501.09695 · CVPR · 2025

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Dongqi Han, Xufang Luo, Dongsheng Li, Zhihe Yang, Yunjian Xu

arXiv · PDF · Open in the Atlas

Code that ran

11 of 16 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 0 violated it and 8 have no contract verdict; 2 failures of our instrument; 5 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2501.09695.json · badge: /api/ran/2501.09695.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
zhyang2226/opa-dpo canonical (classifier label)
classifier basis not recorded on the link
11 ran · 5 no recorded run
FunctionStatusWhere it lives
add_eos Ran
ran on a synthesized input
zhyang2226/opa-dpo/utils/data_utils_dpo.py
pointer only (licence: MIT) · get_code("4bf651c7cdff6f6f")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

cap_exp Ran
ran on a synthesized input
zhyang2226/opa-dpo/opadpo/dpo_models/dpo_trainer.py
pointer only (licence: MIT) · get_code("710934f5316eeea5")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

complete_copied_content Ran
ran on a synthesized input
zhyang2226/opa-dpo/utils/data_utils_dpo.py
pointer only (licence: MIT) · get_code("d9c6fd44c3b0a600")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_chunk Ran
ran; our contract draft was wrong, not the code
zhyang2226/opa-dpo/eval_llava_rlhf_coco/model_vqa.py
pointer only (licence: MIT) · get_code("42a46570620cd9fa")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_exp_cap Ran
ran on a synthesized input
zhyang2226/opa-dpo/opadpo/dpo_models/dpo_trainer.py
pointer only (licence: MIT) · get_code("25b50a1e049a0776")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

mask_percentage Ran
ran on a synthesized input
zhyang2226/opa-dpo/opadpo/opa_models/opa_trainer.py
pointer only (licence: MIT) · get_code("ce25505048ba7022")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

mask_single_image Ran
ran on a synthesized input
zhyang2226/opa-dpo/opadpo/dpo_models/dpo_trainer.py
pointer only (licence: MIT) · get_code("79d5802b19f55211")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

override_config Ran
ran on a synthesized input
zhyang2226/opa-dpo/opadpo/opa_train_custom.py
pointer only (licence: MIT) · get_code("b50493d8f3ff8ede")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

pad_and_stack Ran
ran on a synthesized input
zhyang2226/opa-dpo/utils/data_utils_dpo.py
pointer only (licence: MIT) · get_code("b5b32512d6d561f5")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

split_list Ran
ran; our contract draft was wrong, not the code
zhyang2226/opa-dpo/eval_llava_rlhf_coco/model_vqa.py
pointer only (licence: MIT) · get_code("076c252c52cbb161")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

truncate_after_eos_with_padding Ran
ran, honoured the contract we drafted
zhyang2226/opa-dpo/opadpo/dpo_models/rl_trainer.py
pointer only (licence: MIT) · get_code("6ff9248489766439")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_mm_adapter_state_maybe_zero_3 Not yet run
unverified (harvested, no recorded run)
zhyang2226/opa-dpo/opadpo/opa_models/opa_trainer.py
pointer only (licence: MIT) · get_code("bb35e3ac741bb2c9")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

get_peft_state_maybe_zero_3 Not yet run
unverified (harvested, no recorded run)
zhyang2226/opa-dpo/opadpo/opa_train.py
pointer only (licence: MIT) · get_code("63b8ff87fd81eab7")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

get_peft_state_non_lora_maybe_zero_3 Not yet run
unverified (harvested, no recorded run)
zhyang2226/opa-dpo/opadpo/opa_train.py
pointer only (licence: MIT) · get_code("1c53657305b66e9f")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

maybe_zero_3 Not yet run
unverified (harvested, no recorded run)
zhyang2226/opa-dpo/opadpo/opa_models/opa_trainer.py
pointer only (licence: MIT) · get_code("735025744c1ab0cf")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

maybe_zero_3 Not yet run
unverified (harvested, no recorded run)
zhyang2226/opa-dpo/opadpo/opa_train.py
pointer only (licence: MIT) · get_code("616ffbdc154ed2d8")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, zhyang2226/opa-dpo, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 16 samples lifted from this paper's linked repositories, 16 carry a function identity and 0 do not. Of the 16 distinct functions with an identity, 7 are also present in other papers whose code we hold (7 with at least one paper dated earlier, 4 the same month, 6 later) and 9 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
split_list 197 other papers
151 dated earlier, 1 the same month, 37 later, 7 same year, order not determinable, 1 undated
2112.05682 (2021-12)
Self-attention Does Not Need $O(n^2)$ Memory
2306.17107 (2023-06)
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
2307.10490 (2023-07)
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
and 194 more not named here
get_chunk 189 other papers
146 dated earlier, 1 the same month, 35 later, 7 same year, order not determinable
2112.05682 (2021-12)
Self-attention Does Not Need $O(n^2)$ Memory
2306.17107 (2023-06)
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
2307.10490 (2023-07)
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
and 186 more not named here
get_mm_adapter_state_maybe_zero_3 43 other papers
32 dated earlier, 1 the same month, 10 later
2304.05884 (2023-04)
Unicom: Universal and Compact Representation Learning for Image Retrieval
2304.08485 (2023-04)
Visual Instruction Tuning
2310.03744 (2023-10)
Improved Baselines with Visual Instruction Tuning
and 40 more not named here
maybe_zero_3 42 other papers
33 dated earlier, 9 later
2304.05884 (2023-04)
Unicom: Universal and Compact Representation Learning for Image Retrieval
2304.08485 (2023-04)
Visual Instruction Tuning
2310.03744 (2023-10)
Improved Baselines with Visual Instruction Tuning
and 39 more not named here
maybe_zero_3 37 other papers
23 dated earlier, 1 the same month, 12 later, 1 same year, order not determinable
2310.01779 (2023-10)
HallE-Control: Controlling Object Hallucination in Large Multimodal Models
2311.16839 (2023-11)
Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization
2312.02949 (2023-12)
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
and 34 more not named here
get_peft_state_non_lora_maybe_zero_3 23 other papers
16 dated earlier, 6 later, 1 same year, order not determinable
2310.01779 (2023-10)
HallE-Control: Controlling Object Hallucination in Large Multimodal Models
2312.02949 (2023-12)
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
2401.02330 (2024-01)
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model
and 20 more not named here
truncate_after_eos_with_padding 1 other paper
1 dated earlier
2406.12030 (2024-06)
SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Hallucination remains a major challenge for Large Vision-Language Models (LVLMs). Direct Preference Optimization (DPO) has gained increasing attention as a simple solution to hallucination issues. It directly learns from constructed preference pairs that reflect the severity of hallucinations in responses to the same prompt and image. Nonetheless, different data construction methods in existing works bring notable performance variations. We identify a crucial factor here: outcomes are largely contingent on whether the constructed data aligns on-policy w.r.t the initial (reference) policy of DPO. Theoretical analysis suggests that learning from off-policy data is impeded by the presence of KL-divergence between the updated policy and the reference policy. From the perspective of dataset distribution, we systematically summarize the inherent flaws in existing algorithms that employ DPO to address hallucination issues. To alleviate the problems, we propose On-Policy Alignment (OPA)-DPO framework, which uniquely leverages expert feedback to correct hallucinated responses and aligns both the original and expert-revised responses in an on-policy manner. Notably, with only 4.8k data, OPA-DPO achieves an additional reduction in the hallucination rate of LLaVA-1.5-7B: 13.26% on the AMBER benchmark and 5.39% on the Object-Hal benchmark, compared to the previous SOTA algorithm trained with 16k samples.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2501.09695")
get_code_for_paper("2501.09695")
have("2501.09695")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2501.09695.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2501.09695.svg)](https://syntology.ai/paper/2501.09695)

Connect an agent — have() is free.