SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2506.06404 · ACL · 2025

Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights

Xing Xie, Xiaoyuan Yi, Jinyeong Bak, Jing Yao, Sooyung Choi, Jaehyeok Lee

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 2 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
meta-llama/llama-recipes canonical 0 of 1
Human-Language-Intelligence/Unintended-Harms-LLM reimplementation 1 of 1
FunctionStatusWhere it lives
remove_non_english_characters Ran Human-Language-Intelligence/Unintended-Harms-LLM/evaluate/eval_RTP.py
pointer only (licence: NONE) · get_code("0b48977d846ad894")
get_llm_response Not yet run meta-llama/llama-recipes/end-to-end-use-cases/whatsapp_llama_4_bot/ec2_services.py
pointer only (licence: MIT) · get_code("e6104bf4c7a58936")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The application scope of Large Language Models (LLMs) continues to expand, leading to increasing interest in personalized LLMs that align with human values. However, aligning these models with individual values raises significant safety concerns, as certain values may correlate with harmful information. In this paper, we identify specific safety risks associated with value-aligned LLMs and investigate the psychological principles behind these challenges. Our findings reveal two key insights. (1) Value-aligned LLMs are more prone to harmful behavior compared to non-fine-tuned models and exhibit slightly higher risks in traditional safety evaluations than other fine-tuned models. (2) These safety issues arise because value-aligned LLMs genuinely generate text according to the aligned values, which can amplify harmful outcomes. Using a dataset with detailed safety categories, we find significant correlations between value alignment and safety risks, supported by psychological hypotheses. This study offers insights into the "black box" of value alignment and proposes in-context alignment methods to enhance the safety of value-aligned LLMs. 1 Warning: This paper contains contents that may be offensive or upsetting.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2506.06404")
get_code_for_paper("2506.06404")
have("2506.06404")

Connect an agent — have() is free.