Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Models

Renfen Hu, Ziliang Qiu

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• ziliang6/PACE

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The evaluation of LLMs' creativity represents a crucial research domain, though challenges such as data contamination and costly human assessments often impede progress. Drawing inspiration from human creativity assessment, we propose PACE, asking LLMs to generate Parallel Association Chains to Evaluate their creativity. PACE minimizes the risk of data contamination and offers a straightforward, highly efficient evaluation, as evidenced by its strong correlation with Chatbot Arena Creative Writing rankings (Spearman's ρ = 0.739, p < 0.001) across various proprietary and open-source models. A comparative analysis of associative creativity between LLMs and humans reveals that while high-performing LLMs achieve scores comparable to average human performance, professional humans consistently outperform LLMs. Furthermore, linguistic analysis reveals that both humans and LLMs exhibit a trend of decreasing concreteness in their associations, and humans demonstrating a greater diversity of associative patterns. 1 Chapter Seed The physical world rock, wood, dust, rainbow, headland Kinship son, female, widow, son-in-law, stepdaughter Animals eagle, worm, dove, firefly, midge The body sick, toe, blink, eyelid, earwax Food and drink meal, pepper, crush, ripe, unripe

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2510.12110")
get_code_for_paper("2510.12110")
have("2510.12110")

Connect an agent — have() is free.