SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2404.13599 · EMNLP · 2024

"A good pun is its own reword": Can Large Language Models Understand Puns?

Siyu Yuan, Deqing Yang, Zhijun Xu, ♠ Lingjie

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

As one of the common rhetorical devices, puns play a vital role in linguistic study, including the comprehensive analysis of linguistic humor. Although large language models (LLMs) have been widely explored on various tasks of natural language understanding and generation, their ability to understand puns has not been systematically studied, limiting the utilization of LLMs in creative writing and humor creation. In this paper, we leverage three popular tasks, i.e., pun recognition, pun explanation, and pun generation, to systematically evaluate LLMs' capability of understanding puns. In addition to the evaluation metrics adopted by prior research, we introduce some new evaluation methods and metrics that are better suited to the in-context learning paradigm of LLMs. These new metrics offer a more rigorous assessment of an LLM's capability to understand puns and align more closely with human cognition. Our research findings reveal the "lazy pun generation" pattern and identify the primary challenges in understanding puns with LLMs. The code is available at https://github.com/Zhijun-Xu/PunEval.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2404.13599")
get_code_for_paper("2404.13599")
have("2404.13599")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2404.13599.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2404.13599.svg)](https://syntology.ai/paper/2404.13599)

Connect an agent — have() is free.