SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2403.15412 · EMNLP · 2024

Towards Measuring and Modeling "Culture" in LLMs: A Survey

Ashutosh Modi, Mbzuai, Alham Fikri, Sagnik Mukherjee, Monojit Choudhury, Muhammad Farid Adilazuarda, Pradhyumna Lavania, Siddhant Singh, Jacki O'neill

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present a survey of more than 90 recent papers that aim to study cultural representation and inclusion in large language models (LLMs). We observe that none of the studies explicitly define "culture", which is a complex, multifaceted concept; instead, they probe the models on some specially designed datasets which represent certain aspects of "culture." We call these aspects the proxies of culture, and organize them across two dimensions of demographic and semantic proxies. We also categorize the probing methods employed. Our analysis indicates that only certain aspects of "culture," such as values and objectives, have been studied, leaving several other interesting and important facets, especially the multitude of semantic domains (Thompson et al., 2020) and aboutness (Hershcovich et al., 2022), unexplored. Two other crucial gaps are the lack of robustness of probing techniques and situated studies on the impact of cultural misand under-representation in LLM-based applications. Compilation and details of papers used for the survey can be found via our GitHub repository 1 * Equal contribution 1 https://github.com/faridlazuarda/ cultural-llm-papers

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2403.15412")
get_code_for_paper("2403.15412")
have("2403.15412")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2403.15412.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2403.15412.svg)](https://syntology.ai/paper/2403.15412)

Connect an agent — have() is free.