SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2207.01736 · NAACL · 2022

Probing via Prompting

Mrinmaya Sachan, Ryan Cotterell, Jiaoda Li

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Probing is a popular method to discern what linguistic information is contained in the representations of pre-trained language models. However, the mechanism of selecting the probe model has recently been subject to intense debate, as it is not clear if the probes are merely extracting information or modeling the linguistic property themselves. To address this challenge, this paper introduces a novel model-free approach to probing, by formulating probing as a prompting task. We conduct experiments on five probing tasks and show that our approach is comparable or better at extracting information than diagnostic probes while learning much less on its own. We further combine the probing via prompting approach with attention head pruning to analyze where the model stores the linguistic information in its architecture. We then examine the usefulness of a specific linguistic property for pre-training by removing the heads that are essential to that property and evaluating the resulting model's performance on language modeling. https://github.com/rycolab/ probing-via-prompting

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2207.01736")
get_code_for_paper("2207.01736")
have("2207.01736")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2207.01736.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2207.01736.svg)](https://syntology.ai/paper/2207.01736)

Connect an agent — have() is free.