Accepted at COLM 2026 Main Conference, SciFM Workshop, and Social Sim Workshop

Capability Provenance in Language Models

A Case Study in Social Reasoning

Glenn Matlin Chandreyi Chakraborty Saehee Eom Mika Okamoto Rayan Castilla Louis Jaburi Alvin Deng Taywon Min Lucia Quirke Stella Biderman Mark Riedl

Georgia Tech MATS EleutherAI KAIST AI GT AI Safety Initiative

Abstract

Training data as a capability map.

How does a language model learn to interpret people's beliefs, emotions, intentions, and everyday moral choices? We trace that ability back to the training text that taught it: scoring influence across topic and format categories, reading the language of the highest-influence text, and confirming causally with unlearning. We expand the pipeline to a broad range of capabilities such as theory of mind, moral judgment, and social bias, as well as across multiple open models.

Method

From documents to interpretable corpus regions.

The central move is aggregation: every benchmark query is traced back to many individual documents, then summarized into comparable topic-format regions.

1. Labeled corpus open training data, binned 24 topics × 24 formats 2. Attribution score how each region supports each skill 3. Influence map supportive vs opposing corpus regions 4. Unlearning check forget flagged data, re-test the skill

The core tool

What is an influence function?

The model answered a question correctly. Which parts of its training data taught it that? An influence function estimates how the model's answer would change if one training document had been left out, without retraining the model.

During training, every document nudges the model's internal settings a little. A benchmark question also defines a direction: what would make the model better at this? Documents whose nudge points the same way have positive influence: they supported the skill. Documents whose nudge points the opposite way score negative.

The question's direction “what would make the model better at this?” this document's nudge points the same way positive influence: it supported the skill this document's nudge points the opposite way negative influence: it worked against it
24 topics
24 formats
1Social Life × Q&A Forum 2Science & Tech × Tutorial 3Politics × Documentation 4Literature × Customer Support 5Home & Hobbies × Creative Writing 6Education & Jobs × FAQ

Corpus taxonomy

24 topics crossed with 24 formats.

The corpus is de-duplicated and every document is labeled by what it is about (topic) and what kind of writing it is (format), using an off-the-shelf taxonomy (WebOrganizer). That gives 576 topic-format bins, and the highlighted cells are illustrative examples. Sampling evenly across the occupied bins turns attribution from a document retrieval problem into a structured corpus comparison problem.

Contrastive design

Domain and capability type are separated.

The 2 by 2 design asks whether social reasoning is merely social-domain knowledge, or whether reasoning tasks use different corpus regions than knowledge tasks in the same domain.

Social
STEM
Reasoning
ARC-Challenge science reasoning
Knowledge
MMLU STEM STEM knowledge

Findings

Different capabilities, different data distributions.

Core finding: each capability draws support from its own distribution of training text, not one shared pile.

The signature in OLMo3

Two flavors of text carry social reasoning.

Social reasoning's influence profile across the 576 corpus bins looks unlike the profiles of STEM reasoning and the two knowledge tasks, which closely resemble one another. And the social-versus-STEM contrast is wider for the reasoning pair than for the knowledge pair, while leaning the same direction in both.

Read the high-influence text and it comes in two flavors: interactional writing (dialogue, questions and answers, lots of "you" and "I") and expository writing about how people deal with each other, such as guides and manuals. STEM reasoning draws instead on structured, technical text. The claim that travels is the shape of the distribution, not any single corpus slice.

Q&Aadvicefeelingsreplyintent

interactional writing

guidemanualpolicyproceduresupport

people-centered documentation

leans STEM leans social Reasoning pair SocialIQA minus ARC-Challenge 0.91 0.91 software development literature, education & jobs Knowledge pair MMLU Social Sciences minus MMLU STEM 0.63 0.63 same direction in both pairs, about 1.4 times the spread for reasoning

Reasoning versus knowledge

The split is sharper for reasoning.

Knowledge tasks ask the model to retrieve facts about a domain. Reasoning tasks ask it to work over one. The two stress different training text. Measured by the strongest topic-level influence differences (|Δz|) in OLMo3, the social-minus-STEM contrast is about 1.4 times wider for the reasoning pair than for the knowledge pair, while leaning the same direction in both.

At the topic level, reasoning leans social on literature and education and leans STEM on science & technology and software development. In this controlled 2 by 2 design, capability type differentiates corpus provenance more than domain does.

The causal check

Forget the flagged text and the skill fades.

Correlation alone does not prove the flagged text matters. So we make the model unlearn the high-influence regions and compare against unlearning randomly chosen documents from the same topics. In OLMo3, influence-targeted forgetting damages social reasoning far more than the random controls, while the effect on the other benchmarks is weak or absent. The chart shows the extra accuracy damage in percentage points for the topics where the gap is largest.

The check validates corpus regions in aggregate, not individually causal documents. Which benchmark carries the effect can shift with the training corpus. The benchmark-specific selectivity is what repeats.

Literature +17.53 pp Education & Jobs +15.03 pp Sports & Fitness +8.40 pp Fashion & Beauty +6.93 pp Home & Hobbies +6.87 pp

Data

Fully open.

The study is possible because the OLMo ecosystem opens every layer of the stack.

OLMo 3 The open-weights model whose skills we trace.
Dolma 3 The data OLMo 3 was trained on, with every document open for analysis.
WebOrganizer A labeling system that sorts datasets like Dolma into categories by topic and format.
OLMo 3 evaluation suite Supplies the benchmarks that measure model abilities.
OLMES A shared standard for scoring those benchmarks.

Expansion

Beyond one social skill.

Social reasoning is not one thing. A held-out suite of probes runs through the same attribution pipeline, testing how far the social reasoning lens extends across the social space.

Theory of mind Reasoning about what other people know, want, and believe.
Moral judgment Everyday moral and normative choices.
Social bias Stereotypes and biased framing in social situations.
Pragmatics Reading implied meaning beyond the literal words.

Expansion

Beyond one model.

The same measurements are being re-run across a family of open-data models: different corpora, different token budgets, same question. Which specific regions carry a skill can shift with the corpus. The separation between capability distributions is the pattern we test for.

Roster of open-data models in the study: the OLMo3 Base 7B primary study, in-progress runs for OLMo3 Instruct, Marin, DCLM, and Comma, and planned 32B-scale runs.
Model Training corpus Scale Status
OLMo3 Base Dolma3 7B Primary study
OLMo3 Instruct Dolma3 + post-training 7B In progress
Marin Open mixture 8B In progress
DCLM DCLM-Baseline 7B In progress
Comma Common Pile 7B In progress
OLMo3 Base Dolma3 32B Planned
Marin Open mixture 32B Planned

Citation

Cite the paper.

@inproceedings{matlin2026capabilityprovenance,
  title         = {Capability Provenance in Language Models: A Case Study in Social Reasoning},
  author        = {Glenn Matlin and Chandreyi Chakraborty and Saehee Eom and Mika Okamoto and
                   Rayan Castilla and Louis Jaburi and Alvin Deng and Taywon Min and
                   Lucia Quirke and Stella Biderman and Mark Riedl},
  booktitle     = {Proceedings of the Conference on Language Modeling (COLM 2026)},
  year          = {2026},
  eprint        = {2606.19625},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2606.19625}
}