Historical brief · 2026-11-16

Tracing Knowledge and Reasoning in LLMs

The authors report in this preprint: Researchers propose a task-centric framework combining attention, activation, and representation metrics to analyze internal model processing across 24 benchmarks, revealing systematic differences between knowledge-centric and reasoning-centric tasks.

Source image · arXiv Computation and Language

The authors report in this preprint: Current evaluations of Large Language Models (LLMs) primarily report dataset-level performance scores, which conceal internal processing mechanisms and failure modes. To address this, the authors propose a task-centric framework that combines attention- and activation-based signals with representation-based measures. This approach characterizes benchmarks along a continuum from retrieval-dominant to reasoning-dominant. The authors evaluate 16 LLMs across 24 diverse benchmarks and perform internal-state analyses on representative models and tasks. The results show systematic differences in processing patterns between knowledge-centric and reasoning-centric tasks. The code has been made open source, providing a tool to support targeted benchmark selection and informed model choice.

View selection rationale
PreprintStage not established

Importance 77/100

Signal75Structure80Consequence75Confidence85Momentum60
About this brief
Content type
Research summary
Source type
First-party / official source
Source published
Process
AI-assisted discovery, translation, summarisation, classification and scoring; standards and corrections are maintained by the editorial desk.
Verification
The original URL and publication metadata are retained. This brief is not independently fact-checked unless explicitly stated.
Published by
Polyemergence Editorial Desk · PolyEmergence Pte Ltd.
Conflict disclosure
No financial or commercial conflict declared.

Editorial standards → · Corrections →

POLYEMERGENCE / EXPLORE

Keep exploring

Follow the topic, not just the headline.

Explore the Artificial Intelligence archive →