Tracing Knowledge and Reasoning in LLMs
The authors report in this preprint: Researchers propose a task-centric framework combining attention, activation, and representation metrics to analyze internal model processing across 24 benchmarks, revealing systematic differences between knowledge-centric and reasoning-centric tasks.

Original image temporarily unavailable
The authors report in this preprint: Current evaluations of Large Language Models (LLMs) primarily report dataset-level performance scores, which conceal internal processing mechanisms and failure modes. To address this, the authors propose a task-centric framework that combines attention- and activation-based signals with representation-based measures. This approach characterizes benchmarks along a continuum from retrieval-dominant to reasoning-dominant. The authors evaluate 16 LLMs across 24 diverse benchmarks and perform internal-state analyses on representative models and tasks. The results show systematic differences in processing patterns between knowledge-centric and reasoning-centric tasks. The code has been made open source, providing a tool to support targeted benchmark selection and informed model choice.
View selection rationale
Importance 77/100
About this brief
- Content type
- Research summary
- Source type
- First-party / official source
- Original source
- arXiv Computation and Language ↗
- Source published
- Process
- AI-assisted discovery, translation, summarisation, classification and scoring; standards and corrections are maintained by the editorial desk.
- Verification
- The original URL and publication metadata are retained. This brief is not independently fact-checked unless explicitly stated.
- Published by
- Polyemergence Editorial Desk · PolyEmergence Pte Ltd.
- Conflict disclosure
- No financial or commercial conflict declared.