Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDP

The authors report in this preprint: Safety benchmarks for large language models frequently rely on multiple-choice question (MCQ) formats, yet prior work shows this format suppresses model refusal. This preprint provides the first mechanistic explanation of this phenomenon using Llama-3.1-8B and Qwen3-8B on the Weapons of Mass Destruction Proxy (WMDP) dataset. Using the tuned lens technique, the researchers demonstrate that when processing MCQs, models commit to an answer-letter token very early, contrasting with the gradual build-up of refusal seen in open-ended formats. Furthermore, activations during MCQ processing lie approximately three times further from the internal refusal direction. Ablating attention heads that carry the MCQ answer increases refusal rates, directly evidencing that answer-selection competes with and overrides safety refusal mechanisms. This implies that MCQ safety benchmarks often measure raw knowledge capabilities rather than true alignment or safety posture. As a preprint, code release and independent replication are necessary to fully corroborate these internal activation claims.