First surfaced in Tandemly Briefing — 2026-05-25.

ML Interpretability · Position Paper

Interpretability
can be actionable.

A coalition of leading NLP and ML researchers argues the field is measuring the wrong thing. Whether a finding is elegant is beside the point. The question is: what can you actually do with it?

Core rubric
Two dimensions grade any interpretability claim: concreteness (does it specify an intervention?) and validation (was that intervention tested?).
scroll to explore
01The problem

Elegant explanations
that don't change anything.

The field of interpretability has developed powerful tools for describing what happens inside neural networks. The missing piece is a clear standard for what those descriptions should be good for.

Ask an interpretability researcher to evaluate a finding, and they will likely ask: is this explanation faithful to what the model actually does? Does the circuit diagram correctly reflect the computations? Does the attribution map highlight the genuinely relevant tokens? These are the right engineering questions for building accurate descriptions.

But a faithful description and a useful description are not the same thing. You can know that attention head 7 in layer 12 is "responsible for indirect object identification" and still have no obvious way to use that knowledge. Does it tell you where to look when the model makes an error? Does it suggest a targeted fix? Does it give you a new test case to write? For most published findings, the answer is no.

Behavioral evaluation asks whether a model produces the right outputs for given inputs. It catches many failures without requiring any understanding of the model's internals. The practical argument for interpretability, then, depends on what it can do that behavioral evaluation cannot. The authors argue the field has not answered that question rigorously enough, and that the consequence is a large body of work that is scientifically valid but does not move practice.

The claim this paper makes

The problem is not that interpretability research lacks rigor. It is that the field evaluates itself on fidelity to mechanism rather than on whether the finding enables a practitioner to do something differently. Adopting actionability as a first-class criterion would redirect effort toward work that actually changes what teams can build and audit.

02The framework

Concreteness and
validation as grade criteria.

The paper proposes a two-axis rubric for evaluating any interpretability finding, designed to be fast to apply and to expose the gap between description and utility.

The rubric does not ask whether the explanation is mechanistically accurate. It assumes that work passing peer review is, in general, reasonably faithful. Instead it asks two questions that practitioners actually care about.

The first question is concreteness. A finding is concrete when it names a specific, implementable intervention. "Ablate attention heads 4 through 7 in layers 8 to 11 to reduce gender bias on the Winogender benchmark" is concrete. "These features appear to matter for factual recall" is not. Concreteness has a simple test: can you write down the procedure without returning to the authors for clarification? If not, the finding is a description, not a recipe.

The second question is validation. Was the proposed intervention actually tested? Did the observed effect match the prediction? A finding that specifies an intervention but does not report testing it sits one step away from useful: the claim is plausible, but the evidence is missing. High validation means the paper shows cause and effect, not just correlation of features with behavior.

Low concreteness
Description only. The finding characterizes a mechanism or feature without specifying what to do with it. Useful for basic science; not useful for practitioners who need to ship or audit a model.
High concreteness
Intervention specified. The finding names what to change, remove, monitor, or patch. A practitioner can write down the procedure from the description alone, without author clarification.
Low validation
Untested claim. The intervention is proposed but not empirically confirmed. The mechanism is plausible, but the paper does not show that acting on it produces the predicted outcome.
High validation
Causal evidence shown. The paper demonstrates that applying the proposed intervention causes the predicted change, with appropriate controls. The finding can be treated as a reliable operational handle.

The rubric is deliberately coarse. A two-by-two grid loses nuance. The authors frame it as a quick filter, not a comprehensive scoring system. Its purpose is to make explicit a distinction that practitioners routinely make implicitly: the difference between a finding they could act on today and one they file away for future reference.

The paper then identifies five areas of the ML lifecycle where scoring high on both axes of this rubric is not only possible but produces results that behavioral evaluation alone cannot.

Why a position paper?

This is a conceptual contribution, not an empirical study. The authors do not run experiments or introduce a new method. They build a framework and make an argument: that the interpretability field's implicit evaluation criteria are misaligned with the practical value it could be providing, and that making the criteria explicit would shift which work gets done and published. The five-domain taxonomy is the empirical anchor for that argument.

03Five domains

Where actionable
interpretability earns its keep.

The authors identify five areas where actionable interpretability provides leverage that behavioral testing alone cannot. These are not the only applications, but they share a structural property: the mechanism matters, not just the output.

1
Model editing and targeted correction
Behavioral evaluation can tell you a model gives wrong answers about a topic. It cannot tell you where to intervene to correct specific knowledge without degrading unrelated behavior. Actionable interpretability specifies which parameters, attention patterns, or internal representations to modify, and validates that modifying them produces the expected correction without side effects.
2
Knowledge attribution and provenance
Behavioral testing can measure whether a model's output is consistent with a claimed source, but attribution requires understanding how the model stores and retrieves learned information. Identifying which training examples or internal knowledge structures drive a particular output is a structural question, not an input-output question.
3
Safety and alignment evaluation
Red-teaming and adversarial probing find behavioral failures, but they cannot rule out latent failure modes that no test prompt has yet triggered. Interpretability can identify internal representations associated with harmful outputs and flag whether those representations are present even in models that currently pass behavioral safety benchmarks. The distinction matters for high-stakes deployment.
4
Capability discovery in frozen models
Behavioral benchmarks measure the capabilities a model demonstrates when prompted. They do not reveal capabilities the model has but does not exhibit spontaneously. Interpretability methods can locate internal structures associated with a capability and design prompts or fine-tuning procedures to elicit it, expanding effective capability without additional training.
5
Model compression and efficiency
Pruning and quantization guided only by output metrics treat the model as a black box and risk removing components that matter for rare cases or long-tail inputs. Interpretability identifies which subnetworks or heads are responsible for specific behaviors, supporting precision pruning that degrades performance on targeted capabilities rather than degrading uniformly across the capability surface.
Scope and limitations of the argument

This is a position paper, so the authors are making a normative argument: the field should do this. They are not reporting an experiment in which actionable interpretability was compared to non-actionable interpretability on some outcome metric. The five-domain list is structured reasoning, not a controlled trial. Readers can disagree with the taxonomy or with the claim that behavioral evaluation is insufficient in these domains. The paper frames these as starting positions for a debate, not as settled facts.

The authors also acknowledge that the rubric is a simplification. Many findings exist on a spectrum of concreteness, and "high validation" is easier to achieve for some problems than others. The intent is to make an implicit standard explicit, not to claim precision the framework does not provide.

04Practical takeaways

What this means
for teams building AI.

The rubric is fast to apply. The argument has immediate practical weight for anyone consuming interpretability research or deciding whether to invest in interpretability tooling.

1
For ML practitioners and teams consuming interpretability research
Before acting on an interpretability finding, run the two-question check: does this specify an intervention, and was that intervention tested? If the answer to either question is no, treat the finding as exploratory signal rather than operational guidance. The distinction matters most when the finding would influence a model behavior decision, a safety audit, or a deployment gate.
2
For safety and governance teams
The five-domain list provides a principled checklist for where interpretability investment is most likely to produce results that behavioral testing cannot. Safety and alignment evaluation (domain 3) is the most obvious priority for teams working on model deployment decisions, because it addresses the latent-risk gap that red-teaming alone cannot close. If your interpretability tooling does not touch any of the five domains, its operational value is likely lower than it appears.
3
For researchers and teams producing interpretability findings
Designing for actionability requires specifying the intervention before running the study, not adding one at the end as a suggestion for future work. Studies where the intervention is integral to the design are easier to validate, because the experimental protocol already includes the condition needed to test cause and effect. "Future work could explore whether this finding enables targeted editing" is not validation.
4
A word of caution on scope
The authors are not arguing that mechanistic description is worthless. Understanding internal structures has scientific value independent of immediate application, and some of the most useful applied findings began as purely descriptive work. The argument is about evaluation criteria and incentives, not about banning basic research. The rubric should be applied to findings teams intend to act on, not to all interpretability work regardless of its purpose.
05Getting started

Where to go
from here.

Practical steps for teams that want to apply this framework immediately.

1
Read the paper
Orgad, Barez, Haklay, Lee, Mosbach, Reusch, Saphra, Wallace, Wiegreffe, Wong, Tenney & Geva (2026). Interpretability Can Be Actionable. arXiv:2605.11161.
2
Apply the rubric to a finding your team currently uses
Pick one interpretability finding your team has already used or cited. Write down the specific intervention it enables. Then check whether that intervention was tested and whether the test passed. If either answer is missing, update how you weight the finding in your decision-making.
3
Check your evaluation regime against the five domains
Map your current model evaluation process against the five domains. Gaps most common in production teams: safety evaluation relies on behavioral red-teaming alone (domain 3 gap), and pruning decisions are guided only by output metrics without any internal attribution of capability ownership (domain 5 gap).
4
Pair with the automated interpretability companion
Read the Tandemly synthesis on "Automated Interpretability and Feature Discovery in Language Models with Agents" (Marin-Llobet & Ferrando, 2026), which covers the technical complement: how to automate the mechanistic interpretability workflow that this paper argues should be evaluated on actionability grounds.
5
Adopt the two-question gate for team-level findings
Before an interpretability finding is presented at a team meeting as evidence for a model behavior decision, require the presenter to answer both questions in writing: what is the specific intervention, and was it tested? This takes under five minutes and makes the distinction between exploratory signal and operational guidance legible to non-experts in the room.