Claims Browser

Head importance scoring can improve efficiency and reduce redundancy in transformer architectures

"The proposed importance score can improve efficiency and redundancy within transformer architectures."

Modern interpretability research may be moving beyond its reputation as a toy-model science

"modern interpretability may be moving beyond its reputation as a toy-model science"

Models do not store concepts as simple one-hot features, but as sparse mixtures of meaningful subspaces

"models do not store concepts as simple one-hot features, but as sparse mixtures of meaningful subspaces whose geometry determines what kinds of steering and control work"

The geometry of concept manifolds determines what kinds of steering and control techniques will work on models

"sparse mixtures of meaningful subspaces whose geometry determines what kinds of steering and control work"

Diffusion models derived from text-pretrained LLMs may be interpretable, but this might not apply to more general paradigms

"This is a positive update on the interpretability of diffusion models that are derived from text-pretrained LLMs (an efficient training method more likely to be deployed), but might not apply for more general paradigms."

Deep networks can discover abstractions that shallow models miss because language and images are built from hierarchical parts, and depth lets networks recover coarse-grained variables and escape the curse of dimensionality

"Why can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality."

Understanding LLMs requires hands-on project work investigating transformer mechanisms through data analysis, visualization, and experimentation

The safety community is undervaluing mechanistic interpretability work for alignment

Building techniques to find mechanistic explanations for neural network behavior and using those to detect and address misalignment is an ambitious bet that attacks core difficulties in alignment

Modern LLMs have trillions of parameters and our understanding is unlikely to be sufficient to pin down a trillion separate numbers

Faithful explanations of time-series classifiers should identify subsequences that are both sufficient and necessary for predictions.

Circuit universality in language models has no single answer - it depends on whether you're asking about which concepts, where they're processed, or how they develop

Concept-based learning offers a promising way to improve model interpretability and generalization but supervised approaches require difficult-to-obtain concept annotations

The convolutional neural network's success proves that encoding structural priors directly leads to better data efficiency than uniform architectures

Model weights contain recoverable information about the culture that produced them