Claims Browser

DiffusionGemma maintains high monitorability despite having opaque serial depth

"Recently, Engels et al. found that DG nevertheless maintains high monitorability, for instance by showing that projecting the distribution to its top-k items largely retains performance."

Good performance can be maintained with only the top item in DiffusionGemma, supporting high monitorability

"We strengthen these results by showing that this performance degradation is largely a sampler artifact and good performance can be maintained with only the top item, supporting the case for high monitorability."

Interpretability techniques like probes, steering, and J-lens carry over to DiffusionGemma with performance largely retained

"we also examined how interpretability techniques carry over to DiffusionGemma, including probes, steering, and J-lens. We find that performance is largely retained."

Diffusion models derived from text-pretrained LLMs may be interpretable, but this might not apply to more general paradigms

"This is a positive update on the interpretability of diffusion models that are derived from text-pretrained LLMs (an efficient training method more likely to be deployed), but might not apply for more general paradigms."

Global workspace theory lacks a formal criterion for identifying the mechanism that enables conscious access

"Global workspace theory explains conscious access as the broadcasting of selected information to the rest of the network, but it lacks a formal criterion for identifying the mechanism that enables this access."

R-lens directions for intermediate variables are more causally important than J-lens directions

"R-lens directions for intermediate variables are more causally important than J-lens directions"

R-lens is a drop-in replacement for J-lens that produces clearer readouts on earlier layers of neural networks

"We introduce the R-lens: a drop-in replacement for J-lens that produces clearer readouts on earlier layers"

J-lens readouts in early layers are often noisy and largely uninterpretable

"we find readouts in early layers to often be noisy and largely uninterpretable"

R-lens shows a substantial quantitative advantage over J-lens that increases as models scale

"R-lens shows a substantial quantitative advantage over J-lens that increases as models scale, measured across a variety of evaluation categories"

R-lens produces qualitatively different readouts at earlier layers, significantly reducing incoherent tokens compared to J-lens

"R-lens produces qualitatively different readouts at earlier layers, significantly reducing the amount of incoherent tokens compared to J-lens"

R-lens can capture concepts that J-lens never does, especially concepts appearing exclusively in early layers

"R-lens can sometimes capture concepts that J-lens never does, especially if these concepts appear exclusively in early layers"

Deep networks can discover abstractions that shallow models miss because language and images are built from hierarchical parts, and depth lets networks recover coarse-grained variables and escape the curse of dimensionality

"Why can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality."

Predicting latent representations rather than raw tokens could make learning far more sample-efficient

"why predicting latent representations rather than raw tokens could make learning far more sample-efficient."

Understanding LLMs requires hands-on project work investigating transformer mechanisms through data analysis, visualization, and experimentation

ARC will likely grow rapidly over the next few months

The safety community is undervaluing mechanistic interpretability work for alignment

Building techniques to find mechanistic explanations for neural network behavior and using those to detect and address misalignment is an ambitious bet that attacks core difficulties in alignment

In sparse mixture-of-experts language models, expert subspaces overlap substantially despite the expectation that co-selected experts should contribute distinct representation directions