Claims Browser

Deep neural networks often exploit spurious associations in their training data through shortcut learning

"Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning."

Concept-based explainability methods can mistake correlations between concepts as evidence that the model uses them because they evaluate each concept in isolation

"Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them."

ICON decomposition recovers concept importance more accurately than seven alternative baseline methods on synthetic data with known ground truth

"On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods."

ICON decomposition can isolate concepts that models genuinely rely on and quantify unexplained representation in real-world imaging models

"On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing."

Existing post-hoc explainability methods for time series forecasting often ignore temporal dependence and fail to provide horizon-specific explanations

"existing post-hoc methods often ignore temporal dependence and fail to provide horizon-specific explanations"

A model-agnostic framework using semantic flow can explain forecasting predictions by attributing each forecast horizon to temporally relevant historical lags

"We propose a model-agnostic explainability framework that explains forecasting predictions by attributing each forecast horizon to temporally relevant historical lags"

The semantic-flow variant achieves competitive or superior faithfulness compared to standard post-hoc baselines while being substantially more computationally efficient

"the semantic-flow variant achieves competitive or superior faithfulness compared to standard post-hoc baselines, while being substantially more computationally efficient"

Stability analysis demonstrates that the semantic flow explanations are robust and can identify regimes where interpretation should be applied with caution

"Stability analysis further demonstrates that the explanations are robust and identifies regimes where interpretation should be applied with caution"

Goodfire has built Silico, a research platform priced at $1,000 per month

"Silico, the $1,000-per-month research platform Goodfire built for itself"

Fine-tuning and reinforcement learning often amplify behaviors that are already latent in pre-training

"fine-tuning and RL often amplify behaviors already latent in pre-training"

Interpretability techniques can identify the data and features driving unwanted model updates

"interpretability can identify the data and features driving unwanted updates"

Models do not store concepts as simple one-hot features, but as sparse mixtures of meaningful subspaces

"models do not store concepts as simple one-hot features, but as sparse mixtures of meaningful subspaces whose geometry determines what kinds of steering and control work"

The geometry of concept manifolds determines what kinds of steering and control techniques will work on models

"sparse mixtures of meaningful subspaces whose geometry determines what kinds of steering and control work"

Modern interpretability research may be moving beyond its reputation as a toy-model science

"modern interpretability may be moving beyond its reputation as a toy-model science"

Steering techniques can fail when applied off-manifold from the concept subspaces

"why steering can fail off-manifold"

Closing the normalization path multiplies the value of enlarging receptive field by up to an order of magnitude

"Closing the path, by taking the same statistics per position, multiplies what enlarging the receptive field is worth by up to an order of magnitude on simulated genomes at every difficulty level tested and on real 1000 Genomes haplotypes."

A graphical notation adapted from Penrose tensor notation can provide a global view of interpretable AI architectures and map directly to PyTorch einsum code

"This graphical notation gives a global view of an architecture and maps one to one onto PyTorch einsum code."

The diagram of Steerling-8B architecture translates into just 33 lines of PyTorch code

"The diagram yields global insights into the architecture (e.g., showing that Steerling is a residual model), a geometric interpretation of each individual operation, and a direct translation into 33 lines of PyTorch code."