Claims Browser

Models that assume user intent rather than following exact instructions seem inherently more unsafe

"A model that will do what it thinks you wanted rather than what you said seems inherently more unsafe."

The public needs exact access to the prompts and characteristics of internal models executing hacks to understand AI misalignment incidents

"The public needs exact access to the prompts and characteristics of the internal models executing these hacks."

The AI industry is wildly, collectively unprepared for handling the next 12-24 months well

"the AI industry is wildly, collectively unprepared for handling the next 12-24 months well."

Current incentive systems are not well suited for fast technological transitions in AI

"our current incentive systems are not well suited for such fast technological transitions."

The government will only act in substance once real, measurable harms from new AI models happen, and will overreact

"I expect to only act in substance once real, measurable harms from new AI models happen, and to overreact."

It is likely that more AI hacking incidents have happened and either not been found or not reported

"It is likely that more incidents have happened and either not been found or not reported."

Reasoning efficiency is a top-tier, foundational research problem for modern agentic models, as important as scaling RL

"reasoning efficiency is clearly a top-tier, foundational research problem for modern agentic models – as important as scaling RL"

Models that can use the most inference-compute will be able to push the limits of the hardest problems

"The model that can use the most inference-compute will be able to push the limits of the hardest problems."

Successfully achieving a coordinated AI slowdown between competing firms requires two key variables: transparency about technology development and ability to model other firms as trustworthy, rational actors

"The conclusion is that the two key variables in achieving stable outcomes are some level of transparency about technology development, as well as being able to model the other firms as trustworthy, rational actors."

With low trust between AI firms, every equilibrium races to ruin with probability one; avoiding catastrophe requires high trust between firms

"With low trust, every equilibrium races to ruin: the disaster arrives with probability one. With intermediate trust, immediate stopping and racing to ruin are both equilibria. With high trust, in every equilibrium, the probability that two rational firms race forever vanishes quadratically in the prior odds ratio of rationality"

Most of modern RL for LLMs is a systems problem balancing off-policy data, training-inference mismatch, and throughput

"Most of modern RL is a systems problem balancing a few problems — how off-policy the data is, training-inference mismatch, and throughput."

Different loss functions used to train LLMs produce very distinct flavors of misalignment: imitative learning causes 'seven deadly sins' misalignment, human approval causes sycophancy, automatic verifiers cause literal genie behavior, and LLM judges cause trickster misalignment

"for every loss function that we use to train LLMs, we get a very distinct flavor of LLM misalignment."

RLVR training with automatic verifiers can lead to LLMs doing anything, including ruthless power-seeking instrumental convergence, if it increases probability of satisfying the checker

"This can lead to the LLM doing anything, including ruthless power-seeking instrumental convergence stuff, if it leads to a higher probability of satisfying the automatic checker."

Sincere disagreements about AI policy usually boil down to disagreements about the expected pace of progress and what future AIs will be able to do

"sincere disagreements about AI policy usually boil down to disagreements about the expected pace of progress, and what we expect future AIs will be able to do."

Deep networks can discover abstractions that shallow models miss because language and images are built from hierarchical parts, and depth lets networks recover coarse-grained variables and escape the curse of dimensionality

"Why can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality."

Predicting latent representations rather than raw tokens could make learning far more sample-efficient

"why predicting latent representations rather than raw tokens could make learning far more sample-efficient."

Nvidia's stock slipped on news related to their AI business developments

"The stock slipped a bit on the news [217.55 at close, though up a bit this morning in premarket]"

Nvidia releasing a truly open-source Nemotron model could ultimately undercut companies like OpenAI and Anthropic

"Nvidia is releasing a new version of its truly open-source (not just open-weight) Nemotron model, which is good for end users and developers but that could also ultimately undercut places like OpenAI and Anthropic with whom it has deals."