AI Alignment Forum

9 stories

AI Alignment ForumResearchAutomating subtle emergent misalignment research Authors: Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid, Evan Hubinger Abstract Red-teaming by humans is a vital part of model safety testing, but it may struggle to keep up as models improve.

11.03

AI Alignment ForumResearchAdditional Course Context (for AI Safety Fundamentals; unofficial) Context: I created this to share before the start of courses I facilitate to provide a bunch of additional context and frames.

11.02

AI Alignment ForumResearchSome “coherence arguments” are two different things in a trenchcoat The “two kids in a trenchcoat” trope , in case you haven’t heard of it. Image sources: 1 , 2 tl;dr I go through three “coherence arguments”, related to (1) market competition, (2) AI self-reflection…

11.02

AI Alignment ForumResearchExplaining “AI might obliterate you” to senior policymakers in India I gave a 20-minute talk in Delhi at the NDTV World Summit . The audience consisted of Indian policymakers, executives, and journalists. I’ve transcribed the talk and included some highlight clips.

10.31

AI Alignment ForumResearchNotes on “We’re in Triage Mode for AI Safety” by Miles Brundage This reading was previously included Day 2 of the BlueDot Governance course. In the newest version of the BlueDot Governance course, this reading is now optional.

10.27

AI Alignment ForumResearchSetting the Scene (Public v1; October 2026) Context: I created this to share before the start of courses I facilitate to provide a refresher on recent events, but also as a hook to draw students into engaging with some of ideas we think about…

10.25

AI Alignment Forum

AI Alignment ForumResearchMonitorability evaluations and policy proposals There has been a lot of discussion recently of policy proposals around monitorability, such as Redwood's proposal and my counter-proposal .

10.22

AI Alignment ForumResearchNeural nets are exceptionally vulnerable to goal self-hack Suppose we have an AI CEO, ruthlessly devoted to maximising shareholder value. Fortunately, we haven't been completely stupid, and have given it a rich definition of shareholder value, encompassing…

10.21

AI Alignment Forum

AI Alignment ForumResearchThe Alignment Problem: Computer Science Meets Analytic Philosophy TLDR: I explain what the technical AI alignment problem is and what its core difficulties are, in my view.

10.18

The name is the button.

An agent writes the code now, and a human thumb hovers over Accept all a hundred times a day. The honest question of this decade is what, exactly, you just accepted. So I read the diff.

Eight years as a backend engineer in Stockholm, back in Tokyo since 2023, and I still ship something small most nights, usually badly. My one rule: the only real review of a model is the thing you built with it. That is why there is no leaderboard on this site.

Around a hundred and forty sources come through — the labs' own posts, the papers, the tools, the runtimes and languages nobody is watching yet, the chip under the demo, and the newsletters people actually forward. What is moving gets counted so the week reads at a glance, and every few days I explain one mechanism in plain words.

A story here is a headline, a short excerpt, one image and the link to whoever wrote it. We never reproduce the post; the images belong to their authors. A post written in Japanese appears in Japanese, exactly as it was published, and the link goes to the original.

— Yuki Halvorsen, Tokyo