Research
150 stories
AI Alignment ForumResearchAutomating subtle emergent misalignment research Authors: Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid, Evan Hubinger Abstract Red-teaming by humans is a vital part of model safety testing, but it may struggle to keep up as models improve.
InfoQ AIGitHubPresentation: Beyond the Harness: Canvas for AI Agents Nimisha Asthagiri shares how applying proven product engineering and architecture principles creates a bounded canvas for AI agents.
SemiEngineeringSiliconChip Industry Technical Paper Roundup: Nov. 3 New technical papers recently added to Semiconductor Engineering’s library: Technical Paper Research Organizations Meter-scale quantum interconnects between large quantum processors IBM Research…
SemiAnalysisSiliconGDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains GDPx is a benchmark that covers the most valuable domains for frontier intelligence. It aims to be the world’s best benchmark for measuring general agentic capabilities.
Microsoft ResearchResearchFrom synthetic enterprise data evaluations to real Microsoft Copilot AI confidence At a glance Evaluating Microsoft Copilot at enterprise scale requires test data that captures the full complexity of real organizations, yet real enterprise data contains sensitive information that…
Supabase BlogToolsIntroducing DBArena: open, reproducible benchmarks for hosted Postgres DBArena benchmarks hosted Postgres providers on price-performance, with a public methodology.
Vicki Boykis
Vicki BoykisVoicesSlides on retrieval and ranking with ONNX inference The wonderful DataPhilly meetup community asked me to come give a talk on any topic, and I talked for half an hour on embedding-based retrieval, machine learning model artifacts creation and…
Meta EngineeringLabsMeta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design This post describes the next generation of Meta Andromeda, Meta’s personalized ads retrieval engine, and how co-designing the model, system, and GPU hardware unlocks a new efficiency frontier for ads…
Replit BlogToolsReplit’s Architecture Behind Secure Agent Work Last week’s security disclosures offered a glimpse of the challenges ahead. Critical flaws surfaced across software that much of the internet depends on.
InterconnectsVoicesArtificial Analysis v5 and the benchmarker's dilemma Hill-climbing is easy, knowing what to hill-climb is hard. Making scores go up is expensive , but we have a much clearer playbook for it than we do for creating meaningful benchmarks at the true…
CSET GeorgetownResearchHow Black-Box Distillation Works Introduction If you asked Claude Mythos one hundred thousand questions about cybersecurity, would learning the answers make you better at hacking?
Import AI
Import AIVoicesImport AI 478: Math for alignment; ProgramBench; and Mirendil shows what RSI means Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.
Ahead of AIVoicesBuilding Reasoning Models from the Ground Up: A Full Video Course I have written a lot about reasoning models here over the last two years on Ahead of AI, including a recent article on mechanisms to control reasoning effort levels in contemporary LLMs .
Nature Machine Intelligence
Nature Machine IntelligenceResearchGuidelines for evaluating structure-based methods for protein–ligand binding prediction Nature Machine Intelligence, Published online: 23 October 2026; doi:10.1038/s42256-026-01331-8 Ballester and Tran-Nguyen advocate for transparency in data, code and experimental procedures in the…
Hamel HusainVoicesHow To Use Decision Models for Evals main.content table { display:block; width:100%; max-width:100%; overflow-x:auto; } #evaluation-prompt pre { background:var(--bs-body-bg); color:var(--bs-body-color); padding:18px; border:1px solid…
AI Snake OilVoicesNew paper: Emerging trends in AI agent evaluation By Felix Chen, Stephan Rabanser, Sayash Kapoor, and Arvind Narayanan. See the full author list and affiliations here .
Chips and CheeseSiliconNvidia Vera’s System Architecture: Pushing Bandwidth Nvidia’s Vera server chip aims for high per-core performance. Its Olympus cores have massive out-of-order structures and use high per-clock throughput to meet that goal within a high core count…
Mercari EngineeringJapanElasticsearch の Autoscaling を改善する: scale-out の高速化と HPA tolerance の調整 はじめに 検索基盤チームの mrkm4ntr です。以前、 CPU 使用率による Elasticsearch の autoscaling について書きました。ピーク時間帯に合わせて確保していたリソースを、トラフィックに合わせて増減することで、Elasticsearch のコストを約 40% 削減できました。 この autoscaling には、まだコストを削る余地が残っていました。
Cline BlogToolsCline Agents Harness Unlocks Benchmark-Leading Performance from NVIDIA Nemotron 3.5 Super VL Open weights models have become the new frontier of achieving intelligence more efficiently, and we are super excited to see how the latest NVIDIA Nemotron 3.5 Super VL model continues to push the…
Apple Machine Learning Research
Apple Machine Learning ResearchResearchSelection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action…
GitHub Copilot BlogToolsReviewBench: An open benchmark for AI code review Agentic code review is becoming an essential piece of how development happens. It helps you inspect pull requests, catch issues, and decide what deserves attention before code ships.
Last Week in AIVoicesLast Week in AI #345 - 5 new models, 9 misalignment incidents, some Dots Top News Anthropic and OpenAI race to release smarter and cheaper models Sources: Anthropic launches Claude Opus 5.5 with stricter safeguards for cybersecurity OpenAI launches GPT-6 Sol and Luna…
Hamel HusainVoicesClaude’s new auto eval tool Anthropic released new eval tooling for Claude Code . Their claude-api plugin now includes a new build_eval and hill-climb command that helps you build evals, check the graders, and improve your…
Weights & Biases
Weights & BiasesToolsComparing Modern Transformer Architectures at GPT-2 Scale with ARIA and W&B
Nothing matches this filter yet.