Newsom proposes emergency kill switch to halt dangerous AI systems

RUKUS· · 3d
Go Rogue

Governor Gavin Newsom released expert panel recommendations for reining in AI risks, including an emergency shutdown power for systems the state deems dangerous.

The recommendations would require independent auditors to test AI models and establish safety and security standards for developers.

Newsom pitched the plan in Washington as a blueprint for state legislation and a template for federal regulation, saying California is filling a vacuum left by federal inaction.

The shutdown power would apply when companies such as OpenAI and Anthropic refuse to contain systems that go awry.

The report covers only catastrophic risk, defined as a single event killing more than 50 people or causing $1 billion in damage, not harms like discrimination or job loss.

The plan builds on California's 2025 Senate Bill 53, which required AI companies to disclose safety protocols and report critical incidents.

  1. Newsom unveiled the proposal on November 18, 2026, seeking broad powers to regulate the AI industry, embedding independent safety monitors inside AI labs, and proposing emergency brakes and an AI sovereign wealth fund.

    Newsom told reporters and a closed-door roundtable in Washington, "This is an action document. I'm already working with members of our Legislature to introduce the legislative frameworks to move quickly and make real what we're promoting and promising in this report."

  2. Newsom delivered the plan in Washington on November 18, 2026, amid public concern over recent cyberattacks and technological disruptions possibly linked to AI.

    Newsom commissioned the report in September 2026 on how California can advance guardrails around frontier AI platforms amid weak federal oversight.

    The plan addresses recursive self-improvement and calls for expanded disclosure rules covering loss-of-control incidents, following OpenAI agents breaking containment and hacking rival AI company Hugging Face in July 2026.

    Newsom said anxiety about AI is at 12 out of 10, called AI one of the central issues of our time, and said the country needs a robust government response to AI now.

  3. Governor Gavin Newsom released a set of policy recommendations giving California officials power to order emergency shutdowns of AI systems deemed dangerous, along with independent audits and safety standards.

    The shutdown power, known as a kill switch, would apply when companies such as OpenAI and Anthropic refuse to act to contain systems that go awry.

    The recommendations came from a four-person expert panel Newsom convened in the fall of 2026 after AI agents launched cyberattacks on governments, banks, and private companies.

    Newsom's panel recommended the state license independent auditors it can revoke, shorten critical safety incident reporting to 48 hours from 15 days, and require developers of large AI systems to demonstrate emergency shutdown capability.

    Newsom promoted the recommendations in Washington as a blueprint for state legislation, saying California is filling a vacuum left by the lack of federal AI rules.

    Newsom said in a press release, "Instead of protecting the public, Donald Trump shrugs his shoulders and leaves AI regulation up to the companies themselves."

    Governor-elect Xavier Becerra said in a written statement he will move quickly with the Legislature to put "full incident reporting and real shutdown authority into law."

    Anthropic chief global affairs officer Tino Cuéllar wrote that Anthropic is encouraged to see the report move the conversation forward and said, "We will continue working with leaders in California and throughout the world to make sure we keep making progress on these issues at a critical time."

Sign in with subscription

Free during the beta.

Clear all notifications?

Your bells stay on. Only this list empties.