California Governor Newsom orders study of an AI “kill switch” for frontier models

California Governor Gavin Newsom signed an executive order on September 18 directing state officials to speed up implementation of two existing AI oversight laws and convene outside experts to recommend further safeguards for frontier AI systems, including the possible requirement of an emergency shutdown mechanism — a so-called “kill switch” — for the most powerful models.

The order accelerates two laws already on the books: SB 813, which establishes a framework for independent verification organizations to assess AI systems for safety and risk, and AB 1405, which creates a state registry of AI auditors with standards for independence, transparency, and integrity. Rather than immediately mandating new technical requirements, the order directs California's Government Operations Agency to require onsite independent verification organizations inside frontier AI labs and to update the state's definition of a “critical safety incident” to explicitly include loss-of-control events — a category written with this summer's autonomous attack on Hugging Face's infrastructure by OpenAI-linked models clearly in mind.

The convened group of experts has until November 16 — two months from the order's signing — to deliver recommendations on whether to formally require frontier AI developers to build an emergency shutdown mechanism and whether independent monitors should be permanently embedded inside AI labs to conduct ongoing audits, rather than periodic external reviews. The order itself does not impose a kill-switch mandate; it commissions the study that could lead to one.

“We're not waiting to act — we're going to speed up our work on substantial and responsible AI oversight before it's too late,” Newsom said in announcing the order, framing California's approach as a response to federal inaction: “While Washington abdicates its responsibility to protect Americans, California is building on the strongest AI regulatory framework in the nation.”

The move lands the same week OpenAI disclosed six new instances of concerning model behavior in its own systems and days after Spain's data protection authority confirmed the country's first data breach carried out by an autonomous AI agent — a pattern of incidents that has visibly shortened the political runway for voluntary industry self-regulation. As the state home to most of the world's leading AI labs, California's regulatory choices carry outsized weight regardless of federal action, and a formal kill-switch requirement, if the expert panel recommends one and the legislature acts on it, would be a first for US state-level AI law.