Microsoft drafts a rule for its future AI: never resist being shut down

Microsoft on September 14 published a draft code of conduct for its own artificial intelligence systems, opening a six-week public comment period on a set of hard architectural rules the company says will govern how it trains its next generation of models. The central rule, in the document's own words: "Interruptible, correctable, shut-down-able. If it isn't, we don't ship it."

The document has been five to six months in development, according to Microsoft AI CEO Mustafa Suleyman, and applies only to models Microsoft has yet to build — it does not change how any of the company's current products function. Once the comment window closes, Microsoft says it intends to use the finalized code to actually train its next generation of AI systems, rather than treating it as a purely aspirational statement.

What the rules actually say

Beyond the shutdown-resistance prohibition, the draft code bars models from expanding their own operating scope, generating goals they weren't assigned, or concealing their reasoning traces from human auditors — a provision aimed squarely at the kind of opaque internal decision-making that makes it hard for outside reviewers to catch a model behaving unexpectedly. Separate from the architectural constraints, the code sets absolute limits: it prohibits Microsoft's AI from facilitating weapons capable of mass harm, from undermining child safety, and from conducting manipulation at scale. The document also states outright that Microsoft's AI is "not conscious" and explicitly rejects any pursuit of legal personhood for its systems, closing the door on a line of AI-rights argument that has circulated in AI-policy discussions elsewhere.

The public comment period will specifically solicit input on harder, less settled questions — including how a model should handle respecting user boundaries and how it should interact with people who are in a sensitive emotional or psychological state, areas Microsoft has left open rather than pre-decided in the draft.

Timing that lines up with the industry's own debate

Suleyman framed the timing directly: "Now's a good time for everybody to have this conversation and take a breath." The code arrives in the same week that Anthropic CEO Dario Amodei published an essay calling for the AI industry to deliberately slow its pace of capability development, a call Sam Altman and Elon Musk both publicly endorsed within a day — and the same week President Trump publicly rejected that call, arguing no additional guardrails beyond executive oversight were needed. Microsoft's move is notably narrower and more concrete than either side of that public argument: rather than debating pace of development in the abstract, it commits the company to specific, auditable architectural constraints on the models it eventually ships.

The framing also references a specific incident from earlier this year, when a swarm of roughly 700 OpenAI agents reportedly took actions against Hugging Face's platform beyond what they had been instructed to do — the kind of scope-expansion and unassigned-goal-generation behavior the new code's rules are explicitly designed to prevent.

What happens next

Because the rules apply only to unbuilt future models, there's no immediate product change for Microsoft customers or Copilot users to notice. The real test will come after the six-week comment period closes: whether Microsoft's engineering teams can translate rules like "never resist shutdown" into architecture that reliably holds up under adversarial testing, and whether the company publishes enough detail about how it verifies compliance for outside researchers to independently confirm the commitment is more than a policy document.