The fear always arrives as the same word. Rogue. A leadership team reads that an AI agent somewhere deleted a production database or quietly leaked records for two weeks before anyone noticed, and the question lands in my inbox: how do we keep our agents from going rogue. Fair instinct, wrong frame. Almost nothing that gets labeled a rogue agent is actually rogue. It is an over-permissioned, under-audited agent doing exactly what it was built to do. That should worry you more than the sci-fi version, because the sci-fi version isn’t the one showing up in incident reports.
This is where AI agent governance stops being a slide in a deck and starts being the difference between a tool and a liability. If you are a prime evaluating a subcontractor, or an IT leader being asked to approve agents inside a government tenant, the useful question isn’t “could this go rogue.” It’s “who scoped what this thing is allowed to touch, and can we prove what it did after the fact.” Those are answerable questions. They just require someone to do the unglamorous work before deployment instead of during the breach.
The word “rogue” is doing too much work
At the end of 2025, OWASP published the Top 10 for Agentic Applications, the first serious taxonomy of how these systems actually fail. Rogue agents sit dead last on that list, ASI10, and the placement is the whole story. The category covers genuinely misaligned or compromised agents that keep acting after they should have stopped, impersonate other agents, or persist across sessions doing harm. Real, and rare. The nine failure modes ranked above it are where the incidents come from: goal hijacking, tool misuse, identity and privilege abuse, supply-chain compromise, unexpected code execution, memory and context poisoning, insecure agent-to-agent messaging, cascading failures, and humans trusting a confident, polished, wrong answer.
Read that list again and notice what it is. A catalog of engineering and governance gaps, not a catalog of machine uprisings. An agent does not need malice to cause damage. It needs autonomy, access, and a small shove in the wrong direction. One poisoned document in a SharePoint library the agent was told to trust will do it. A connector with broader scope than the task requires will do it. An indirect prompt injection buried in an email the agent was asked to summarize will do it, and to your security team the telemetry will look indistinguishable from normal use. Industry surveys through early 2026 put the share of organizations that have already seen agents behave in unintended or unauthorized ways near half. The agents weren’t rebelling. They were badly built and loosely governed.
What actually goes wrong
Three failures account for most of what leadership is afraid of. First, over-permissioning. An agent gets handed a human’s session or a service account with far more reach than its job needs, so a task scoped to “answer HR policy questions” technically has the keys to send mail, write files, and call admin APIs. Second, prompt injection. The agent ingests untrusted content as part of its normal work and treats an instruction hidden in that content as if it came from you. Third, no audit trail. When something goes sideways, nobody can reconstruct which tool the agent invoked, against which data, on whose authority. That last one is the quiet killer in a regulated environment.
“The model decided” is not a defense in an audit, a deposition, or a public records request.
NIST’s AI Risk Management Framework names the structural reason this is hard. An agentic system can launch a chain of irreversible actions, deleting data, sending communications, changing configurations, before any human observes that it has gone wrong. The gap between action and observation is a new risk dimension, and so is delegation: when one agent spawns sub-agents, accountability for the whole sequence smears out across the chain. NIST is building agentic guidance to close that gap, with an interoperability profile expected later in 2026, but you do not get to wait for it. The agents are already in the tenant.
The unglamorous engineering that prevents it
The governance principle worth internalizing is least agency. Least privilege controls what an agent can access. Least agency controls how much freedom it has to act on that access without checking back with a human. Autonomy is a capability you grant deliberately for a specific task, not a default you leave switched on. Give an agent a blank check to “figure out the best way” and you have built an insider threat that a single malicious prompt can recruit.
In practice, governing an agent properly means a short list of things done well. The agent gets its own identity with scoped permissions, not a borrowed human login, so you can see and restrict exactly what it can reach. Retrieval is citation-bound, so answers trace back to source documents instead of being generated from the model’s confidence. High-consequence actions sit behind a human approval gate rather than firing autonomously. Data-loss policy is re-validated at the moment of the call, not just at design time. Every tool invocation is logged with identity, timestamp, and outcome. And there is a kill switch that actually revokes access across the deployment, tested, not theoretical. This is the work behind every custom AI agent I build, and it is the same discipline I bring to Copilot Studio development work, where the platform now exposes agent identities, sensitivity labels, runtime policy, and audit logging that most teams have not turned on or configured correctly.
None of this is exotic. The natural-language-to-Graph administrative agents and citation-bound policy agents I have engineered for production government environments are governed exactly this way: scoped identity, bounded tool set, gated writes, full audit. The result is an agent that is useful and boring at the same time, which in regulated work is the highest compliment available.
Why GCC breaks the commercial playbook
Here is where most agent guidance falls apart for government buyers. It assumes you are in commercial Microsoft 365. GCC (Government Community Cloud) is a different animal, and the difference is not cosmetic. The newest models and the shiniest governance features land in commercial first and reach GCC later, sometimes in a different shape, sometimes not at all. As of writing, several flagship orchestration models are generally available everywhere except GCC, and several of the agent-governance controls being demoed at conferences are commercial-first previews. If your agent design assumes a model or a control that GCC does not have yet, you do not have a compliant design. You have a roadmap dependency wearing a compliance label.
The constraints are the point, not an obstacle to route around. Solutions for these environments have to be architected to operate inside Microsoft’s FedRAMP-authorized GCC boundary and aligned to CMMC and NIST 800-171 control objectives, with data residency and audit expectations that commercial tenants never have to think about. A commercial agent pattern lifted into GCC and declared compliant is how agencies end up with the exact ungoverned-agent problem they were trying to avoid. Building for GCC from the start, then applying the same rigor to commercial M365 with lighter procurement overhead, is the model I work in. The discipline does not change between the two. The constraints do, and pretending otherwise is how you fail an assessment. This is the core of my Washington government AI consulting and the broader automation and governance work I deliver.
Govern it and build it, same engineer
Most of the market splits this in two. One firm writes the governance policy, a different one builds the agent, and the gap between the document and the deployment is exactly where the rogue-agent story gets written. I do both. I am a solo, veteran-owned small business (VOSB), which means the person scoping the controls is the person writing the orchestration and signing off on the audit logging. No account manager relaying requirements to an offshore build team. Direct access to the engineer who is accountable for what the agent does.
For leaders who need their people fluent before anything ships, I run GCC Copilot and Power Automate training workshops that cover what these agents can and cannot safely do. For organizations that want a governed agent in production rather than a slide deck about one, the GCC AI Jumpstart scopes, builds, and hands off real workflows with a governance framework attached. And for teams standing up Microsoft 365 Copilot who want it done without the over-permissioned shortcuts, that is the entire job.
There are no rogue agents in well-governed environments. There are only the agents you scoped, the permissions you granted, and the audit trail you can produce when someone asks. If you are a prime that needs a sub who can do this correctly the first time, or an IT leader who would rather govern agents than explain one, let’s talk.
Written in a personal capacity. Views are the author's own and do not represent any employer. No client or employer systems, data, or configurations are described.