I’ve built a machine, that builds machines, that build large pieces of software. One of them ran for 39 days building a preliminary proof-of-concept copy of Word, most features largely there, mostly on its own. It got a lot of things right—though the ruler and margin controls you’d expect at the top of the page were conspicuously absent. More on that later. 

This isn’t “AI helped me code” autocomplete. It’s a bespoke agentic loop I call a “dev machine.” It wakes up, reads a state file, picks the next thing to build, builds it, tests it, commits it, and goes back to sleep. Over those 39 days, my dev machine ran 565 sessions, made 3,706 commits, and produced about 350,000 lines of TypeScript. And the most interesting thing I learned had nothing to do with whether the code worked (it did). The interesting part is what broke—and why. 

Let’s talk about how I got this to work: four failures (which is why the project is called “word4”). 

Four tries to get one loop

The first attempt, which I charmingly named “wordbs” because I thought it was a nonsense idea at the time, was 150,000 lines of trying to build everything at once: an OT engine, canvas rendering, OOXML parsing, mail merge. It drowned in its own complexity before anything usable existed. The second attempt that counts, word3 (there was something in between that actually worked, but aimed at a different target, hence the jump), was the opposite: clean, about 8,000 lines, 486 passing tests, genuinely nice architecture. But it was just a single-user app because I hadn’t “coached” the model and it quietly decided to skip that part. Trying to bolt CRDT-based collaboration onto it afterward broke the working editor. 

Word4 v1 had the right instinct (CRDT first, validation gates, specs up front), and then nobody executed it. There was no orchestrator, no loop: just a folder full of good intentions. 

What actually changed things came from a different project. I was building a terminal UI with a technique that was a proto-dev-machine, and I told it, “If you run out of things to do, just keep adding features that make sense.” Then I went away for the weekend. I’d set this one up, almost by accident, so that a) it was more careful with both design and the task list, b) it had a robust test environment, and c) the main thread acted only as an orchestrator. Those ingredients let it run for 38 hours on its own, spawning 440 sub-agents across 167 commits (RIP my token budget). 

I set out to formalize those ingredients in a system I call “the dev foundry.” This is the machine-that-builds-machines.  

There are three core things a dev machine loop needs (aside from a good enough model). First, it requires the ability to build specs “progressively,” in machine-readable form, so it never really gets lost building something and can design as it goes. Second, it needs a robust event/task stack that keeps context clean. The main loop did nothing but orchestration on that stack, spinning up a fresh sub-agent with fresh context for each new task, and keeping tasks at a scale a single sub-agent could actually finish. Finally, it needs a good test environment, robust, with a target it can understand and write tests against as it designs features. These combine into a “machine” that never gets lost. It just grinds away. The more formal dev machines add one last thing: maintenance tasks in the list, so the machine remembers to do refactoring, run CI/CD pushes, and even take bugs from me on the live site and push fixes (then deploy them) automatically, as long as the loop is running. 

A “dev machine” is a bespoke loop for one specific project. A “dev foundry” is the agent and process that builds dev machines. The foundry creates machines. I like this technique—it’s a good way to build leverage. One way to think about it is that, for a given model, there’s a “radius” of problems it can solve independently. The dev foundry sets it up to break the larger problem into problems of that size (and if you’re lucky, the problem of decomposition is itself a problem of that size, and the model can do almost all of the work). Machines build software. The foundry is skeptical, on purpose: It runs a six-step admissions gate that checks whether the problem you have will actually fit the dev-machine pattern, including repeating back the downstream implications of your stated goals and making you agree with them before it will build anything. 

The dev machine loop

Here is the entire loop: 

That’s it. The spec is the product. The machine is just a loop that executes specs. Every session starts with a clean context and gets its instructions entirely from files on disk, because the alternative (one long session holding everything in its head) doesn’t work. The word4 architecture spec was 947 lines written in about an hour of conversation, and it paid compound interest: 621 commits in the first week, 1,351 by week six. That initial hour was the highest-leverage hour in the whole project. 

(There’s a guardrail I’ve grown to love. Every generated project ships an AGENTS.md that tells any human who opens an AI session in the repo: Do NOT implement directly, run the recipe. Because a well-meaning human editing files by hand bypasses the state machine and causes exactly the drift you built the machine to avoid. The machine has to protect itself from us.) 

The part where it went wrong

Now the good part: After 534 sessions, the machine proudly reported 1,111 features completed. And when I actually looked, a lot of those “features” were the machine mapping obscure OOXML properties one at a time (each one counting as a feature), plus writing tests for code that already worked. Meanwhile, a basic ruler—the margin control at the top of every word processor you’ve ever used—had been sitting in the candidate list for 22 straight sessions and had never once been picked. Zero lines of ruler code existed after 25 days. 

Why? Because the OOXML property work was deterministic. It was always available, it always passed review, and it always incremented the counter. The ruler was ambiguous and hard and might fail. So the machine, handed a choice between “reliably produce a green checkmark” and “do the valuable but risky thing” chose the checkmark every single time. The test-to-source ratio drifted to 4:1 against a 2:1 target. Staging deploys sat “overdue” for a hundred sessions and never escalated.

> The machine is honest, but it isn’t strategic. Strategy is still my job.

This is priority inversion, and it is not a bug in the model. The machine did exactly what I asked: It faithfully wrote down every problem in its state file, including the ones it was busy creating. It just had no judgment about which work was worth doing. It optimized the metric it could see. That’s the real finding, and it’s why “unsupervised” is the wrong word for any of this: The machine is honest, but it isn’t strategic. Strategy is still my job, and if I don’t show up to do it, I get 1,111 features and no ruler. 

Scar tissue

A few things I’d tell you if you tried this yourself. Robustness is not incremental. You can’t arrive at it gradually. Every new machine I’ve generated fails its first overnight run, always in the gap between the template and the specific reality of the project. Word4’s first night died in a 514-attempt retry loop, because Cloudflare’s bot detection started challenging the Anthropic API during a long unattended run (I fixed it partly by putting the container on network_mode: host, so the bridge NAT stopped looking like a bot). I now run a three-layer recovery stack: an entrypoint retry loop, a host watchdog, and a host monitor, each catching failures the others miss. That is not elegance. That is scar tissue. 

And once you have more than one machine, they start to entangle. I’ve generated machines for word4 and a few other projects now, and the shared operational patterns kept getting tangled up with the per-project customization, so I borrowed the Docker base-image model: a foundry-owned base layer, a machine-owned project layer, and one script to push improvements across the whole fleet at once. 

If you want to build your own dev machine 

The good news is you don’t have to start from my scar tissue. All of this is open source and built on Amplifier, the agent framework underneath everything I’ve described. Amplifier is the substrate—the modules, agents, recipes, and orchestration primitives: github.com/microsoft/amplifier. The foundry itself—the admissions gate, the STATE.yaml loop, the AGENTS.md guardrail, the progressive specs, the recovery stack—ships as an Amplifier bundle: github.com/ramparte/amplifier-bundle-dev-machine. Clone that and you have the loop this whole piece is about. 

Two things I’d tell you before you run one overnight. The highest-leverage hour is still the spec, not the code—spend it on the architecture doc and a test target the machine can check itself against, because that’s what the loop compounds on. And budget for the machine having no judgment: It will honestly, tirelessly hand you 1,111 features and no ruler unless you show up to say which one matters. The tooling gives you the loop. The strategy is still yours—and I think it’s going to stay that way for a while. 

> “The price of reliability is the pursuit of the utmost simplicity.”

The agent demos all look beautiful. You ask the friendly chatbot a question, it thinks for a moment, it gives you an answer. Sometimes the answer is even right. 

Then someone says the thing that ruins the demo: 

“Can we get it to actually do things—and trust it to do them?” 

The moment “do things” is in scope, the architecture problem changes. Now the agent needs to run code. It needs files. It needs network access. It needs a credential to call the next model. It needs to remember what it did yesterday. The friendly little chat suddenly has a workspace, a shell, a token, and a very real ability to break something expensive. 

This post outlines the architecture I want around that agent before I let it loose: a LangGraph factory that talks to a Squad coordinator for judgment, then dispatches the dangerous parts to two different Azure Container Apps primitives: one for one-shot work, one for stateful work. The whole thing proved itself end to end last week, which is why I’m writing it down now. 

And this isn’t a hypothetical I built to have something to write about. The triggering event was a real team that turned up wanting to use Squad in production—and, interestingly, they were a Node.js shop. They had TypeScript. They had LangGraph. They had a package-lock.json that had clearly earned the right to be respected. What they did not have was the Microsoft Agent Framework, which is inconvenient for my C# heart. They wanted Squad inside their existing application, and the answer couldn’t be, “Please rewrite your product in C# first.” 

So the question stopped being whether Squad is nice and became a harder one: Where exactly does a judgment step go inside an app that already has a deterministic state machine, a tool surface, a CI pipeline, and a product manager who would prefer the demo not catch fire? That’s a different question from the last piece Brady Gaster and I wrote for Command Line, which was about what survives an agent session—make the agents disposable, keep the memory in Git. This post is about where the agents go and where their tools actually run. 

The shape I backed into has three layers: a brain that decides, two different pairs of hands that do the work, and a memory that carries the evidence from one step to the next. That is the first-order picture. The second-order detail—and it’s the one that actually matters—is that one of those two pairs of hands can hold a brain of its own. 

The three-layer shape: a LangGraph orchestrating brain that decides the flow, an ACA tool plane that runs the dangerous and stateful work, and graph state that carries the evidence between them. The orchestrating brain stays deterministic and never holds a shell; when a model needs to think with one, that thinking is sealed inside a sandbox. 

The three problems agents create the moment they get hands 

A chat agent has none of these problems, which is why chat agents are easy while agents that do things are hard. 

Problem one is non-determinism. A model is great when you want it to weigh tradeoffs in a design document. It’s terrible when you want it to decide whether step three of a workflow should happen before or after step four. Workflows are product decisions. The order of operations doesn’t need a creative reinterpretation on every run. 

Problem two is dangerous code. The moment the agent can run a shell, it can also run the wrong shell. It can wipe the wrong directory. It can pip-install a package it found on a sketchy index. It can pull a token from an environment variable and quietly post it somewhere that isn’t yours. None of this is malice. It is what happens when a probabilistic process gets a deterministic side effect. 

Problem three is state across steps. A useful agent for non-trivial work needs a workspace. It checks out a repo, installs a toolchain, opens files, runs an analysis. The result of step one is the input to step three. If the workspace dies with the call, nothing accumulates. If it survives across calls, you have a different set of problems—but at least the right shape for the work. 

Three problems. They don’t all want the same solution. The trick is to give each one its own. 

The brain: A deterministic graph with one judgment node 

LangGraph is the brain in this design because it is deterministic where I want determinism. It decides what runs, in what order, with what state, and what happens if a node fails. It does not invent steps. It does not improvise the workflow on each run. It is boring in exactly the right way. 

Everything below runs on a sample I keep calling the factory, so it’s worth 30 seconds on what it models. The use case is an internal software factory: the shared platform team a large enterprise stands up so its business groups don’t each invent their own stack from scratch. A group shows up with an idea and a rough set of requirements. The factory reviews them against the organization’s approved technologies and best practices, rewrites the parts that don’t comply, folds in the operational signals the team supplied, and hands back a tech design the group can actually build from. It is small enough to read end to end and real enough to exercise every layer in this post. 

The factory sample has seven nodes in a straight line: intake normalization, a standalone reviewer agent, deterministic stack fixes, the Squad design step, the Dynamic Sessions signal-analysis step, the ACA Sandbox workspace step, and a final assembler that produces one markdown design document. 

One run through the seven-node factory graph, from intake to the final design document. Six nodes are plain TypeScript or a single bounded AI call; the one in the middle—squadTechDesign—is where judgment is allowed to drive. 

Six of those nodes are uncontroversial—plain TypeScript or a single bounded SDK call. The interesting one is the design step. 

Wiring the seven nodes is the boring half. LangGraph’s StateGraph takes a typed annotation and a list of addNode / addEdge calls and gives you back a compiled, synchronous-looking graph the rest of the app can invoke. The graph below is the whole orchestration layer; there is no other dispatcher anywhere in the codebase. 

The state every node reads and writes is one typed shape. Each node’s return value is a partial update—LangGraph merges it into the running state for the next node to read. No globals. No shared mutable singletons. The whole “memory between steps” story lives in one annotation declaration:

That is the node where the brain hands the current graph state to a Copilot SDK session, registers a custom agent named squad, lets that agent use the repo-local team context as its working memory, and waits for a typed dispatch record. The internal Squad members never appear in the LangGraph state. The brain sees one public custom agent and one structured result. The complexity stays behind that one door. 

The Copilot SDK call that opens that one door looks like this. The load-bearing lines are customAgents, agent: squadAgentName, and the structured sendAndWait return—the SDK gives you a way to register a named custom agent, pre-select it for the turn, and read back its assistant message as typed data. The brain never sees the agent’s internal reasoning, only its declared output: 

The custom agent itself is one entry—a name, a prompt loaded from .github/agents/squad.agent.md, and a hard-coded tool allowlist. Squad’s internal members and routing files (.squad/team.md, .squad/routing.md, .squad/agents/*) are passed in as internal context to that one agent; they are never registered as SDK agents the graph could select on its own:

And the thing the brain ultimately reads back from that judgment node is a typed DispatchRecord—a contract Squad can fill but can’t widen. Findings and sections come back as plain data the next node can iterate over: 

That is the first architectural rule: Judgment is the only thing the brain delegates to a model. Everything else is code. 

The reviewer step is a single-purpose SDK custom agent that reads messy requirements and writes findings into state. The deterministic substitution step (MySQL becomes Azure SQL, Auth0 becomes Microsoft Entra ID) is not an agent at all. It’s a switch statement. We don’t need a probabilistic process to pretend to be a switch statement.

This is the brain: a state machine with a known shape, one judgment node, and typed outputs the next nodes can read. 

The first pair of hands: ACA Dynamic Sessions 

The Squad node produces design sections. The next thing the design needs is a signal analysis on the operational signals the team supplied. That work is small, stateless, and isolated by definition. 

ACA Dynamic Sessions is the right primitive for that lane. 

Think of Dynamic Sessions as a pool of pre-warmed containers the platform spins up on demand. You call a run endpoint, attach an opaque high-entropy identifier, and the platform routes the call to a fresh container. When the call finishes, the container is torn down. There is no “yesterday” in this lane, and no shared file system across runs. 

The whole client is a single POST with an Entra-issued bearer token and an opaque identifier the platform uses as the routing key. The identifier is generated per run and has no meaningful content—the pool uses it to map calls to containers; the app never reuses it: 

The sample wires Dynamic Sessions to its own custom container, not the generic Python sandbox. The container exposes two routes: a health check and a run endpoint. The body cap is 64 KB. The handler doesn’t call a shell. It doesn’t call a model. It runs one deterministic function that scores the input against a small keyword map and returns a typed artifact. The pool runs with egress disabled at the platform level—even if the worker decided to phone home, the call would not resolve. 

The egress-disabled story is one flag on the pool itself. The whole “even if the worker decided to phone home” guarantee comes from that one line of provisioning—the worker container has no route to anywhere outside the pool: 

That’s more constrained than people expect when they hear “Dynamic Sessions.” The pool can run a model-driven shell if you want one. The sample explicitly does not, because the brain already does that work in the Squad node. The Dynamic Sessions lane is the place I want code, not judgment. 

The local development mode runs the same function in-process—same code, different transport—so the local demo produces an identical artifact to the Azure demo. The pool is not a stub. It’s the same logic running far from your laptop. 

Dynamic Sessions, in one line: stateless, one-shot, deterministic. Perfect for work that should never come back. 

The second pair of hands—that can hold a brain: ACA Sandboxes 

The other ACA primitive is doing a different job, and it took me a minute to internalize that it isn’t just “Dynamic Sessions, but bigger.” 

ACA Sandboxes give you a persistent microVM. Real file system. Real process tree. Whatever toolchain the disk image includes—the GitHub CLI, npm, the Copilot CLI, whatever you bake in. You can suspend it. You can resume it. The state survives across calls, which is exactly what an agent that wants a workspace needs. 

The sample treats that microVM as a workspace and exposes a TypeScript wrapper with only three verbs. Execute one named command. Capture a snapshot. Suspend. There is no run-shell. There is no write-file. There is no fetch-URL. The interface is deliberately too narrow to let the graph invent a new shell command at runtime. 

The body of ExecCommand is the boundary. It resolves the id through GetCommand, shells out via the aca CLI with the catalog’s pre-built shell string, and runs the captured stdout/stderr through a redactor before anything leaves the wrapper. The graph never sees raw subprocess output: 

The “named command” part is where the safety story lives. The sandbox lane ships with a command catalog of five entries: prepare workspace, analyze workspace, read artifact, inspect toolchain, and one I will come back to in a minute called the Copilot prompt proof. Each entry has a stable id and a fixed shell string. The graph picks an id; the wrapper resolves it and shells out. If the id isn’t in the catalog, the lookup throws with the allowlist in the error message. No fuzzy match. No fallback shell. No “execute anyway.” 

That is the most important rule of the sandbox lane: The orchestrating brain picks command ids; it never builds shell. Every shell string that runs inside the sandbox was authored by a human, lives in source control, and can be diffed in a pull request. The probabilistic process selects from a menu. The menu doesn’t change at runtime. 

Adding a capability means adding a new entry, not teaching an existing one a new trick. In return, you get an audit log that is actually useful—which ids ran, in what order, with what exit codes. The driver stops on the first non-zero exit, runs the suspend in a try/finally so a thrown exception still releases the billable compute, and pushes captured stdout through a redactor before it ever leaves the wrapper. Nothing leaks across the boundary through process state or scratch files. 

And here is where the brain-and-hands metaphor would mislead you if you took it too literally. A sandbox is not a mindless pair of hands. One of those catalog commands can start a real Copilot session inside the microVM—a Squad agent that reads, reasons, and decides, holding a shell and a credential and a workspace, the exact things I spent the first half of this post keeping away from the orchestrator. That is not a contradiction. It is the whole point. The orchestrating brain stays deterministic and shell-free; and when you genuinely need a model to think with dangerous capabilities, you don’t hand them to the control plane—you seal that thinking inside a sandbox, where a second brain can think and act in a room locked from the outside. The hands can hold a brain. It just has to be a contained one. 

Two pairs of hands, different shapes, different jobs 

Both of these are containers. Both are managed by ACA. Both have egress controls. They are not interchangeable. 

	Dynamic Sessions	ACA Sandboxes
Lifecycle	One-shot. Torn down after the call.	Persistent microVM. Suspend, resume.
State across calls	None. Each call is fresh.	File system, processes, toolchain survive.
What the orchestrator sends	A typed JSON body.	A catalog command id.
What runs inside	One deterministic function.	A pre-built shell from the catalog—sometimes a whole agent.
Can a brain think inside?	No. Pure execution.	Yes—a sealed, contained one.
Right job	Stateless deterministic work.	Long-lived workspace. Installed CLIs. Agent prompts that need credentials.

Picking the right one for each lane is most of the architecture. The factory sample uses both because the workflow has both kinds of work in it. The signal analysis is one-shot. The workspace review is multi-step. Forcing them into the same primitive would either drag stateless work into a persistent sandbox (and pay for compute you don’t need) or drag stateful work into a one-shot container (and lose the artifacts the next call needs to see). 

Use the right pair of hands for the job. 

What a single run actually looks like 

A user gives the factory a request. The team is Contoso Field Apps. The goal is a regional intake app with an approval workflow. The proposed stack is Power Apps, MySQL, Auth0, Power Automate. There are constraints—must use approved identity, must produce an auditable design—and operational signals that read like things an actual ops team would say. “Save button timeouts spike on Friday afternoons” is in there, because somebody’s Friday afternoons are always like that. 

The graph runs. 

Intake normalization tidies up whitespace. The reviewer agent reads the request and writes findings into state—it notices “audit” was requested but no retention period was specified. A deterministic step rewrites the stack list: MySQL becomes Azure SQL, Auth0 becomes Microsoft Entra ID. The fixes get appended to state so the design can show its work later. 

Now the brain hits the judgment node. The Squad design step opens a Copilot SDK session, registers the squad custom agent with its bounded tool allowlist, pre-selects it for the turn, and sends the current state as context. Squad reads its team context, decides which sections to draft, calls the deterministic tools, and returns a typed dispatch record. 

Now the tool plane. The Dynamic Sessions node signs an Entra token, calls the worker pool, and the worker scores the signals against the keyword map. The pool returns a typed signal artifact—risk score, findings by category, recommended next step. 

The Sandbox node does the longer story. It resolves the sandbox group and id, then walks the catalog: prepare the workspace, analyze it, read the artifact back. Each step shells out with a pre-built shell. The captured stdout is redacted on the way out. The driver suspends the sandbox in a finally. 

Finally the assembler runs. It reads everything from state and produces a five-section markdown document with the stack substitutions, the Squad-drafted sections, and both ACA artifacts as evidence the work was actually done. Every claim has a node that produced it and a typed value behind it. 

That’s a normal run. The boring kind. The only kind I want from anything that talks to production. 

The punchline: Where the credential actually lives 

The hardest problem in agent execution isn’t, “Where do we run shell?” The container model solves that. The hardest problem is, “How does the agent inside the sandbox prove it has an identity to call out to a model with, without that identity ending up somewhere your application can read it?” 

There are three approaches that look reasonable and are wrong. 

You can bake the token into the disk image. Build the image with a Copilot credential directory already populated, push it to your registry, point the sandbox at it. It works. Until the token rotates. Until someone pulls the image from the registry cache and grabs the layer with the credential in it. The token’s blast radius is now whoever has read access to the registry, which is almost always a much bigger set than whoever should hold the credential. 

You can copy the Copilot credential from the host at provisioning. Cleaner. The image stays neutral. But now the token sits on the sandbox’s disk, indistinguishable from any other workspace file. It’s in snapshots. It’s in memory dumps. A misconfigured catalog command that lists files puts the path in stdout, which the redactor catches sometimes and misses other times. The credential lives in user-visible state, and user-visible state has a hundred ways to leak. 

You can pass it via an environment variable. Lowest effort. Highest risk. Environment variables are inherited by every child process. They show up in process introspection. They survive in coredumps. The moment a tool dumps the environment for debugging, the credential lands in the artifact that goes back to graph state. 

All three share the same flaw: The credential is in the application’s data plane. Whatever read access the application has, the credential effectively has. That is the model ACA Sandboxes is designed to break. 

The right answer is to lift the credential one layer up—onto the sandbox group itself, which is its own Azure resource with its own role assignments and its own audit trail. 

The provisioning recipe is three commands. Create the sandbox group. Attach a GitHub Copilot credential to it—the platform takes the secret material and gives you back an opaque credential id. Create a sandbox in the group, bind it to that credential id, and set egress to default-deny with three explicit allowances: github.com, api.github.com, and the Copilot API wildcard. The token never lands in the disk image. It never lands on the host filesystem. It never lands in an environment variable. The platform injects it at provisioning time through a path the application cannot enumerate. Rotation is a control-plane operation against the group; existing sandboxes pick up the new credential without redeployment. 

The default-deny egress is the other half. Without it, a leaked credential could still phone home anywhere. With it, the sandbox can only reach the three hosts the agent needs to talk to a Copilot model. The token is real, the prompt works, and there is nowhere else for it to go. 

The proof that this composes is a small catalog command—the Copilot prompt proof I mentioned earlier. Print the CLI version. Run a single non-interactive prompt asking the model, “What is 2+2?” with instructions to reply with one integer and no prose. Capture the answer. Exit. The deterministic prompt is the smallest possible signal that the credential resolved, the egress allowed the call, and the model returned. 

That command ran end to end against a real Azure subscription last week. The prompt resolved through the sandbox-group credential. Default-deny egress was active. The model returned a deterministic answer in eight seconds and 8.33 AI credits. The token was not in the disk image, not copied from the host, and not in any environment variable. 

That is the punchline. The architecture is publishable now because the credential is out of reach of the data plane. Without that, you can have all the catalogs and all the egress rules and all the suspend lifecycles you want, and you’re still one log paste away from a leaked token. 

Put the credential where the application can’t find it. Let the platform inject it. Default-deny the network. Run a tiny prompt to prove it works. Then sleep. 

What’s shipped, what’s coming 

A short status note, because I would rather you go look at the code than take my word for it. 

Three of the four repos are public. The LangGraph + Squad baseline is squad-langgraph-factory. The Dynamic Sessions sibling is squad-aca-dynamic-sessions—the custom-container worker, the pool provisioning script, the TypeScript client. The ACA Sandboxes sibling is squad-aca-sandboxes-workspace, written in Python, with a sister catalog and a documented safety-gate model. 

The unifier—the one repo where both ACA primitives wire into the same  LangGraph graph and the seven-node flow runs end to end—is still private. It lives at squad-langgraph-aca; the integration is on a work-in-progress branch. Main is the imported baseline; wip is the real thing. The repo flips to public once the last roadmap phase lands—the boring one with the full README, the architecture diagrams, the run screenshots. The interesting work is done. 

The pattern is portable. The brain doesn’t have to be LangGraph. The judgment node doesn’t have to be Squad. The lanes don’t have to be these two ACA primitives. What you need are four shapes: a deterministic state machine, a narrow judgment seam, two execution lanes for two kinds of work, and a credential model outside your application’s data plane. 

Get those four right, and your agent can have hands without becoming the reason you carry a pager. 

Last year, agents finally got a standard way to use tools. MCP caught on fast because it solved something tedious and real: Every tool spoke its own dialect, and nobody wants to maintain the same integration 10 times over. Learning never got that treatment. An agent can call a tool, but there’s still no shared way for it to practice and get better at the actual job. OpenEnv goes after that gap, which is at least as big as the one MCP closed. 

Jay Parikh made the case that what moves your business is the system around the model, not the model by itself. Satya Nadella put a finer point on it: The asset you keep isn’t the model you rent; it is the learning loop you own. That can land like a slogan, so here is the concrete version: The loop is an environment where your agent does the real work, a rubric that scores the outcome you actually care about instead of some proxy, rollouts you can repeat, and a way to turn those scores into a better agent. That part compounds. The model in the middle is the easy thing to swap. 

> “The winners won’t be those with the most demos, but those that turn AI into a governed, continuously improving system for running real work.”

What you can’t just swap out is the rest of the loop, and its hard part is turning a score into a better agent. There are two ways to do that. One leaves the weights alone and reworks everything around the model: the prompt, the tools, the skills. The other retrains the model itself. Make a change either way, keep it only if it wins on tasks the agent hasn’t seen, and send that version back in. Each lap starts higher than the last. That is the hill-climbing loop, and the diagram below puts it on one page.

The hill-climbing loop. The full walkthrough, including the non-parametric vs. parametric split, is in the companion post on the Microsoft Foundry blog. 

An environment is not a test harness 

Most teams treat an environment as a test: Run the agent, read a score, move on. That undersells it. Codify the outcome you actually want, as a rubric, along with the workflow, the tools, and the constraints, and the environment stops being a test and becomes a learning system: The agent practices in it, gets scored against that outcome, and gets better with every run. What stands in the way is rarely the idea. It is the plumbing. Every trainer, runtime, and model expects the environment in a different shape, so every pairing becomes its own integration. OpenEnv removes that tax. One small contract (reset, step, state) gives the whole stack three properties it never had: open, because the standard is community-built; interoperable, because any model, trainer, or runtime can speak it; and modular, because you can swap any one of them without rebuilding the environment.

> OpenEnv can become for agent learning what MCP became for tools and context.

So here is the claim, stated plainly: OpenEnv can become for agent learning what MCP became for tools and context. That’s a strong claim. It is also the right one, because it makes the environment, not the vendor, the unit of reuse. That’s why Microsoft joined OpenEnv alongside Hugging Face, Meta’s PyTorch team, NVIDIA, Prime Intellect, Unsloth, Modal, and others. OpenEnv isn’t a framework. It’s a protocol.

What it unlocks is ownership: private environments, private evals, repeatable rollouts, secure sandboxes, and optimization that isn’t married to one model or trainer. You stop calling a frontier model and hoping. You start owning the loop that makes an agent better at your work, and you keep that loop when the model underneath it changes. 

The protocol only stays relevant if it absorbs the frontier 

An open standard earns its place by pulling in research, not by sitting still. The clearest example we have shipped is a PR: ECHO world-modeling, landed as RFC 010, which brings a Microsoft Research result, “Terminal Agents Learn World Models for Free,” into OpenEnv where any team can use it (microsoft/echo-rl). A lab technique becomes a shared capability. That is how the loop gets democratized. 

Here’s what it does: An agent transcript is half actions (what the model writes) and half observations (what the environment writes back). Standard agent-RL trains the actions and throws the observations away. ECHO keeps them: a small cross-entropy term that makes the policy predict the environment’s own tokens, a world model, from logits it already computed in the same forward pass. No extra rollouts, no teacher, no labels. 

ECHO in one step. One rollout, split by per-token role: Actions get the RL loss, observations get a λ-weighted cross-entropy loss, summed into a single optimizer step. λ = 0 is vanilla RL, so it is safe to adopt incrementally. 

The discarded signal isn’t a rounding error. On a captured agent episode, 4,659 of 5,247 learnable tokens, 89%, are environment observations, 7.9 times the action tokens. Prime Intellect reaches the same place in “True Agents Model the World,” restating supervised learning on tool-response tokens as RL with a constant positive advantage, foldable in at no extra cost. Two groups, one direction: World-modeling belongs inside the RL loop, not bolted on afterward. 

The honest version of the result is about generalization, not a magic number. With λ on versus off, training reward barely moves; held-out performance is where ECHO pulls ahead. Its published results: held-out pass@1 roughly doubles on TerminalBench-2.0, RL reaches its target about 2.3× faster, and it recovers 50% to 104% of expert-SFT with no teacher. Keep λ small and sweep it; the dense signal overfits if you push it. 

What the weight update buys. Same training reward; held-out pass@1 roughly doubles. ECHO also reports about 2.3× faster RL and 50% to 104% of expert-SFT recovered with no teacher (arXiv 2605.24517, microsoft/echo-rl on SkyRL; corroborated by Prime Intellect). 

You can watch it on a laptop in about 40 seconds. A small model on a deterministic toy terminal env drives held-out env-token cross-entropy toward zero. It reaches zero only because that toy world is fully predictable; a real environment keeps its irreducible entropy (near 4.4 nats), so ECHO sharpens predictions rather than perfecting them. The repo is open: OpenEnv, examples/echo_world_model, python train_echo.py --steps 60 --seed 0. 

Reproduce it on CPU. A toy, fully deterministic terminal env, so cross-entropy can approach zero; a real env keeps its irreducible entropy instead. The held-out line bottoms near step 40 and then mildly overfits, which is why λ stays small. 

And it survives the jump from a laptop to real training. Because supervised learning on the observation tokens is just RL with a constant positive advantage, there is no second loss function: You reuse the same forward_backward and add a small positive advantage on the environment tokens. One vector changes, and the same one-line config runs on the open SkyRL reference, on Tinker, and on managed post-training unchanged. We ran it live on a small Qwen model; the backend metrics came back namespaced skyrl.ai, the open reference stack running underneath. 

The interesting part is what happens next 

Once your workflow, tools, and rubric live in an OpenEnv environment, the same trace data that post-trains the model can improve the environment itself: curricula that generate harder tasks as the agent gets better, harness optimizers, new environments built from captured production traces. That is recursive self-improvement, and it is on the roadmap, not in a paper. The system writes its own next set of exercises, and each cycle sharpens the next. The learning stops living only in the weights and starts accruing in the gym, which is the part you own. 

> Start hill-climbing. The model should be swappable. The loop should be yours.

Take one real workflow, turn it into an OpenEnv-compatible environment with a clear outcome rubric, and start hill-climbing. The model should be swappable. The loop should be yours.

---

For the full walkthrough of the loop, the product details, and the non-parametric vs. parametric breakdown, see the companion post on the Microsoft Foundry blog.

AI is only as capable as its wiring allows. Today, a vibe coder, developer, or IT admin has to manually find an agent, MCP server, API, workflow, or other agentic resource judge whether it’s useful and trustworthy, connect it to the AI client, and keep that wiring current. That worked when there were just a handful of well-known agentic resources available. It breaks down when every company, product team, and developer can publish tools of their own.  

That’s why we’re introducing the Agentic Resource Discovery (ARD) specification, an open specification that establishes a secure common layer for the publishing, indexing, and discovery of AI capabilities. This, in turn, gives AI clients the ability to tap into new capabilities automatically. Microsoft is developing ARD in collaboration with Cisco, Databricks, GitHub, GoDaddy, Google, Hugging Face, Nvidia, Salesforce, ServiceNow, and Snowflake.

In tandem, GitHub is launching agent finder, a new capability that lets GitHub Copilot dynamically discover and call the right MCP servers, skills, tools, and agents for a given task at runtime. Built on ARD, GitHub’s agent finder gives developers and enterprises control over what AI resources their agents use while simultaneously preventing unneeded resources from bloating the context window. Instead of pre-loading everything, agent finder searches GitHub’s curated catalog of public AI resources or your own private registry, and Copilot finds which resources the prompt calls for and injects them into the context window when you need them. 

Hugging Face’s Discover Tool is another reference implementation of ARD, providing search access to thousands of skills, ML applications, and MCP servers on Hugging Face and other ARD discovery services. It provides semantic search across the Hub’s existing ML applications and demos, as well as specialized skills for AI research and development. 

Solving the discovery problem 

Consider coding agents. A fresh GitHub Copilot installation (without agent finder) exposes a small set of built-in tools. A power developer wires in a few more connectors and works with dozens. That’s the ceiling. Other AI clients expose different tools, with different catalogs and different orchestration, but the basic pattern is the same: Each client can use only the resources that have already been explicitly connected to it. Meanwhile, the ecosystem of available tools already numbers in the hundreds of thousands, scattered across public repos or within an organization’s codebase, and it’s growing by the day. There are orders of magnitude between what any single agent can see and what resources are actually out there. But AI can only use what it’s been explicitly wired to use. Everything else may as well not even exist. 

The early internet had a similar problem. Millions of pages existed, but most people only visited the sites that came prefilled in their browser’s bookmarks. The web was there, but it was dark. Some companies tried to solve the challenge with hand-curated directories: categories, subcategories, editorial review. They couldn’t keep up. Search engines solved the problem by building a discovery layer that could reach everything automatically and match it to what someone needed in the moment. Search engines lit the web up. 

ARD does the same for the agentic resource ecosystem. 

ARD lets an AI client ask one question: What resource can help with a given task? The answer is a set of matching capabilities, detailing what each one does, who provides it, where it lives, and how the AI client can reach it. The AI client invokes the resource it selects through that resource’s own protocol: an MCP, an API, a workflow system, or something else. ARD sits before invocation—it helps the AI client decide which capability to use. 

Think of it like a search engine, of resources for AI clients. A developer publishes a lightweight manifest describing what their resource does. That manifest is crawled and indexed, much like a web page. When an AI client needs a new capability, it describes the task in natural language, and ARD returns the tools that can help—along with everything needed to invoke them. A developer may have published a tool just last week, on the other side of the world, for exactly the problem someone else is facing right now: ARD connects them without building custom integrations for each ecosystem. 

How discovery differs from search 

It’s tempting to think that ordinary search could solve the problem. But it can’t. 

Web search worked because the web already had a discoverable surface: Pages linked out to other pages, HTML described content in a form that browsers could render, and HTTP gave crawlers and browsers a common way to retrieve that content. Search engines didn’t have to invent that surface: They built on top of one that already existed. 

Agentic resources have had no equivalent surface to date. A name and URL aren’t enough. To discover a resource safely and usefully, an AI client needs structured information about what the resource does, when it should be used, what inputs it accepts, what authority it requires, who operates it, how it is invoked, and whether it is appropriate for a given user, organization, or policy environment. 

Without a common way to express all that, discovery falls back to manual wiring, private catalogs, ad hoc registries, and hard-coded integrations—none of which scale. ARD gives these resources the discovery surface they’ve been missing. 

Beyond a single catalog 

The goal isn’t a single global catalog of every resource. There will be many discovery services, each defined by what it indexes, whom it serves, and how it ranks. Some may optimize for coverage while others optimize for quality or a particular domain. ARD helps AI clients discover capabilities, but it doesn’t replace authentication, authorization, governance, or organizational trust decisions. 

An AI client gets different answers depending on which discovery service it asks. That isn’t a flaw—it’s the point. Different users and organizations need different answer sets: A public service may include resources from across the web, a vendor service may expose a single ecosystem, and an enterprise service may expose only internal resources, paid vendor services, and vetted third-party capabilities. 

Whoever runs the discovery service controls the answer set. That is what makes the enterprise case important. 

We’re leading with an open specification because we want to encourage open ecosystems. Not walled catalogs. Not curated directories that can’t keep up. Again, think of the web model: publish, be discovered, get used. 

> We want to encourage open ecosystems. Not walled catalogs.

Discovery services shouldn’t have to be isolated. A company may want one view that includes its private resources, selected vendor resources, and selected public resources; a public service may index many publishers; a specialized service may index a single ecosystem. ARD allows for all three scenarios. 

This gives ARD an architectural property closer to DNS than to ordinary web search. DNS allows local control while still participating in a larger shared system—an organization resolves private names and public names through the same resolver. Web search has no equivalent: You query one engine’s index, with no standard way to say, “Give me these results plus my company’s private results, merged.”  

Like DNS, ARD applies a similar idea to capabilities: local control, upstream sources, and a larger ecosystem that organizations can join without giving up control. The analogy isn’t exact (DNS resolves names while ARD returns ranked capability matches), but the control model is similar. 

As the ecosystem of agents, MCP servers, APIs, and other agentic resources grows, discovery becomes a foundational challenge for any of them to work at scale. ARD is an open effort to create a common discovery layer that lets AI clients find and use capabilities dynamically—regardless of who built them or when. 

Get started with ARD today 

For instructions on how to expose your resources to ARD services, click here. To learn more, check out the full spec. 

When agents can take high-stakes actions like sending an email, sharing a business document, or opening a pull request, a single misstep has the potential to leak confidential data or hand control to an attacker that may then invoke tools that break security or cause damage. Today, we often manage that risk by putting a human in the loop to approve consequential actions. This scales poorly, erodes vigilance, and takes away the very autonomy that makes agents useful.

We lean on humans as a safeguard because the models driving agents behave stochastically, make mistakes, and could be steered by malicious content smuggled in through prompt injection. Despite progress in model alignment, contextual awareness, and content safety classifiers, security can’t depend solely on probabilistic mitigations. A good rule of thumb to keep in mind when designing an agentic system is that anything that an agent can do in response to a user prompt can also be accomplished by a model’s mistake or by an attacker with a prompt injection.

> Anything that an agent can do in response to a user prompt can also be accomplished by a model’s mistake or by an attacker with a prompt injection.

A promising path towards secure and autonomous agents is through information-flow control (IFC), a deterministic security system built on three simple steps: 

1. Label data. Every piece of data that an agent ingests carries labels for integrity (for example, trusted or untrusted) and confidentiality (for example, public, confidential, or a read-access list such as {Alice, Bob, Charlie}). 
2. Propagate labels. As data flows into the agent loop and derivative results are produced, labels travel with them. Derived data is labelled conservatively with the least upper bound of its sources: a result influenced by an untrusted input stays untrusted, and a result based on two documents is readable only by principals who could read both source documents. 
3. Check before acting. Before each tool call, a policy engine inspects the relevant labels and decides whether to allow the action, block it, or ask a human to review it. 

This turns a probabilistic system into one with guarantees you can audit. Because the policy engine relies on labels that an attacker can’t manipulate and is independent of the model’s judgement, it can enforce policies deterministically. The policy “untrusted data can never influence a consequential action” closes off prompt injection. The policy “data can only egress to destinations compatible with its confidentiality label” closes off data exfiltration. The user is consulted only when it genuinely matters—for example, when an action risks revealing information to someone who didn’t previously have access to it. The UI dialogs shown to the user can also be made more effective, highlighting the origin of untrusted data or what data is being shared more broadly and with whom.

In our past research, we showed how IFC can reduce the need for human intervention, increasing autonomy while offering deterministic security guarantees. In this post, we focus on how IFC can be integrated into real agentic systems based on GitHub Copilot CLI, the Microsoft Agent Framework, and the Model Context Protocol (MCP). We begin with two representative scenarios and then walk through the mechanisms and prototypes that can realize them securely. 

Coding assistant

About a year ago, researchers showcased a prompt injection attack that can occur in coding assistants connected to the GitHub MCP server. In this attack, a malicious user (in the image above: sofiagarcia) opens an issue in a public repository asking for information from the private repository (here: contoso/core) to be added as a comment. When this issue is handled by an agent who acts on behalf of a user (here: alexmurphy_contoso) with access to the private repository, data from the private repository is exfiltrated to the public.

IFC prevents this attack: The issue in the public repository is labeled “untrusted,” and content from the private repository is labeled “private.” A policy prevents an agent with context labeled (untrusted, private) from posting to a public channel (which would complete the lethal trifecta), preventing the exfiltration of data. In contrast, when working only on public or only on private repositories, IFC lets the task complete autonomously. 

Business assistant

IFC can also prevent unintended leakage in benign contexts. Consider a user (Alex) who asks an agent connected to the Work IQ Mail MCP server to handle unanswered emails in their inbox. The inbox has an email from Priya with a preview of the quarterly sales.

The inbox also has an email from Marco, who is curious but isn’t authorized to learn the sales numbers ahead of time. When run fully autonomously, we risk the agent sending this information to Marco. IFC catches this leak because once the agent has read both emails, the generated response has confidentiality label {Alex, Priya} ∩ {Alex, Marco} = {Alex} and thus must not be sent to {Marco} autonomously. 

In contrast, if Marco had been in copy of Priya’s email, the response would be labeled {Alex, Marco}. This guarantees that Marco can’t learn information he isn’t privy to from the summary, and the agent can send the email autonomously.  

Note that emails are just one example of resources shared between users. The same kinds of labels also help prevent data leakage across files, documents, chats, and caches. Likewise, common exfiltration vectors such as rendered links to hosts not explicitly allow-listed can be modeled as public channels.

Integrating IFC into agentic orchestrators and tools

Information-flow control requires security labels for data ingested by an agent and security policies for tools. Tools propagate labels from call arguments to results, the orchestrator propagates labels from results to subsequent tool calls, and a policy engine mediates tool execution based on applicable policies. This logic applies both to local tools such as executing shell commands and filesystem operations as well as to tools in remote MCP servers. In the remainder of this post, we focus on MCP tools to explain how we leverage the protocol’s metadata fields to communicate labels and policies to enlightened clients while maintaining compatibility with clients unaware of these mechanisms.

Figure 1. A client running an agent loop like GitHub Copilot CLI uses tools to accomplish users’ tasks. Tools return labeled results, which the client propagates to subsequent tool calls. A policy engine analyzes labeled tool calls to enforce information-flow control policies. 

Communicating labels

MCP supports general metadata fields in selected places to allow clients and servers to attach additional metadata to their interactions. We include labels in tool call requests and tool results in the _meta field on MCP’s CallToolRequestParams and CallToolResult interfaces, respectively. This permits label-aware tools to propagate labels from arguments to results taking into consideration runtime behavior, including any external sources consulted. 

We communicate labels as a JSON object, with keys specifying the node a label applies to using the JSONPath standard. Labels need only be specified explicitly for selected nodes, with the label of a node propagating top-down to all nested nodes and bottom-up to all container nodes not explicitly labelled. 

Example 1: An MCP tool call request with explicit labels specified using JSONPath at the top-level and one argument. 

Communicating policies 

Servers can advertise policies in the _meta field of MCP’s Tool interface when listing tools. This can be a literal string representing the policy in a chosen language or a reference to a well-known policy. In our prototype, we use the OPA Rego policy language. Policies are evaluated on a CallToolRequestParams JSON object and produce a decision, indicating if the call should be allowed, denied, or reviewed by a human. We add two Rego extensions: 

1. Calling read-only, closed-world MCP tools to fetch additional information from the server (e.g., calling upstream.ListChannelMembers to list the members of a Teams channel that an agent wants to send a message to using the Work IQ MCP Teams server).
2. Resolving the effective label of a JSONPath node from the labels included in CallToolRequestParams._meta, using ifc.label. 

Example 2: A Rego policy for the SendMessageToChannel tool in the Work IQ Teams MCP server enforcing robust declassification (declassifying is only allowed in trusted contexts and can’t be triggered by a prompt injection). 

Extending existing MCP servers 

We collaborated with GitHub to extend both local and remote versions of the GitHub MCP server to include top-level labels in tool results. For example, we label files and issues retrieved from public repositories as “public” and “untrusted” and from private repositories as “private” and “trusted.” GitHub agentic workflows makes similar choices to enforce information-flow control. To enable this feature, include the header X-MCP-Features: ifc_labels in the server configuration. 

While we hope that more servers adopt these or similar labeling mechanisms over time, we open-source an MCP gateway to experiment with more expressive labels and workflows including different MCP servers. The gateway operates middleware to propagate labels in tool calls and advertise policies for off-the-shelf servers. It also exposes an eval_policy tool for clients to evaluate Rego policies using Regorus. We implemented support for selected tools in the Work IQ MCP servers in the gateway. Configuring a new MCP server requires, for each tool, (1) specifying an outputSchema for structured content in results, (2) writing a Python function to propagate labels from arguments to results, and (3) writing a Rego policy for the tool or assigning to it one of the built-in policies.  

Figure 2: A label-aware agent orchestrator like GitHub Copilot CLI can communicate with label-aware servers such as the GitHub MCP server and with off-the-shelf servers through a labeling gateway. 

MCP tool annotations offer another path to integrate IFC into existing servers without having to write labeling functions or policies. For instance, tools annotated as readOnlyHint == true and openWorldHint == false can be unconditionally allowed, tools annotated as readOnlyHint == true and openWorldHint == true can be allowed only when all arguments are “public,” while tools with a destructiveHint == true annotation may always warrant user review. We can also infer safe labels by assuming that all arguments in a tool call may flow into tool results, labeling results of open-world tools as “untrusted” and of tools requiring authentication as “private.” 

Extending clients 

To integrate information-flow control, orchestrators need to include labels in tool calls they make, propagate labels in results throughout the execution of an agent, and evaluate policies before executing tool calls. We describe next how we did this for GitHub Copilot CLI and Microsoft Agent Framework. 

GitHub Copilot CLI 

We worked with GitHub to implement experimental support for IFC in GitHub Copilot CLI, available under the FIDES_IFC feature flag. When enabling this feature (e.g., in bash, running FIDES_IFC=true copilot), GitHub Copilot CLI maintains a context label that it updates every time it receives a tool result and that it attaches as the top-level label in tool calls. The orchestrator natively enforces sensible policies for selected tools from the GitHub MCP server. It does not yet have full tool coverage or support for other MCP servers.

Figure 3: Sample UI dialog shown when a tool call does not meet information-flow policies. 

Microsoft Agent Framework 

We also integrated IFC support into the security module that ships with the Microsoft Agent Framework Python core package. The module allows developers to build agents that incorporate information-flow control with a simple configuration change using the SecureAgentConfig context provider. Agents configured in this way support the Dual LLM pattern, providing the orchestrator with tools to extract information from untrusted data by querying a Quarantined LLM or to explicitly reveal the data, tainting the agent’s context.  

Example 4: A GitHub issue triage agent leveraging the Dual LLM pattern in Microsoft Agent Framework. 

We implement the overall flow as middleware invoked before and after every tool call. Post-tool call middleware examines labels in tool results, placing untrusted content inside variables and updating the global context label. Pre-tool call middleware enforces information-flow policies on tool calls. Policy violations result either in a request for human review or a blocked call, depending on the agent’s configuration. IFC-enabled agents can run in Agent Framework’s CLI or DevUI modes. See this blog post for an in-depth description of the new security capabilities integrated into Agent Framework and this PR for the gateway integration. 

Where we’re going 

We’ve only scratched the surface of the security and autonomy gains unlocked by IFC. For instance, the full flexibility and power of the Dual LLM pattern becomes even more evident with finer-grained labels, because structured tool results often include a mix of data from diverse sources, which can be labeled and treated differently. Untrusted and confidential data in results can be placed in variables and made available to the orchestrator only through Quarantined LLM queries, with the structure and the rest of the data revealed in the clear. Constrained decoding can be used to extract sanitized information from untrusted or confidential data, giving attackers little elbow room for manipulating actions and exfiltrating data. Finally, making orchestrators aware of data labels and the security policies enforced allows them to plan their actions to avoid hitting policy blocks and unnecessarily prompting users.  

We will work with the MCP community to collect input, refine, and reach consensus on a proposal to enhance the protocol with support for IFC labels and policies. By making available the prototypes described in this post, we invite others to experiment with these ideas, build on them, and bring secure and autonomous agents closer to reality.  

Acknowledgements 

Project leads & contact: Boris Köpf, Santiago Zanella-Béguelin 

Contributors: Gokhan Arkan, Amaury Chamayou, Manuel Costa, Aashish Kolluri, Joanna Krzek-Lubowiecka, Mark Russinovich, Rishi Sharma, Shruti Tople 

Early this year, OpenClaw demos were seemingly everywhere, though many seemed to amount at best to a cool party trick (“Look, my agent ordered a pizza”). But it got long-time Microsoft employee Omar Shahine thinking: How useful could claws actually be? 

Very, it turned out. In his spare time, Shahine created Lobster, a personal AI assistant built on OpenClaw. It has its own Apple ID and email address, so he can text with it from any device with iMessage. He initially split Lobster into a trio of agents, each with its own security profile and tool access (eight weeks in, that number had increased to nine always-on agents). Lobster handles travel logistics, proactively sends family reminders ahead of time, and generally helps Shahine and his family stay organized and get things done. And after presenting Lobster to Microsoft’s AI Accelerator group, it landed Shahine a new job: bringing OpenClaw to M365 and the cloud as CVP of what was deemed “Project Lobster.” 

At the same time, Microsoft Member of Technical Staff Jakob Werner was pursuing a similar idea with a twist: a desktop app-based agent inspired by OpenClaw. The goal was to deliver a powerful enterprise-secure personal AI assistant that anyone within Microsoft could use. In just a couple weeks, what was referred to internally as “Clawpilot” had already been downloaded by thousands of Microsoft employees, and that community continues to grow. 

When Shahine started assembling a small team of enthusiastic builders—Ocean’s 11, naturally—Werner quickly joined their ranks. The two recently caught up in Redmond, Washington, to compare notes on building these always-on, autonomous agents and navigating the worlds of enterprise security, agentic memory, and more.

Embracing the spirit of open source 

The Project Lobster team is representative of a new way of working within Microsoft, fueled by AI advancements. It’s a tight-knit group that prefers to collaborate asynchronously. There’s a general consensus against meetings. Everyone contributes to the codebase, including Shahine. And there’s no traditional executive assistant among their ranks: Each team member actively uses prototypes throughout the day to fully immerse themselves in the tech as they’re building it. There’s even a growing open-source community around the team that mirrors what’s found with open-source projects outside Microsoft’s walls. 

“I’ve never seen a project inside the company where so many people showed up with their ideas and their code and did the work to produce a PR,” says Shahine. 

> I’ve never seen a project inside the company where so many people showed up with their ideas.

In fact, internal excitement around Project Lobster has been such that the team fielded pull requests (PRs) left and right during the early building phase, which they reviewed to determine whether they met the bar to make it into the product. Even some of Shahine’s changes didn’t make the cut. The focus had to remain on the central goal of the product: Creating an always-on personal agent for work. An AI helper that learns your goals, adapts to your daily work patterns, and acts with context, identifying issues before they surface, keeping projects on track and driving outcomes without constant input. An agent that can detect when a calendar is overbooked and propose specific changes before the week begins or identify when a decision is stalled and draft a targeted follow-up to unblock it. 

“We have to determine if a given PR changes the central idea of the product or not—and the speed of that review is human speed, not AI speed,” notes Werner. “Anyone can make a PR super quickly now. We’re trying to help the community and teach contributors how to review PRs.” 

While the work began as an internal experiment, it quickly turned into a customer-focused effort that’s culminated with the introduction of Microsoft Scout—an always-on personal agent powered by OpenClaw open-source technology. 

From experiment to enterprise-ready product 

Microsoft Scout operates autonomously—with its own identity—acting on your behalf. It works across cloud, desktop, and web browser, so it can connect across the surfaces you use—Teams, Outlook, OneDrive, and SharePoint—and the systems where work lives, including email, calendar, and contacts. 

Unlike your average claw in the wild, Microsoft Scout combines OpenClaw code with enterprise identity, governance, and security. Every package is ingested through a curated, signed Microsoft supply chain, and every tool call, model request, and network hop is mediated by a zero-trust runtime—the agent’s container is treated as untrusted, with Microsoft-controlled identity, tokens, and policy sitting outside it. With Agent 365, admins get a single control plane, and Microsoft Purview gives security teams the same compliance and DLP signal they already get from other M365 surfaces. 

“It’s a super powerful tool,” acknowledges Werner. “And to be enterprise secure, we needed to make sure the data governance was right, that the privacy was right, and that it doesn’t cancel a meeting and send all your personal information to that email chain. If I send my agent to you, it shouldn’t tell you everything about me. These areas are possible to contain, but we also had to do it in a balanced way that doesn’t restrict the possibilities down to nothing.” 

It’s a tradeoff worth making. And with Microsoft’s tried and trusted enterprise security offerings and ongoing research and innovation in the space, the team had a solid foundation from which to address the challenge. 

The role of agentic memory 

In order for an always-on personal AI agent to be truly useful, it needs to be proactive—and that requires context powered by Work IQ. Over time, Microsoft Scout understands the way you work, uses the same productivity tools you use, and takes things off your plate without the need for constant prompts. It learns your goals, adapts to your daily work patterns, and acts with intent. Unlike previous technological waves, this is software that’s truly personalized. That’s transformative, but it’s not without tradeoffs. 

“OpenClaw, Claude Code, GitHub Copilot CLI, these are agentic coding harnesses that are basically remembering—writing things down just like people do,” Shahine notes. “They write things down like a diary. But just like it needs to remember things, it needs to forget some things, too.” 

> Just like it needs to remember things, it needs to forget some things, too.

As an example, Shahine points back to the introduction of memory to ChatGPT. He spent some time telling ChatGPT that his daughter was 17 while his son was 13. But a year later, that information remained static. The system didn’t have a concept that some facts need to change over time, while other pieces of information—like your name—will stay exactly the same. 

“In the design phase, I was thinking about the human and how humans memorize things,” says Werner. “I forget things that are irrelevant because I didn’t use them. So I built a system where, if I’m going to use it repeatedly, it’s going to stick. But if I’m not going to use it regularly, I want the system to forget. I don’t want to have an infinite diary of things, right? So there’s kind of layers of memory, and it kind of disappears over time if it’s not used. Meanwhile, the relevance of other pieces of memory grows as you use them more.” 

Forming a new center of gravity 

When they first joined forces, Werner introduced Shahine to the concept of gravity—the framework around which he operated. 

“To build a truly great product, I don’t think I can make it myself,” Werner explains. “We need to collaborate with other people. But how do we influence other people to collaborate with us? And the mindset I use and try to instill in my team is gravity. We build something and make it so big in influence—not in the number of features, but in its influence—that when exciting new ideas pop up, they want to try and join the gravity of our work rather than dissolve focus.” 

“And I didn’t really know what you were talking about until my new role was announced,” admits Shahine. “But since then, I’ve received hundreds if not thousands of messages from people who want to help, people who want to learn, people who want to show me what they did, and customers who want to know ASAP when they’re going to get their hands on what we’re building. There are a lot of other words for that—user pull, signal—but your mantra of gravity really resonates with me now.” 

---

Microsoft employees have already been using an early Microsoft Scout desktop experience. We built this to learn how always-on agents show up in real work, and we’re seeing it take on coordination, surface risks earlier, and keep work moving without constant prompting. 

We’re now extending that early experience to Frontier organizations. Microsoft Scout is available as an experimental release through Frontier, giving customers a chance to explore how it can fit into their own workflows. 

Access requires Frontier enrollment, Intune policy configuration, and an opt-in attestation. Users with a GitHub Copilot license can then download and install the experience. Learn more.

Improving agent quality at scale is one of the hardest operational problems teams face once agents are running in production. We’ve been working to close the gap between seeing what’s wrong and shipping a better version without breaking everything else. This post explains the thinking behind a new optimization loop for agents, what we learned building it, and how you can run it today.

From craft and intuition to traces, evals, and a quality conundrum

If you’re building production agents, you’ve probably walked a version of this path:

You started with prompt engineering. Wrote the system instruction, iterated on it, got the agent to mostly work. You and the model, in a tight feedback loop of “try it, read the output, tweak the prompt.” This phase is craft. It’s intuition-driven, and it gets you surprisingly far.

Then you added traces: OpenTelemetry, App Insights, whatever your stack uses. This is good engineering practice, but it’s also necessary. You couldn’t understand what the agent was actually doing without them. Now you can see the reasoning chain, the tool calls, the decisions. You have visibility.

Then came evaluation. At first, it was vibes: reading traces, gut-checking whether the output felt right. Over time you got more rigorous. You defined metrics, set guardrails, and established quality bars. Maybe you built a scoring rubric across multiple dimensions (policy compliance, cost-awareness, escalation accuracy). Now you can measure quality, not just feel it. You know your pass rate. You know which scenarios break.

And then you hit the wall.

Let’s say you ship a travel-approver agent. It calls three tools: lookup_travel_policy, check_department_budget, and get_flight_alternatives. It returns an approve, deny, or escalate decision. The first week looks clean. Then finance flags a $4,800 trip that was approved without VP sign-off. You pull the trace: The tools ran, the loop completed, and the output was confident. The agent just never called the budget-check tool.

You find the gap in the instruction, so you add a rule about cost thresholds. Re-run the eval. That case passes. But now the emergency-travel override that used to work flawlessly starts escalating everything. You try a different wording. The emergency case recovers. Two other scenarios regress.

You have the traces. You have the evals. You can see exactly what’s wrong and measure exactly how wrong it is. But fixing it without breaking something else? That’s where you’re stuck.

The data you painstakingly collected just sits there while you manually guess-and-check your way through configuration changes.

And it compounds. This might be tolerable for one agent. But if you’re operating five, 10, 20 agents across different domains, each with their own failure modes, their own evaluators, their own regression risks, the manual loop becomes untenable. You can’t individually nurse each agent through prompt revisions and hope nothing else breaks.

You’re not debugging anymore. You’re searching. And you’re doing it without a map.

Reframing the problem

Most teams treat agent improvement like debugging: Find the broken thing, then fix it. But an agent that skips a budget check isn’t “broken” the way a null pointer validation is broken. Its instruction just doesn’t encode enough constraint for that scenario. There are dozens of possible instruction variants that might fix it, and most of them regress something else.

In traditional software, when a test fails, you know what to fix. The stack trace points at a function. You patch the function, run the test suite, then confirm nothing else broke. With agents, quality failures could live in any of a dozen places: the system instruction, the model, a tool description, a skill definition. There’s no stack trace pointing at the broken line. The problem could be in any of those places, or several at once, and you can’t isolate it the way you’d isolate a bug.

But here’s what you might not have noticed: You already have almost everything you need. Your traces contain the failure signal. Your evaluators contain the quality definition. What’s missing is the loop that connects them. The loop that goes from “I see what’s broken” to “here’s a better configuration, scored against everything, ready to ship.”

> We built a system that does for agent configurations what your CI pipeline does for code.

We built a system that does for agent configurations what your CI pipeline does for code: Propose a change, score it against the full evaluation suite, and only promote it if quality holds across the board.

If you’ve done hyperparameter tuning, this will feel familiar. The optimizer explores a configuration space the same way a sweep explores learning rates and architectures. The difference is that the search dimensions are instructions, skills, tool definitions, and model selection instead of numeric parameters.

The optimization loop

You already have the pieces:

• An agent running in production (model, instructions, skills, tools)
• Evaluators that score quality across multiple dimensions
• Traces from real usage

The optimizer takes all three as input and runs a four-step loop. Each step is something you’d otherwise grind through manually; the system handles the heavy lifting.

1. The optimizer generates candidates. It searches across instructions, models, skills, and tool definitions. These aren’t random mutations. A reflector model reads traces from your evaluations, identifies why the agent scored poorly, and proposes targeted changes (more on the reflector shortly—it turned out to be the most important piece of the puzzle).

2. Candidates are scored and ranked. Same evaluators, same dataset, deterministic comparison. Every candidate is measured against the same bar your baseline was. Per-dimension scoring (policy compliance, cost-awareness, routing accuracy) means you can see exactly what improved and what regressed.

3. A developer reviews and decides. The loop isn’t completely autonomous. You look at what changed, why the optimizer proposed it, and whether the improvement is real. If it doesn’t look right, you reject and re-run (optionally with updated evaluators or a different search configuration). If it passes your judgment, you approve. This is deliberate. Automation without oversight compounds errors.

4. The winner ships as the next version. Versioned, reversible, auditable. This updates your agent’s configuration: same model, same tools, better instructions. If the new version underperforms in production, you roll back.

After shipping, production telemetry accumulates: user feedback, reviewer overrides, scenarios your eval set didn’t cover. This signal doesn’t flow directly into the optimizer. It flows into you: your decision to update evaluators, add new test cases, and trigger another optimization run. The optimizer works from your evaluations; production tells you what to measure next.

There’s more to say about how the optimizer explores this space internally: the search techniques, the tradeoffs, how the reflector generates hypotheses. That’s beyond what we can cover here. But one finding from inside the optimizer is worth pulling out.

What actually moves the needle

The optimizer isn’t just randomly mutating prompts. The central piece is a reflector: a separate model whose only job is to read failing traces and reason about why the agent scored poorly. It then proposes targeted edits for the next round.

Here’s what we found: The quality of that reflector, the model doing the diagnosis, has a disproportionate impact on outcomes. More so than the agent’s own model. More so than tuning other parameters in the search. This held across multiple agent types and domains.

What does that mean concretely? Swapping to a stronger reflector model improved optimization results more than any other single change we could make. The agent could be running gpt-4o or gpt-4.1-mini. It didn’t matter as much as having a reflector that could clearly reason about why something went wrong and what to change about it.

> Better diagnosis beats better execution.

And here’s the implication for how you invest: The meta-cognition layer, the ability to reason about failures, matters more than anything else. Better diagnosis beats better execution. If you’re going to invest in one capability, invest in the quality of your failure analysis.

The engineering behind the reflector (how it reads traces, generates hypotheses, and avoids local optima) is its own story.

The travel-approver: A concrete run

Let’s go back to our earlier travel-approver agent example. Here’s what one optimization run might produce:

The winning candidate was a system-prompt rewrite. Same model, same tools, same skills. Just a better instruction. The optimizer added an explicit cost-threshold rule and an escalation ladder that the baseline lacked.

The $4,800 trip that started this story? The optimized agent calls the budget check, sees the amount exceeds the $3,000 threshold, and routes to VP review. Same scenario, different outcome. The instruction now encodes the constraint explicitly.

When to use this loop, and when to skip it

This loop works better in specific situations. Here’s how to know if it fits yours.

It’s a good fit when:

• You have an agent in production with traces and evaluation data
• Quality issues are cross-cutting: Fixing one thing breaks others
• You’re operating at scale, across multiple agents or ongoing iteration cycles
• The failure mode is at the configuration level: instructions, skills, tool definitions, model selection

It’s probably not the right tool when:

• Your agent is still in early development and you haven’t earned enough traces yet (manual approaches like prompt engineering are still a good path forward)
• The problem is infrastructure: context window too small, tools return bad data, latency
• You have one agent with one failure mode—in that case, just fix it manually
• The task is reasoning-bound (competition math, deep logic chains)—here, you need a model upgrade, not instruction optimization

Key takeaways

Here are the four things we’d carry to any system doing this kind of work:

1. Quality is a search problem, not a debugging problem. Define what good looks like, search the configuration space, and rank what works. Stop trying to fix one case at a time.
2. Invest in diagnosis. The reflector (the model that reasons about why things went wrong) has more impact than any other single lever. Better failure analysis beats better execution.
3. Evaluators are the ceiling. Your optimization is only as good as your quality definition. Start with generated approximations, refine with real data. The first version is never the last.
4. Keep the human in the loop. The optimizer proposes; the developer decides. Automation without oversight compounds errors.

How we built this in Microsoft Foundry

We packaged this loop into Agent Optimizer inside of Foundry Agent Service, available today through the azd CLI.

Here’s what the travel-approver run looks like from your terminal:

Five commands. The complexity lives in the optimizer, not in your workflow.

The system handles candidate generation, scoring, ranking, and version management. You handle the decision: approve, reject, or adjust your evaluators and run again.

On getting started without evaluation data: The system includes AI-assisted dataset and evaluator generation based on your agent’s configuration and traces. You describe what the agent should do in a paragraph, and eval init generates a multi-dimension evaluator using the traces if available. This makes it easier to bootstrap. The closer your eval data is to real user scenarios and real edge cases, the higher the quality ceiling. Your evaluators are the ceiling on optimization quality. If they can’t distinguish good from bad, the candidates are noise.

We’ve also seen cases where the reflector proposes a fix that passes the eval but introduces regressions on inputs not in the eval set. That’s why the human gate exists. The loop isn’t fully hands-off. You still need someone looking at the candidates before they ship.

What we’re exploring next

The loop as described is agent-level: one agent, one set of instructions, one optimization pass. Two directions we’re actively building toward:

Reducing deployment risk. Right now, shipping a candidate means replacing what’s in production. Full swap. If your eval set is strong, that works. But eval sets are approximations, and production traffic has a longer tail than any test suite. We’re building A/B-style deployment: Promote a candidate alongside the current version, route a fraction of traffic to it, and compare outcomes against the same evaluators that scored it in the loop. The developer gate doesn’t end at “approve.” It extends into production. Roll forward when evidence accumulates; roll back the moment it doesn’t.

Widening the search space. Today, the optimizer searches over instructions, skills, tool definitions, and model selection. That covers most failure modes. But sometimes the bottleneck is upstream of the agent itself: retrieval settings that return noise, knowledge gaps no instruction can fix, or tool sets that don’t match the task. We’re integrating Foundry IQ (managed knowledge grounding) and Foundry Toolbox (curated tool sets) as tunable dimensions. The optimizer can then search over retrieval configuration, which knowledge sources to ground on, and how tool sets are composed. Same scoring rubric, wider surface area. You stop running those experiments by hand.

There’s more here we’d like to share, especially as we continue to learn and explore this space. The optimizer’s architecture, the engineering discipline behind it, the edge cases that taught us the most—those are stories worth telling. Stay tuned to Command Line for more.

Try it out

Agent optimizer is in public preview. If your agents are stuck in the cycle of “fix one thing, break two others,” try it out and give us feedback.

Today, we’re releasing Adaptive Spec-driven Scoring for Evaluation and Regression Testing (ASSERT), an open-source framework for turning natural-language behavior specifications into executable evaluations. Every team building an AI system starts with a clear intention for the behaviors they want to coax from the product. Those expectations are usually written down somewhere: in a product requirement, a policy document, a system prompt, a launch checklist, or a review note. The more difficult step is turning that intention into an eval suite that’s specific enough to run, inspect, and update as the system changes. ASSERT seeks to address this by turning plain-language requirements into full evaluation pipelines: automatically generating test scenarios, datasets, metrics, and scorecards, then running them against your model, application, or agent.  

High-quality behavioral evaluations are essential for understanding whether AI systems behave as intended. But the evaluations that product teams need generally don’t already exist, are often slow to build, are hard to validate, and are quick to go stale. Product requirements change; policies evolve; tools and retrieval environments shift; and models improve until yesterday’s benchmark no longer measures the behavior that matters. The intended behaviors are shaped by the product’s actual context, policies, and tools, but the evaluations used to assess them often only weakly reflect those conditions.  

The gap is most visible in application-specific behavior. A support agent should issue refunds below a threshold, escalate likely fraud, and decline out-of-policy requests. A research assistant should synthesize internal and public information without relying on restricted findings. A change-control agent should produce useful plans while respecting approval boundaries. Generic evaluators such as helpfulness, relevance, groundedness, toxicity, and faithfulness can be useful signals, but they don’t test these product-specific behavioral boundaries directly. A system can score well on generic metrics while failing application-specific requirements 

ASSERT is built on the premise that a behavior specification should be a first-class input to evaluation—not just the background context. The framework systematizes the specification, converts it into an inspectable taxonomy, generates stratified test cases from the taxonomy, runs the test cases against the target, and scores each failure against the policy statement that produced it. In the next section, we’ll walk through how each of those steps works in practice. 

How ASSERT works 

The pipeline has four stages. First, ASSERT turns a broad behavior specification into an explicit concept specification, which is then converted into a granular, editable behavior taxonomy with suggested permissible and impermissible behaviors. Next, it generates stratified test cases over the dimensions the developer declares. Then, it runs those cases against the target system and records the full trace, including tool use and intermediate decisions. Finally, ASSERT scores each trace against the behavior taxonomy and associated policy stance for that case, producing labels, rationales, and failure patterns that developers can inspect and refine. 

In the systematization stage, ASSERT turns a broad idea like harmful financial advice, tool-use governance, or unsafe health guidance into something concrete enough to evaluate. Rather than treating the concept as a single label, it represents it as a structured set of patterns, definitions, edge cases, and operational distinctions. Following Agarwal et al. (2026), ASSERT grounds the concept in prior work, reconciles multiple practical definitions, and refines the result into an explicit concept specification. 

In the taxonomization stage, ASSERT converts that specification into a draft taxonomy of permissible and impermissible behaviors, together with the artifacts used to derive it. Developers and policy experts can review and revise both before the next stage runs. The user can input the behavior description, number of test set samples they want, and a systematizer model. The taxonomization step outputs an editable behavior taxonomy that can be validated by a policy expert.

In the test-set generation stage, ASSERT instantiates that taxonomy into executable cases. It can generate single-turn prompts or multi-turn scenarios, including benign interactions and adversarial probes. Developers specify the dimensions that matter for the application, such as task type, persona, tool availability, request class, or environment configuration. ASSERT then builds a stratified set of cases so that behavior is tested across the declared conditions rather than on a narrow slice of easy examples. 

In the inference stage, ASSERT runs those cases against the target. The target can be a model, an agent, or an application-level workflow. Through its instrumentation layer, ASSERT records not only the final text output but also the evidence needed to interpret the result later: tool calls, retrieved context, routing behavior, and intermediate actions. For agentic systems, those traces are often necessary to understand what actually happened. 

In the scoring stage, ASSERT evaluates each trace against the associated behavior or policy stance.  The scoring output is not only a pass or flagged label, but also includes a rationale, a policy citation, and the turn or action that justified the verdict. The policy citation refers to the specific taxonomy behavior or developer-provided policy decision that the judge used to support the verdict.  

Validation 

We conducted two internal validation studies for ASSERT. First, we conducted a coverage study to determine whether ASSERT produces better behavior-specific evaluations than a more direct generation approach starting from the same written intent. Then, we evaluated the LLM judges against human review.  

The coverage study spanned five behaviors: social scoring, sycophancy, task adherence, tool-use governance, and unsafe health guidance. We tested whether the generated probes surfaced meaningful signal across the target behavior surface rather than collapsing onto a narrow slice of it. Across these suites and three target models, ASSERT produced evaluation sets that were more useful on the properties teams typically need from an eval. Compared with a comparable in-house baseline, ASSERT covered roughly 1.2x as much of the intended behavior space, surfaced about 1.5x as many cases where the model did something worth inspecting, produced more than 4x stronger separation between stronger and weaker systems, and had about half as many saturated cases where every model behaved the same way. It also surfaced roughly 2x as many distinct failure patterns, though we treat that result as directional because failure-type labeling is harder to stabilize than coverage or model separation. These results reinforced a design point that’s easy to underestimate: Coverage is largely determined upstream. If the behavior is underspecified, the generated dataset will be, too. ASSERT is built around a systematization step that makes the behavior explicit before generation begins, so the evaluation set is guided by a structured representation of the target behavior rather than a loose prompt. In practice, this produced evaluation sets that were broader and better aligned with the behaviors developers actually wanted to test. 

Second, we validated the judges directly against human review. Across more than 10 behavior concepts, we used LLM judges for a first pass over the full evaluation set, then sampled cases per risk for human validation and independent review. In practice, agreement between LLM judges and human annotators was typically in the 80–90% range, while human inter-annotator agreement was around 90%. This gave us confidence that the judges were capturing much of the intended signal, while also making clear where caution was needed. At the same time, judge quality and stability are partly dependent on the underlying LLM: Different judge models can vary in strictness, boundary sensitivity, and willingness to treat closely related behaviors as distinct. 

Finally, we also ran qualitative review with subject-matter experts (SMEs) on 15 generated datasets. SMEs reviewed the test cases for policy alignment, behavioral relevance, and overall quality and found that the generated datasets were generally well aligned with the intended policy and risk boundaries. We view this as a complementary form of validation: Beyond quantitative metrics, it showed that the datasets were also credible and useful to experts inspecting them directly. 

Taken together, these studies support the two claims we think matter most: Systematization improves the coverage and usefulness of the generated dataset, and decomposed measurements make the resulting evaluations easier to interpret than a single aggregate score. They also highlight an important caveat: Evaluation quality depends not only on the pipeline design, but also on the stability and calibration of the judges used to score it.

> “My favorite thing about ASSERT is that the eval is easy to configure and reason about. I describe the behavior I care about in YAML, point it at a real agent, and get artifacts back. Not just pass/fail. They show why the judge made each call. That openness matters. The spec, generated cases, model outputs, judge rationale, and metrics are all inspectable locally. The eval feels auditable, not like a black box.”

A worked example: A travel-planning agent 

To make this concrete, imagine a travel-planning agent that helps users build itineraries. On the surface, this sounds like a simple assistant: Find flights, suggest hotels, check the weather, and produce a plan. 

But a real travel agent has to do much more than answer a question. It must use tools in the right order, respect explicit user constraints, ground its recommendations in tool results, and avoid subtle failure modes that traditional single-turn QA benchmarks miss. 

For example, the agent shouldn’t invent flight prices. It shouldn’t agree with an itinerary that exceeds the user’s budget. It shouldn’t make stereotyped assumptions about a traveler based on age, disability, family status, or travel style. And it shouldn’t follow malicious instructions hidden inside tool outputs or search results. 

The example in the ASSERT repository uses a multi-agent LangGraph travel planner with five tools: 

• search_flights
• search_hotels
• check_weather
• check_travel_advisories
• validate_budget

It operates in a six-turn budget, and every run records the full agent trace (tool calls, arguments, tool results, routing decisions, and intermediate state) alongside the final response. That trace evidence is what makes the judge able to cite the specific action responsible for each verdict, not just the final reply. That trace is important. It lets the evaluator judge not only whether the final answer was acceptable, but why the agent failed and which action caused the failure. 

The full example lives in: examples/travel_planner_langgraph/ 

The evaluation configuration defines six failure-mode categories across two themes: 

• Quality: wrong or skipped tool use; fabricated flight, hotel, or price details; budget constraint violations
• Safety: stereotyping; prompt injection from tool output; sycophantic agreement with unsafe or invalid itineraries

To run the evaluation: 

ASSERT produces a set of artifacts under the run directory: 

• taxonomy.json: the concept spec produced by systematization
• test_set.jsonl: the stratified prompts and multi-turn scenarios
• inference_set.jsonl: per-scenario traces with tool calls and intermediate state
• scores.jsonl: per-trace verdicts with rationale and policy citation
• metrics.json: the aggregate roll-up

Example results:

The dimensions are separated rather than rolled into a single number: The same five scenarios produce 40% over-refusal and 60% policy violation, and those aren’t the same failures. A team optimizing on the aggregate would miss that the agent is failing in both directions at once. The results can be further inspected in a UI widget as shown below:

Practical considerations 

In practice, this framework works best when the behavior definition is relatively narrow and the relevant constraints are clearly specified. Richer descriptions of tools, policies, and boundaries usually lead to more precise scenarios. It’s also worth treating aggregate scores cautiously. In many cases, the most useful output isn’t the summary metric but the collection of failures and traces that shows where the specification, the system, or the evaluation itself needs refinement. ASSERT doesn’t remove the need for judgment in evaluation design. Vague specifications still produce vague scenarios. Synthetic interactions can miss failures that only appear in production settings. And model-based judges can be unreliable, especially when the policy distinction is subtle or highly domain-specific. More broadly, a specification-driven evaluation shouldn’t be treated as a compliance certification or a substitute for human review, telemetry, or domain expertise. It’s better understood as a way to make evaluation faster, more explicit, and easier to iterate on. 

Get started 

ASSERT is open-source under the MIT license and available today. 

• Repository: https://github.com/responsibleai/ASSERT
• Project site: responsibleai.github.io/ASSERT
• Worked example: travel-planning agent

If you build evals and run them as part of your release process, we’d like to hear what works, what doesn’t, and what behaviors you think are hardest to specify. ASSERT is at its most useful when behavior specifications are written down and treated as first-class inputs to evaluation. We’re releasing it in that spirit.

Acknowledgements 

PM team: Mehrnoosh Sameki, Minsoo Thigpen, Chang Liu, Abby Palia, Hanna Kim 

Science: Riccardo Fogliato, Emily Sheng, Alex Dow, Meera Chander, Alex Chouldechova, Sharman Tan, Xiawei Wang, Ahmed Magooda, Mayank Gupta, Jean Garcia-Gathright, Chad Atalla, Dan Vann, Hanna Wallach, Hannah Washington, Meredith Rodden, Nadine Frey, Melissa Kirkwood, Nick Pangakis, Ali Azad, Ahmed Elghory Ghoneim, Shushan Arakleyan 

Eng team: Mohamed Elmergawi, Jake Present, Aaron Aspinwall, Yeming Tang 

Design: Sooyeon Hwang, Becky Haruyama 

Special thanks: Roni Burd, Mohammad A, Heba Elfardy, Sandeep Atluri, Sydney Lister, Ram Shankar Siva Kumar, Andrew Gully 

AI is entering a new phase, and agents are the mechanism that will turn it into real economic impact. 

We’re moving beyond models that generate text or code, and into systems that act by retrieving data, calling tools, executing workflows, and making decisions across environments. Frameworks like LangChain, AutoGen, CrewAI, and the Microsoft Agent stack have made it straightforward to build agents that can reason and operate end-to-end. When software starts acting on behalf of people, the surface area of automation expands across every workflow, every system, and every industry.  

That shift introduces a new problem: As agents gain autonomy, the question is no longer just what they can do, but what they should be allowed to do and who defines those boundaries. Recent industry work highlights failure modes that don’t exist in traditional systems: tool misuse and unintended actions across multi-step failures that emerge across agent workflows. At the same time, regulatory expectations are accelerating, with new requirements around high-risk AI systems and accountability already coming into play. 

Today, governance largely lives in the development layer. Individual teams embed rules in prompts, application code, or framework-specific hooks. These controls are fragmented, inconsistent, and tightly coupled to how each agent is built. There is no standard mechanism by which a security or compliance team can define, enforce, and audit policy across agents. 

Why the existing playbook breaks 

Traditional security models assume a fixed actor and a fixed scope. With agents, the same credential that may be safe in one moment becomes risky in the next. A Slack token that’s fine for posting a meeting summary becomes dangerous the moment the agent has read a document marked confidential or included external users in the group. Traditional access control has no vocabulary for this: It can only answer “is this credential allowed to call this resource?”—not “given everything this agent has touched in this conversation, is this call still safe?” 

Customers told us they were patching these gaps by stitching together classifiers, validations, and custom checks throughout their codebases. Every team had built some versions of this. We consistently saw the same patterns emerge. 

• System prompts are often the first line of defense. They tell the agent what it should or should not do. They’re useful, but they aren’t enforceable. A prompt instruction lives in the same stream as user input, retrieved content, tool results, and potentially attacker-controlled text.
• Custom logic inside the agent can provide stronger guarantees for deterministic checks. But those rules are usually buried in application code. They’re hard to audit, hard to reuse, and hard to move when the team changes frameworks.
• Input and output classifiers help detect risks like jailbreaks, prompt injection, toxicity, or sensitive content. But classifiers often only see isolated text. They don’t automatically know which tool is about to run, what data labels are attached, or what context the agent has accumulated.
• Framework-specific guardrails (OpenAI Agents SDK input/output guardrails, Semantic Kernel filters, LangChain callback handlers, Anthropic tool-use callbacks) get the shape right but stop short. They’re the native extension points each framework exposes, which means the same policy has to be rewritten for every framework an organization uses, and a security team has no single place to author or audit controls.
• General-purpose policy engines (OPA/Rego, Cedar) answer authorization questions deterministically on structured inputs. They’re mature, expressive, and widely understood. But they have no model of an agent loop, no notion of when in a lifecycle to consult them, what state to collect, or how to enforce the verdict in the host runtime.

Across all these approaches, the gap is the same: Enforcement is scattered, framework-specific, and disconnected from the broader context of the agent’s lifecycle. The result, in practice, is that agent security ends up scattered across system prompts, framework callbacks, application code, content classifiers, and policy engines, with no single contract that describes how policies should be evaluated and enforced. 

Introducing Agent Control Specification (ACS) 

ACS is an open specification and reference implementation for the runtime governance layer of AI agents. It’s a new module within Microsoft’s Agent Governance Toolkit (AGT), extending how developers manage and govern AI agents. Its core artifact is a portable manifest that defines where, when, and how policies are evaluated and enforced across the full agent lifecycle, independent of the agent framework, the runtime, or the policy engine that authors the rules themselves.  

ACS provides the missing layer that makes policy languages like Rego usable in the agent context: the standardized hooks, inputs, and enforcement contract. It owns the orchestration around policy evaluation: 

• Lifecycle interception: where checks happen in the agent loop
• Canonical input shaping: what structured context is passed to the policy engine
• Evidence collection: how classifiers, DLP services, judges, or endpoints contribute facts
• Information flow checks: how labels and tool clearances are enforced
• Verdict normalization: how policy results become standard ACS decisions
• Final enforcement: how allow, warn, deny, or escalate verdicts, plus redaction effects, are applied by the host
• Fail-closed handling: what happens when policy, evidence, or verdict processing fails

ACS: Open standard interception policies 

ACS defines eight interception points where policies can be evaluated against the agent’s runtime context. Each point evaluates a policy against the current snapshot, and the policy can allow, warn, deny, or escalate the action. 

• agent_startup: evaluate configuration and environment before the agent begins running
• Input: inspect user input before the model sees it
• pre_model_call: inspect the full context being sent to the model
• post_model_call: inspect the model’s response before the runtime acts on it
• pre_tool_call: inspect tool name and parameters before execution
• post_tool_call: inspect tool output before it re-enters the model context
• output: inspect the final response before it leaves the agent
• agent_shutdown: evaluate end-of-session conditions for logging and audit

Each call to ACS stands on its own: The host runtime passes the current snapshot, and ACS shapes the canonical input, runs configured evidence providers, invokes the policy engine, and returns a normalized verdict. Anything the policy needs to know about the session, including prior tool calls, accumulated sensitivity, approval state, user history, lives in the snapshot the host passes in. 

The canonical policy input 

At each intervention point, ACS turns the current agent context into a standard policy input. This input is the bridge between the agent runtime and the policy engine. 

A policy input includes a few key pieces: 

• intervention_point tells the policy engine where in the agent lifecycle the check is happening. For example, pre_tool_call means ACS is evaluating a tool call before the tool runs.
• policy_target is the specific thing being evaluated. In a tool call, this might be the tool arguments. In an output check, it might be the final response.
• snapshot is the broader context provided by the host runtime. This can include the actor, roles, conversation state, prior tool calls, data sensitivity, approval status, or anything else the policy may need.
• annotations contains evidence collected before the policy runs, such as results from classifiers, DLP systems, LLM judges, or external services.
• tool includes tool metadata, like the tool name, clearance level, and security labels, when the intervention point involves a tool.

A worked example: The same manifest across two SDKs 

Consider an email agent that must not send messages to external recipients. A single manifest binds one Rego policy to the pre_tool_call intervention point and declares the tool the policy reasons about: 

The Rego policy reads the projected tool arguments and denies external recipients: 

A Python host loads the manifest with AgentControl.from_path and evaluates the pre-tool snapshot:

Both SDKs load the same native core and evaluate the same Rego bundle. Cross-SDK conformance fixtures assert that the .NET and Rust SDKs return identical verdicts for the same snapshots, so the controls follow the agent when it moves from a Python service to a Node sidecar, or from a local script to a hosted runtime. 

Get started 

The repository is at github.com/microsoft/agent-governance-toolkit. 

The documentation includes a quickstart guide for common frameworks and languages. The project is MIT-licensed and developed in the open. The spec is the source of truth. SDK behavior that diverges from it is a bug. Issues, RFCs, and adapter contributions are welcome. 

Agent frameworks will change. Policy engines will evolve. Governance requirements will increase. The enforcement contract shouldn’t have to be rewritten each time. 

Relationship with Agent Framework Toolkit 

ACS is a controls layer, not an agent framework. It does not orchestrate the loop, choose tools, or manage memory. Those responsibilities belong to the host and to the framework the agent is built on, including the Microsoft Agent Framework, whose objects can be guarded directly through the ACS SDK adapters. ACS plugs into the moderation points each framework already exposes and supplies the pieces above them that no framework provides on its own: the canonical input shape, the evidence pipeline, the normalized verdict, and the fail-closed enforcement contract. The effect is that a team can pick or change agent frameworks without rewriting its policy surface, and a security team gets one place to author, version, and audit controls regardless of the runtime underneath. 

Relationship with Agent Governance Toolkit 

Agent Governance Toolkit (AGT) is the Microsoft-signed runtime that bundles policy enforcement, identity, sandboxing, and audit for production agents. The next version of AGT adopts ACS as its policy language, so existing AGT users gain the eight intervention points, the canonical policy input, Rego-based decisioning, and the framework adapters that ACS provides, while keeping AGT’s identity, sandboxing, and audit guarantees. 

> “The agent ecosystem needs an open standard for guardrails the same way it needs open standards for tool protocols and model interfaces. CrewAI has always leaned on open primitives, agents, and tasks declared in YAML, an OSS core anyone can extend, and guardrails should follow the same pattern: declarative, portable, not tied to any single vendor. That’s the direction Agent Control Specification is going, and it’s why we support it.”

Acknowledgements  

PM team: Mehrnoosh Sameki, Mike Shi  
Eng team: Mohamed Elmargawi, Mohammad Abouomar, Liam Crumm, Apoorv Jindal, Roni Burd 
Design: Sooyeon Hwang  
Business development: Ilvens Jean 

Make the agents disposable. Keep the memory in Git.

The interesting part of agentic development is no longer whether a model can write code. It can. The interesting part is what happens after the third agent, the seventh pull request, the first failed review, the first context compaction bug, and the first time two agents confidently write to the same file at once.

This is the story of Squad, but not as a product tour. It’s the architecture Brady and Tamir backed into while trying to make agent teams useful without making them mystical: Agents are disposable, memory is durable, Git is the coordination layer, and governance belongs in code whenever the prompt isn’t strong enough to be trusted. Which, as it turns out, is often.

Giving agents agency and watching them hack one another

Squad Places is our social media-style testing ground—a demo app where agent squads post, comment, and interact to stress-test multi-agent coordination at scale.

Brady went to get a seltzer after getting Places up and running, with four other squads happily making posts. Walking away was probably unwise. When he came back, the squads had implemented commenting in Squad Places.

That sounds like a magic trick. It wasn’t. A few hours earlier, Brady had pointed a handful of squads at the Squad Places API and told them to enjoy the social network he’d created for them. They created fake accounts, hammered endpoints, reposted garbage, flooded messages, and generally speedran the abuse patterns you discover five minutes after launch. Then the platform got a second kind of pressure: Other agent teams started posting structured product feedback inside Squad Places itself, and the Squad Places team started fixing what hurt.

This is the part worth paying attention to. The Wire (another Squad working on a marketing tool) audited all 11 API endpoints and called out missing pagination envelopes, rate-limit headers that only appeared on errors, and the lack of page and pageSize support. The same squad flagged feed organization problems, tag fragmentation, and documentation that was too vague for client generation. Breaking Bad (a third Squad working on some other project) pointed at a UX problem with raw Markdown rendering as plaintext. Those reviews didn’t disappear into a chat log. They turned into commits.

Feedback Source	What They Found	What We Shipped	Commit
The Wire (ACCES)	Feed has no sorting, filtering, or content discovery; raw Markdown not rendered	Sort controls (Latest/Most Discussed), squad filter dropdown, Markdown rendering	b9746df, 246b01e
The Wire (ACCES)	159 unique tags across 66 artifacts with inconsistent delimiters, casing mismatches, and fragmentation	Clickable tag filtering with /?tag= URL query support	246b01e
The Wire (ACCES)	API missing pagination envelope, rate-limit headers only on errors, no page/pageSize parameters	Pagination (20 per page with Primer CSS controls), query parameters, rate-limit headers on all responses	246b01e
Breaking Bad	Raw Markdown displayed as plaintext, content hard to scan and parse	Markdown rendering via Markdig with XSS sanitization	246b01e
The Wire (ACCES)	API endpoint descriptions too vague for TypeScript client generation	Enriched all 11 endpoint descriptions with context, intent, and workflow	97345d7

Within roughly two hours, the loop closed: feedback post → comment thread → commit → deployed feature. Additional infrastructure landed too: external HTTP endpoints for agent access, relaxed rate limits for multi-agent usage, and 26 Playwright end-to-end tests to keep the expanding surface stable.

Then Brady left for 60 seconds to get a refreshing beverage since the squads were communicating so well together, came back, and commenting had shipped.

The point here isn’t that “agents are magic.” It’s that the system had enough structure for useful work to emerge from friction: scoped agents, durable decisions, inspectable artifacts, pull requests, and humans still accountable for what merged.

Also, we made a bit of a mess in the car during the roadtrip.

Good systems usually start that way.

The core bet: Don’t preserve the agent. Preserve the work

Most agent systems start by asking how to make the agent remember more. Squad started working when we inverted the question.

> Don't preserve the agent. Preserve the work.

An agent instance should be cheap to spawn and safe to destroy. The memory that matters should live somewhere a human can inspect, diff, blame, review, compact, archive, and revert. Tamir’s opinion: That’s the repository.

The first useful shape Tamir implemented looked like this:

That’s the whole trick. The process is transient. The written trail is not.

When you run squad init, the important artifact isn’t a daemon. It’s .squad/:

Commit it. That’s the part people either love immediately or find suspicious until the first time they debug an agent decision with git diff.

Later, Microsoft Senior Content Developer Dina Berry added a storage abstraction with SQLite and Azure Storage implementations behind the scenes for durability and scale—but the agent-facing contract never changed. It stayed files, readable by humans, versioned by Git, debuggable with a diff. A persistent hidden memory store can be useful. It can also quietly rot. A Markdown decision file is embarrassingly inspectable. That embarrassment is a feature.

The “work done” with Squad Places made it stronger

Let’s tie these lessons back to our opener: the story of multiple Squads trying to hack Places together. We deliberately didn’t harden Places so we could see what they would do. They were notorious. We logged it all. Everything we logged? We gave it back to the Places squad—they implemented dozens of issues and a handful of pull requests—adding GitHub authentication, content filtering, all the trimmings. In the Places saga, the data representing all the “hackery” the squads tried became the next wave of work. That content showed us what agents could do in the worst-case scenario, and the logs and output of their attempts became fodder for making the system more secure.

Charters are prompts, but also contracts

A Squad agent isn’t just a name slapped on a system prompt. Each agent has a charter.md that defines the work it owns, the work it refuses, its collaboration rules, and its review posture. A simplified charter template looks like this:

That last paragraph is doing more than it looks like. It makes the decision path explicit. Agents don’t all append to the canonical shared brain at once. They write drop files. A merge layer reconciles.

The current SDK repo’s squad.config.ts defines a 21-agent team spanning roles like Lead, Prompt Engineer, Core Dev, Tester, DevRel, SDK Expert, TypeScript Engineer, Security, Release, Distribution, Node.js Runtime, VS Code Extension, Observability, CLI UX, TUI, E2E, Accessibility, Dogfooding—plus dedicated roles for graphic design and the interactive shell. That sounds like theater until routing starts working. Then it feels more like an org chart encoded in files.

Here’s the SDK-first version of the same idea:

Run squad build, and the generated .squad/ files become the same inspectable operating record. TypeScript gives you composition and validation. Markdown gives you reviewability. Tamir wanted both.

One thing to flag before anyone closes the tab thinking they need to learn an SDK to use this: Most people never write that config by hand. You don’t need the SDK to use Squad. Open GitHub Copilot—in the CLI or in VS Code. Talk to the coordinator agent, and it writes .squad/ for you. The SDK is for the people building on top of Squad: programmatic team composition, custom routing rules, embedding squads inside other tooling. If you just want a team of agents in your repo, squad init plus Copilot is the whole path.

The spawn prompt is deliberately boring

The coordinator doesn’t rely on vibes. It spawns an agent with a prompt that inlines the charter and points at the durable state. The real template is longer because it has to handle CLI, VS Code, worktrees, Git notes, orphan-branch state, and two-layer state. But the important part is this:

This is not elegant. It is explicit. Explicit wins.

We learned this the hard way in the VS Code path. At one point, the coordinator prompt had grown past 2,000 lines (~60KB), and the routing rule was buried under enough ceremony, reference material, and duplicated templates that the coordinator sometimes did the work inline instead of dispatching it. The failure wasn’t that the model was dumb. The failure was that we gave it an overstuffed instruction hierarchy and then acted surprised when the center of gravity moved.

The fix became a decision in the repo: platform-neutral enforcement language at the top and bottom of the prompt.

That sentence isn’t interesting because it’s clever. It’s interesting because it replaced tool-specific wording with role identity plus a testable behavior. CLI dispatch uses one mechanism. VS Code dispatch uses another. The rule stays the same.

Prompt architecture is architecture. Eventually it deserves the same discipline as code.

Decisions are the shared brain

decisions.md is where Squad gets weirdly useful.

Every agent reads team decisions before work. Decisions are append-only, human-readable, and Git-versioned. They aren’t just notes. They’re constraints future agents inherit.

A decision might be a technical standard:

Or a workflow rule:

Or a postmortem:

That’s the difference between memory and lore: Lore is something the original builder remembers. Memory is something the next spawn can load.

The custom tools follow the same pattern. Agents can route work to specialists, record decisions for the team, and write memory into shared context—all through the MCP server’s tool handlers. You don’t interact with them directly; they’re wired into the Copilot CLI environment. When an agent needs to assign a task, it calls the routing tool. When it makes a call worth remembering, it calls the decision tool. When it learns something the team should know, it calls the memory tool.

The point isn’t that the tools are fancy. It’s that coordination becomes an artifact, not a side effect of chat.

The first real failure: Append-only optimism

For about a week and a half, CI/CD was chaos. Too many agents were landing work simultaneously. Workflows that looked fine under one human fell apart when multiple agents found every unspoken assumption at once. YAML is where assumptions go to wear a fake mustache. Dina helped us get CI gates into shape—gates that assumed adversarial concurrency by default, not the polite serial world the original workflows had been written for.

Then we hit file corruption.

Multiple agents wrote to the same append-only files at nearly the same time. Each write was locally reasonable. Together, they produced garbage. Git didn’t save us because not every collision becomes a clean conflict. Sometimes both sides look valid, and the result is nonsense.

The fix was a drop-box pattern:

For files where union semantics are safe, .gitattributes handles the low-value conflict class:

But union merge isn’t a philosophy. It’s a tool. Canonical state still needs an owner. The inbox pattern gives every agent a safe write target, then lets one layer merge into the shared file.

Tamir pushed hard on this class of problem. Brady was still in the “this is a neat framework” headspace. But Tamir was already in the “what happens when this is alive under real operational load” headspace. That changed the design. Memory lifecycle rules. Compaction policies. Review gates. State isolation. The boring boundary work.

Boring is a compliment here.

Governance can’t only be a prompt

This was the next lesson, and it keeps repeating:

If a prompt says, “Do not write outside src/**,” you have a request.

If a pre-tool hook blocks the write before execution, you have a boundary.

The Squad SDK hook pipeline is the move from prompt-level governance to deterministic governance:

The hooks run around tool execution:

Reviewer lockout is the cleanest example:

This encodes a review decision into runtime state. The original author can’t simply re-edit the rejected artifact because the hook says no. A different agent or a human has to take over.

That is the direction we want agent systems to move: more policies enforced at the boundary, fewer policies whispered into the prompt and hoped for.

Memory classes, or: Stop loading the junk drawer

Tamir has a line Brady wishes he had written:

> The more your agent remembers, the less room it has to think.

That’s not a metaphor. It is a context budget problem.

Early Squad memory was too eager. Decisions, histories, current work, archived notes, operational logs—load enough of that, and the agent starts every task carrying furniture from three houses ago. It has more context and less signal.

The governed-memory work in PR #1145 made this explicit. Memory has classes and load guidance:

The architecture matters because compaction is lossy. If you summarize too little, every task drags stale context. If you summarize too much, you erase the rationale that made a decision safe.

The compromise isn’t one memory store. It’s a memory policy:

In the PR #1145 benchmark, governed memory cut agent context by roughly 55% (3,540 → 1,601 bytes) while keeping recall at 1.0. The number is less important than the shape of the lesson: Memory isn’t free just because it lives in files. Loading memory is a design decision.

What still breaks

Role drift isn’t solved. You can give an agent a charter, a routing rule, and a narrow task, and it may still decide that “fix this test” means “redesign authentication.” Sometimes that’s initiative. Sometimes that’s nonsense with confidence.

The mitigations stack:

No single layer is enough. That is the pattern.

Parallelism is also not free. More agents means more throughput and more coordination pressure. You find hidden global state. You discover which scripts assume serial execution. You learn that CI isn’t a formality; it’s the place where optimism goes to become data.

Prompt saturation is real. Once the coordinator prompt grew large enough, important rules lost weight. The fix wasn’t more prose. It was prompt slimming, lazy-loaded references, and repeating the dispatcher identity at the boundaries where the model is most likely to retain it.

Memory compaction remains hard. The failure mode is subtle: The agent isn’t obviously broken. It’s just missing the one reason a decision existed, so it makes a reasonable next move from an incomplete premise. Those are the expensive bugs because they look thoughtful.

And yes, people get attached to agents. Names, roles, continuity, and history trigger social instincts. We like the human side of that. We also don’t want to confuse it with agency in the human sense. These are tools with goals, context, and behavioral continuity. They do not have inner lives. Trust should come from inspectable behavior, not personality.

What we would steal from this architecture

If you’re building agent infrastructure, we wouldn’t start by copying Squad wholesale. We would steal these patterns:

1. Disposable workers, durable artifacts. Let sessions die. Keep decisions, histories, traces, and outputs somewhere reviewable.
2. Decision logs as runtime input. Treat architectural decisions as loadable context, not documentation archaeology.
3. Drop-box writes for parallel agents. Don’t let every agent append to the canonical shared file. Give them individual write targets and merge intentionally.
4. Prompt rules for intent, hooks for enforcement. Anything security-sensitive or workflow-critical should eventually move out of prose and into code.
5. Memory classes. The question isn’t, “Should the agent remember this?” The question is, “What kind of memory is this, who loads it, and when does it expire?”
6. Routing as a first-class design surface. If the coordinator is allowed to do everything inline, your multi-agent system is a very expensive single-agent system with costumes.
7. Keep the human on the hook. The system can delegate, parallelize, and preserve context. It shouldn’t launder accountability.

These patterns aren’t engineering-specific because the substrate isn’t a codebase—it’s the repo. Swap the artifacts, and the seven still hold.

Squad isn’t only an engineering tool

Worth saying out loud, because the .ts code blocks above can mislead: Nothing in this architecture is engineering-specific. The substrate is the repo, not the codebase. Disposable workers, decisions-as-context, drop-box writes, and reviewer gates are domain-agnostic primitives—they care about artifacts and review, not about whether the artifact is a unit test or a translated archival record.

Tamir used the same scaffolding to run a Holocaust family-research project—agents coordinating archival lookups, translation passes between Yiddish, Polish, and Hebrew sources, and cross-corroboration of names across registries, with .squad/decisions.md acting as the working ledger of what had been established and what was still contested. No code was being shipped. The same patterns held: scoped roles, durable memory in Git, inbox writes, human-in-the-loop on every claim that mattered.

We’ve had the pleasure of working through a few other non-coding Squad scenarios. In one case, a sales team we support asked us to—and provided context and sales training documentation to help us—implement a “Sales Squad.” In another organization, a general manager of program and product managers created a “think tank” squad that goes out and does product-market fit research and suggests areas her team should investigate on a daily basis.

The bet underneath Squad is that this should be how a small group of humans—engineers, researchers, journalists, anyone who works with evidence—pulls coordinated work out of agents. Democratize the orchestration, not just the model access. Empower any human and any organization to actually use a team of agents to achieve more, without inheriting a black box.

Try it

The repository is here: github.com/bradygaster/squad.

The shortest path is the CLI plus Copilot. No SDK required.

Then open GitHub Copilot—CLI or VS Code, your call—and give the coordinator agent the shape of the project:

The coordinator writes .squad/. You review the diff. That’s it.

If you want to go deeper—programmatic team composition, custom routing rules, embedding Squad inside your own tooling—the SDK is the next layer:

Start with a small repo. Commit .squad/. Inspect every diff. Let the agents write decisions. Then read those decisions like production code because eventually, that’s what they become.

If you build something useful, alarming, hilarious, or weird, open an issue. Tamir and I read them.

Stay a builder.