Microsoft for Developers
Get the latest information, insights, and news from Microsoft.
Latest posts
The Microsoft 365 Copilot Agent’s Playbook: A Practical Livestream Series for Building Better Agents
Building on Microsoft 365 Copilot? Here's your playbook. Declarative agents are quickly becoming one of the most exciting ways to extend Microsoft 365 Copilot and bring organizational knowledge, workflows, and tools directly into the flow of work. But as agent capabilities grow, so does the need for practical guidance: How do you build agents that are useful, grounded, extensible, and measurable? That’s why we’re launching The Microsoft 365 Copilot Agent’s Playbook, a four-part livestream series designed to help developers, makers, and technical teams understand how modern Microsoft 365 Copilot de...
How to test agent experience changes without shipping them
Most changes you think will improve AI agent behavior won't. We tested a dozen hypotheses on a real project upgrade scenario and the majority failed. Learn how to emulate documentation, API, and MCP server changes locally so you can validate what works before shipping anything to production.
How to test agent skills without hitting real APIs
Your agent skill calls an API. The moment you start evaluating it, every run either costs money or mutates production data. Learn how to mock APIs transparently so you can run evals without changing your skill or hitting real endpoints.
Building AX evals that actually work
This is the eighth and final article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. You've read seven articles about what to measure, why benchmarks don't transfer, and what hidden variables can do to your results. Now you actually have to build the thing. Most teams build an eval, run it, get scores, and feel good about the numbers. The trouble is that these evals produ...
Let’s Learn GitHub Copilot App – Free Virtual Training Event
Join us for a free online event series kicking off July 16 to learn how to get started with the GitHub Copilot App!
The hidden variables in your agent eval
This is the seventh article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. You build an eval. You run it on your machine. You get a score. Your colleague runs the same eval on their machine and gets a different score. Same scenario, same setup. What changed? In the previous article, we covered why public benchmarks can't tell you which model works best for your stack. T...
Don’t rewrite your CLI for agents
There's advice making the rounds: replace your CLI args with a single payload so agents can use your tool more effectively. The thinking being, that agents already think in structured formats, and nested data maps cleanly to JSON. Flat args on the other hand, force awkward conventions like repeating to delimit multi-value groups, which is inherently ambiguous. Not to mention, that the agent needs to get the types of all values right. It's a reasonable hypothesis, and we wanted to know if it holds up under measurement. The data we collected, showed something interesting. What we tested We built a synthetic CL...
Not all model upgrades are upgrades
A new model drops with lower per-token pricing and better benchmarks. You switch. A week later someone asks why the agent is burning 12x more tokens on the same task while producing worse output. We ran 150 agent tasks across 15 scenarios on two models, Claude Sonnet 4.6 and Claude Sonnet 5, using GitHub Copilot Chat in VS Code on Windows. The scenarios covered two types of work: architecture and design tasks grounded in Microsoft Learn documentation, and SharePoint Framework project upgrades. Sonnet 5 is the newer model with 33% lower per-token pricing across every token category. The assumption we wanted to te...
What AI benchmarks are not telling you
This is the sixth article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. We love benchmarks. A new model drops, the leaderboard says 92% on SWE-bench, and your timeline declares it "the best coding model." You switch to it, run your agent on your codebase, and outcomes are... the same. Maybe worse. The leaderboard said 92%, so what happened? In the previous article, we ...