The Truth / scorecard

SharePoint agents: one click to an expert on your mess

"The lowest-effort agent in the stack — instantly useful on well-kept sites, instantly embarrassing on the sites you actually have."

6/10
Best for: Site owners with a curated, current document library who want instant Q&A without building anything Skip if: Your sites are document graveyards — the agent will faithfully serve the graveyard updated June 2026

What's genuinely good

  • Creation is genuinely one click: any site owner gets a working Q&A agent scoped to the site, no Studio, no schema, no developer
  • Permission-trimmed by default — the agent only answers from what the asking user can already access, which makes security review mercifully short
  • Site-scoped Q&A is the right shape: 'what's our travel policy?' against the HR site beats tenant-wide retrieval lotteries
  • Pay-as-you-go means unlicensed users can use them — no per-seat barrier between a good agent and the people who need it
  • Scoping an agent to selected folders or files turns a decent agent into a sharp one in two minutes

What sucks

  • It's a mirror for site hygiene, and most site hygiene is bad: five versions of the policy doc means confidently inconsistent answers
  • Customization is shallow — compared to declarative agents or Studio, you get scope and instructions, not behavior, actions, or workflows
  • Discoverability is dismal: agents exist, but users don't know which sites have them or why they'd ask one anything
  • Metered PAYG costs surprise unmanaged tenants — nobody owns the bill until finance asks what 'Copilot Credits' are
  • Stale content has no expiry: the agent answers from the 2022 handbook with exactly the confidence it answers from the current one

The honest assessment

SharePoint agents are the most honest product in the Copilot lineup: they promise almost nothing — click a button, get a Q&A agent for this site — and they deliver exactly that. The question is never whether the agent works. It’s whether your site deserves one.

The mechanics are genuinely good. Creation is one click by a site owner, no Copilot Studio, no manifest, no IT ticket. Security is the strongest part of the story: agents are permission-trimmed, meaning they answer only from content the asking user could already open. That single property removes the scariest conversation from the rollout. And the pay-as-you-go model removes the other barrier — people without M365 Copilot licenses can still use a site agent, with consumption billed against Copilot Credits, so a good agent on the HR site can serve the whole company, not just the licensed tier.

When the site is well-kept, the result is quietly excellent: “what’s the per-diem rate for Germany?” against a maintained policy library returns the right answer with a citation, and a hundred Teams pings to the HR team evaporate. This is the right scale of ambition — site-scoped retrieval over curated content is the retrieval problem AI can actually solve today.

The problem is the word curated. The average SharePoint site is fifteen years of sediment: drafts beside finals, 2022 beside 2026, “FINAL_v3_revised_USE THIS ONE.docx.” The agent ingests all of it democratically and answers from whatever retrieves best, with uniform confidence. It is a mirror for your site hygiene, and most mirrors in most tenants show something unflattering. Add shallow customization — scope and instructions, but no real actions or behaviors without graduating to declarative agents or Studio — plus weak discoverability and a metered bill nobody is watching, and you get the 6: a good product whose median deployment lands on bad inputs.

The workarounds that change the score

Three habits push SharePoint agents from a 6 to a personal 8:

  1. Scope to a curated subset, not the site. Don’t point the agent at fifteen years of sediment — point it at one folder containing only current, authoritative documents. A weekend of curation plus narrow scoping is the difference between an oracle and a rumor mill.
  2. Write instructions like a job description. “Answer only from the Policies library. If documents conflict, prefer the most recently modified and say a conflict exists. If the answer isn’t in scope, say so and name the site owner.” Two minutes of instructions eliminates half the failure modes.
  3. Assign an owner and a billing watcher. One named person prunes stale documents quarterly; one admin watches PAYG consumption monthly. Agents with owners stay sharp; orphaned agents decay into confident liars and surprise line items.

What Microsoft won’t tell you

  • The one-click pitch hides where the work went: it moved from building the agent to curating the content. The click is free; the curation was always the actual product.
  • Permission trimming protects confidentiality, not correctness. The agent will never leak what you can’t see — but it will happily serve you the obsolete version of what you can. Those are different guarantees, and only one is in the marketing.
  • PAYG is a feature and a trap in the same clause. No license barrier means organic adoption; metered billing with no assigned owner means the first governance conversation happens when finance forwards the invoice.

Bottom line

SharePoint agents are the cheapest experiment in the entire Copilot portfolio: one click, no licenses required, permission story already solved. Run that experiment on your best-kept site and you’ll likely keep it forever. Run it on a typical site and you’ve built a confident expert on your mess — which, to be fair, is what you asked for. Curate first; the click can wait a weekend.

← All scorecards