Your Agent Is Only as Good as the Analysis That Goes Into It

by | Nov 16, 2026

Many AI agents being built today in organizations are completely lacking in analysis. They are being built from assumptions in someone’s head.

A developer got a request: “build an agent that . . . .”. They wrote some instructions, connected a few tools, tested a handful of cases, and implemented it. It worked well in the demo. Then real customers arrived with real complexity, and the gaps started showing. Missing knowledge. Fuzzy boundaries. No plan for what happens when the agent gets stuck.

None of that is a technology problem. It is an analysis problem.

An Agent Is a Bundle of Decisions

Every agent embodies dozens of choices. What is it for? What is it allowed to do? What does it know? What tools can it use? When does it ask for help? How do we know it is working?

If nobody makes those choices deliberately, they still get made, just implicitly, by whoever happened to write the instructions. That is how agents end up with overlapping responsibilities, missing guardrails, and success criteria nobody agreed on.

This is where BAs come in. Making implicit decisions explicit is exactly what we do.

The Agent Canvas

In my courses, we use an agent canvas to capture the analysis every agent needs. It is a one-page view covering:

  • Scope, role, and purpose: what problem it solves, for whom, and what is explicitly out of scope.
  • Success metrics: outcome, process, and experience measures.
  • End-user touchpoints: where people interact with it.
  • Tools: what systems it can read from and act on.
  • Instructions and knowledge: how it should behave and what it needs to know.
  • Memory needs: what it must remember across interactions.
  • Errors and escalations: what happens when it fails or gets stuck.
  • Human in the loop points: where people review, approve, or decide.
  • Constraints and guardrails: what it must never do.
  • Monitoring and risks: how we watch it and what could go wrong.

Each box is a conversation. Each conversation surfaces assumptions that would otherwise show up as production failures.

Tip 1: Start With Scope and Purpose, and Be Ruthless About It

A common agent failure I see is scope creep, or too broad of a scope.

Write one clear sentence describing the agent’s core job. Then list what is out of scope just as explicitly.

Tip 2: Treat Knowledge as a Requirement

An agent’s knowledge is its requirements, written for a machine. Which policies does it need? Which version is authoritative? Who keeps it current?

BAs are often the only people who know where the real rules live, including the ones that exist only in experienced staff’s heads. Capturing that knowledge, and identifying the gaps, is high-value analysis work.

Tip 3: Design the Failure Paths Before the Happy Path Ships

Agents handle the expected case well. The unexpected case is where trust is won or lost.

For every agent, walk through the error and exception scenarios. What if the data is missing? What if the customer is upset? What if the request is ambiguous? What if a downstream system is down? Define what the agent does and where it hands off. Then test those paths before go-live.

The Analysis Is the Asset

Here is something I find exciting. A well-built agent canvas outlives the agent. When the model changes, the vendor changes, or the business changes, the analysis remains. It becomes durable organizational knowledge about how the work should be done.

The teams that invest in that analysis will build agents that work and keep working. BAs can make sure that happens.

Learn the Agent Canvas

My AI/BA course series walks through the full agent canvas with a real case study, from scope to monitoring, plus how to coordinate multiple agents.

Visit www.maven.com/angela-wick to explore current courses and upcoming cohorts.

Agents are built fast now. Make sure the thinking behind them is solid.

Build Your AI-Ready BA Skills

Explore five live, instructor-led courses designed specifically for Business Analysts:

  • The Business Analyst Role in the Age of AI
  • AI Literacy for Business Analysts
  • AI-Accelerated Business Analysis
  • AI-Enabled Systems & Process Design
  • Analysis for Agentic AI: Enabling Real Outcomes

Choose the courses that fit your goals, or complete the full series to build your AI-ready BA skills.