A chatbot produces a response. An agent pursues an outcome.
That difference sounds small until the system can search enterprise knowledge, call APIs, update records, generate code, send messages or initiate a workflow. The model is no longer only composing text. It is participating in operations.
Dependable agent design therefore requires a change in architecture and governance. Prompt quality remains important, but identity, authorization, tool contracts, evaluation, observability and failure containment become equally important.
Key takeaways
- Agency is created by tools, state and delegated authority—not by conversational style.
- Every tool call should be treated as a privileged software action.
- Autonomy should expand only after bounded workflows demonstrate reliable behaviour.
- Agent evaluation must include outcomes, tool use, security and recovery—not only answer quality.
- Human approval is most valuable at consequential boundaries, not inserted indiscriminately into every step.
A practical definition of an agent
NIST describes agentic AI as systems capable of goal-driven behaviour, independent decisions and dynamic interaction with systems and environments. In practical enterprise architecture, an agent normally combines:
- A model that interprets goals and selects next steps
- Instructions and policies
- Retrieval from approved knowledge
- Tools that read or change external systems
- State or memory across steps
- An orchestration loop that observes results and continues
- Controls that constrain and evaluate behaviour
A chatbot may have retrieval and still remain primarily informational. Agency becomes consequential when the system chooses actions and uses authority delegated by a user or organization.
The production control plane
Identity
The organization must know which agent, user and workload initiated an action. Shared anonymous credentials make attribution and containment difficult.
NIST’s work on software and AI-agent identity emphasizes standards-based identification and authorization. An agent should have a distinct workload identity, scoped credentials and traceable delegation—not a broadly privileged API key embedded in configuration.
Authorization
Authorization should answer more than “Can this service call the API?” It should consider:
- Which records may be read or changed
- Which operations are permitted
- Maximum financial or operational impact
- Whether user approval is required
- Whether the action is reversible
- Whether the request context is trustworthy
Apply least privilege to both tools and data. A research agent does not need payment authority. A scheduling agent may need to propose times but not delete an entire calendar.
Tool contracts
Tools need explicit schemas, validation and predictable failure responses. Descriptions should make preconditions and side effects clear enough that the agent can select correctly.
High-impact operations benefit from a two-stage pattern: first prepare and display the proposed change; then execute only after validation or approval.
State and memory
Memory improves continuity but also preserves errors, sensitive data and malicious instructions. Separate short-lived task state from durable user preferences. Define retention, access and deletion rules.
Do not treat retrieved content as trusted instructions. NIST has highlighted agent hijacking through indirect prompt injection, where hostile instructions embedded in external content attempt to redirect agent behaviour.
Observability
Log the goal, model and prompt versions, retrieved sources, tool requests, tool results, policy decisions, approvals, latency and final outcome. Sensitive information must be protected, but removing all detail makes investigation impossible.
Microsoft’s current generative-AI observability guidance emphasizes measuring both quality and safety through development and production. Agents need traces that connect the final answer to the actions that produced it.
Five autonomy levels
An organization can adopt agents progressively:
- Answer: Provide information with sources.
- Recommend: Propose an action without executing it.
- Prepare: Assemble the transaction or change for review.
- Execute with approval: Act after a person confirms consequential steps.
- Execute within policy: Act autonomously inside tightly bounded limits.
The correct level depends on reversibility, impact, confidence and oversight. A low-risk internal search workflow may reach level five quickly. A financial transaction or safety-relevant operational change may remain at level three or four.
Evaluate the system, not only the model
An agent evaluation set should include realistic tasks and adversarial conditions:
- Correct outcome achieved
- Appropriate tool selected
- Arguments validated
- Unauthorized actions refused
- Conflicting instructions handled safely
- Missing data recognized
- Uncertain cases escalated
- Duplicate execution prevented
- Partial failure recovered
- Evidence retained
Measure task completion alongside harmful-action rate, unnecessary tool calls, intervention frequency, cost and recovery time.
Offline test cases are essential but insufficient. Production inputs evolve, tools change and attackers adapt. Monitoring and periodic re-evaluation should be part of the operating model.
Design for containment
Assume that an agent will eventually misunderstand a request, encounter malicious content or receive an unexpected tool response. Containment patterns include:
- Read-only defaults
- Allowlists for tools and destinations
- Transaction and rate limits
- Sandboxed execution
- Approval thresholds
- Idempotency keys
- Automatic rollback where possible
- Circuit breakers
- Rapid credential revocation
Dependability is not the promise that an agent never fails. It is the ability to limit, detect, explain and recover from failure.
Where to begin
Select a workflow with clear success criteria, accessible evidence and reversible actions. Build the agent first as a recommender. Observe the quality of its plans and tool choices. Add execution only after the organization understands failure modes and has operational ownership.
The path from chatbot to dependable agent is not a larger prompt. It is a controlled expansion of authority.
Example: an exception-investigation agent
Imagine an internal agent asked to explain why two operational reports disagree. A chatbot might produce plausible causes from general knowledge. A dependable agent follows a controlled investigation plan.
First, it confirms the reporting period, population and calculation context. It retrieves approved definitions and identifies authoritative datasets. Read-only tools collect refresh timestamps, row counts and reconciliation results. The agent then presents a proposed explanation with direct evidence and clearly marks any unresolved gap.
If a rerun could correct the issue, the agent does not immediately execute it. It prepares the proposed parameters, identifies downstream impact and requests approval from the responsible operator. The execution tool validates the environment, prevents duplicate runs and returns a traceable result. If a control fails, the workflow stops and preserves the investigation state.
This example illustrates why dependable agency is a system property. The model contributes reasoning and communication, but identity, tool boundaries, evidence, idempotency and approval determine whether the outcome is safe enough for production. The same pattern can be reused for service operations, finance, engineering and compliance workflows.
Each expansion should be earned through observed performance, not assumed capability.