In 2023, AI agents were mostly demos. In 2024, many teams have tried to put them into production, and discovered that an agent that works 80% of the time in a demo can be unusable when it fails unpredictably on the other 20%.
The good news is that the toolkit has matured. OpenAI introduced Structured Outputs in August, guaranteeing responses that match a JSON schema. Graph-based frameworks such as LangGraph make agent control flow explicit. Tracing and evaluation tools are widely available. And models keep improving at tool use; OpenAI's o1-preview, released this month, shows what longer reasoning before answering can do on hard problems.
Here are the practices we have found make the biggest difference.
1. Constrain the control flow
The most reliable "agents" are often workflows with LLM steps, not open-ended loops. If you know the steps (classify the request, retrieve data, draft a reply, check policy), encode them explicitly and let the model handle the parts that need language understanding.
flowchart LR
I[Incoming request] --> C[LLM: classify intent]
C -->|refund| R[Fetch order + policy]
C -->|question| K[RAG over help centre]
C -->|other| H[Human queue]
R --> D[LLM: draft decision]
D --> P{Policy check<br/>deterministic}
P -->|within limits| A[Execute + reply]
P -->|exceeds limits| H
K --> Rep[LLM: draft answer] --> AReserve open-ended agent loops for genuinely open-ended tasks, and even then cap steps, time and cost.
2. Use structured outputs everywhere
Parsing free text is a major source of agent failures. Define schemas for every decision the model makes:
from pydantic import BaseModel, Field
from typing import Literal
from openai import OpenAI
class RefundDecision(BaseModel):
action: Literal["approve", "deny", "escalate"]
amount: float = Field(ge=0)
reason: str
policy_clause: str
client = OpenAI()
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "Decide refunds strictly according to the policy provided."},
{"role": "user", "content": f"Policy:\n{policy}\n\nOrder:\n{order_json}\n\nRequest:\n{message}"},
],
response_format=RefundDecision,
)
decision = completion.choices[0].message.parsed # a validated RefundDecisionThen validate business rules in code (amount at most the order total, customer eligible) rather than trusting the model.
3. Design tools like good APIs
Tools are the agent's hands. Poorly designed tools cause most real-world agent failures.
- Narrow and specific:
get_order(order_id)rather thanquery_database(sql). - Strict schemas with enums, ranges and required fields.
- Clear descriptions that state when to use the tool and what it returns.
- Helpful errors: "order_id must look like ORD-123456" lets the model self-correct.
- Idempotent where possible, because agents retry.
- Least privilege: each tool has only the permissions its job requires.
4. Manage context deliberately
Agents degrade as their context fills with irrelevant tool output. Summarise long results, return only needed fields, and keep system instructions concise and prominent. For long tasks, persist state outside the prompt and give the model a compact view of it.
5. Evaluate continuously
You cannot improve what you do not measure, and agent behaviour changes whenever you change a prompt, model or tool.
- Trace everything: every model call, tool call, input, output, latency and cost.
- Build evaluation sets from real traces, especially failures.
- Score at multiple levels: was the right tool called with the right arguments? Was the final outcome correct? Did it follow policy?
- Run evaluations in CI before deploying any change.
- Use models as graders carefully: calibrate them against human judgements.
6. Design for failure
- Timeouts and retries with backoff on model and tool calls.
- Step and budget limits so runaway loops end gracefully.
- Fallbacks: a simpler path or a human when confidence is low or validation fails.
- Human checkpoints for irreversible or high-impact actions, with a clear summary of what will happen.
7. Security is part of reliability
An agent that can be manipulated is not reliable. Treat content from users, documents, web pages and tool results as untrusted. Keep authorisation decisions in code, not in the prompt. Log actions with the identity of the user and the agent. Test with adversarial inputs, including prompt injection in retrieved content.
A maturity path
- Assistant: the model drafts; humans act.
- Supervised agent: the model acts on low-risk steps; humans approve high-risk ones.
- Autonomous within bounds: the model acts end to end inside strict policies, with monitoring and escalation.
Move up the path one workflow at a time, only when evaluations and production metrics justify it.
The bottom line
Reliable agents are mostly good software engineering applied to a probabilistic component: explicit control flow, strict interfaces, defensive validation, observability and testing. The model provides the language understanding; everything around it provides the reliability.
Our agentic AI security team builds production agents this way, with evaluation, observability and security designed in from the start.