An AI agent is software that uses a large language model to reason about tasks, choose tools, take actions, and repeat until it reaches a goal. Unlike a chatbot, which only responds to prompts, an agent acts.
Building one is surprisingly accessible. A minimal agent runs in around 40 lines of Python. The hard part is not building it — it is making it reliable, cost-effective, and accountable in production. This guide covers the architecture, the code, the frameworks, and the production concerns that separate a demo from a system your business can depend on.
What Is an AI Agent and How Does It Differ from a Chatbot?
A chatbot takes a prompt and returns a response. An agent takes a goal and figures out how to achieve it.
The core loop is: Observe → Think → Act → Observe. The language model decides the next step at each turn, not a hardcoded workflow. If the agent needs information, it searches the web. If it needs to save data, it writes to a file. If it needs to run a calculation, it calls a function. The LLM orchestrates the process, choosing from the tools available to it.
One practitioner who builds production agents across multiple industries describes the distinction clearly: a workflow is A → B → C, always in that order. An agent is “you give your large language model a toolbox and say — fix the washing machine. It figures out which tool to use and in what order.”
There are several types. A single agent pairs one LLM with a set of tools. A multi-agent system uses multiple specialised agents that collaborate — one handles research, another writes reports, a third checks quality. Reactive agents respond to triggers. Goal-driven agents pursue objectives autonomously over time.
The business case is straightforward: agents handle complex, multi-step tasks that previously required human judgement. Research, data analysis, code generation, customer support triage — all of these are now within reach.
How Do You Build an AI Agent from Scratch?
Here is the step-by-step process, from blank file to working agent.
Step 1: Define the Goal
Be specific. “Answer customer questions using our documentation” is a goal. “Be helpful” is not. One agent, one mission. As one experienced builder puts it: if a single agent tries to be a doctor, a receptionist, and a radiologist, it will hallucinate 40% of the time.
Step 2: Choose a Language Model
Pick a model based on your requirements for cost, latency, context window, and tool-calling capability. Leading foundation model providers offer APIs with different price-performance tradeoffs. For a tutorial or prototype, entry-level models cost fractions of a penny per call.
Step 3: Define the Tools
Tools are what make an agent more than an autocomplete. A tool is any function the agent can call — a web search, a database query, an API call, a file write, a code execution step.
In Python, creating a custom tool is as simple as writing a function and wrapping it with a decorator or helper:
from langchain.tools import Tool
from duckduckgo_search import DDGS
search = DDGS()
search_tool = Tool(
name="web_search",
func=search.run,
description="Search the web for current information"
)The name and description matter. The LLM reads them to decide when to use each tool. A vague description means the agent picks the wrong tool at the wrong time.
Step 4: Build the Agent Loop
The core logic: send a prompt and context to the LLM, parse the response, if it requests a tool call then execute the tool, feed the result back, and repeat until the agent returns a final answer.
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([
("system", "You are a research assistant. Use tools when needed."),
("placeholder", "{chat_history}"),
("human", "{query}"),
("placeholder", "{agent_scratchpad}"),
])
agent = create_tool_calling_agent(llm=llm, prompt=prompt, tools=[search_tool])
executor = AgentExecutor(agent=agent, tools=[search_tool], verbose=True)
response = executor.invoke({"query": "What is the population of France?"})When you run this with verbose mode on, you see the chain of thought: the agent reasons about what it needs, calls a tool, observes the result, and decides the next step. One tutorial demonstrated an agent that, given a question about a country’s population, decided to search the web, parsed the results, and returned a cited answer — all autonomously.
Step 5: Add Memory
Without memory, agents start from zero every conversation. Short-term memory holds the current conversation context. Long-term memory persists across sessions using a vector database or structured store.
Production memory is more layered than this. Experienced builders describe a taxonomy: working memory for active reasoning, episodic memory for past interactions, semantic memory for domain knowledge (typically a retrieval-augmented generation setup), and procedural memory for workflows encoded in system prompts.
Step 6: Add Guardrails
Input validation, output filtering, cost limits, timeout controls, and human-in-the-loop checkpoints for high-stakes actions. Any action that creates, deletes, spends, or sends data above a defined threshold should require human approval. The cost of being wrong always exceeds the cost of an approval step.
What Frameworks Should You Consider?
The framework you choose depends on how complex your agent needs to be.
| Approach | When to Use | Complexity | Flexibility |
|---|---|---|---|
| LLM provider SDKs | Simple, single-purpose agents | Low | High |
| Orchestration frameworks | Multi-step chains, RAG, tool calling | Medium | Medium-High |
| Agent-specific frameworks | Multi-agent collaboration | High | Medium |
| No-code platforms | Non-developers, simple workflows | Low | Low |
LLM provider SDKs give you maximum control with minimum abstraction. You call the API, handle tool calls, manage state. This is the right starting point for most developers building their first agent.
Orchestration frameworks like popular open-source chaining libraries provide pre-built components for tool calling, memory, prompt templates, and output parsing. Good for rapid prototyping. One tutorial showed a research agent built in under 40 lines using these abstractions. The tradeoff is that the framework can add complexity when you need to customise behaviour.
Agent-specific frameworks are designed for multi-agent systems. They handle collaboration patterns — which agent hands off to which, how state flows between them. One production builder demonstrated structuring agent collaboration as a state graph: agent A ingests data, agent B detects anomalies, and conditional edges determine whether the flow continues to classification or terminates. “We can’t just leave it to our AI agents — they wouldn’t know how to collaborate.”
No-code platforms offer visual agent builders. Limited customisation but fast deployment for non-technical teams.
Start with the SDK for simple agents. Move to a framework when you need multi-agent orchestration or complex memory.
How Do You Take an AI Agent to Production?
This is where most projects fail. A working demo is not a production system.
Hallucination Prevention
LLMs hallucinate, and agents compound hallucination across reasoning, tools, and memory simultaneously. Three mitigations work in practice.
Chain of verification: the agent generates an answer, then generates verification questions about that answer, then revises. Research shows a 28% hallucination reduction from making the agent interrogate itself. In graph-based frameworks, that is just two extra nodes.
Enforce structured output: if your agent returns free text when you expect structured data, that is a reliability failure. Use schema validation at the output layer. Define the exact fields your response must contain and reject anything that does not conform.
Human-in-the-loop for high stakes: any action above a risk threshold gets human approval before execution.
Prompt Injection Protection
Sanitise inputs. Use system prompts with explicit guardrails. Implement output filtering. One builder describes fine-tuning a model on 50 example responses — correct versus dangerous — to bake safety rules into the model’s behaviour rather than relying on prompt-level instructions alone.
Cost Management
Set token budgets per task. Cache repeated queries. Use cheaper models for simple subtasks and reserve expensive models for reasoning-heavy steps. One practitioner reported spending only a few pennies during hours of continuous API usage by choosing an efficient model tier.
Observability
Log every agent step: the prompt, the LLM response, the tool called, the tool result, the next decision. “A two hundred thousand dollar hallucination went unnoticed for eleven days” in one audited system — because nobody was watching the output logs. Tools exist for tracing agent execution in both hosted and self-hosted configurations.
How Do You Track and Monitor AI Agent Performance?
Once agents are in production, three questions matter: How long do tasks take? How much do they cost? Are they producing good results?
Time Tracking for Agents
Log session duration, compute time, and task completion time for every agent run. This is not optional — it is the foundation for billing clients for agent work. If an agent spends three hours on a legal research task, that time has value and needs to appear on an invoice.
For a practical guide to setting this up, see how to track time for AI agents.
Cost Tracking
Token usage, API call costs, tool execution costs, and infrastructure overhead. Without this data, you cannot price agent services accurately. Track at the task level, not just the aggregate.
Quality Metrics
Monitor three signals in production. Output schema compliance: if your validation pass rate drops below 95%, something has changed in the data or model. Tool call success rate: high retries mean a tool is failing and the agent is covering for it. Latency per agent node: a spike tells you exactly which node broke before your users notice.
The Business Case
Firms deploying AI agents need the same accountability and time tracking they apply to human workers. A unified view of human and agent work is essential for billing, capacity planning, and profitability analysis. Firms that track agent time and cost from day one will have pricing data when the market matures. Those that wait will be guessing.
For more on billing for agent work, see our guide on AI agent billable hours.
Key Takeaway: Build agents with a specific goal, one mission per agent, structured input and output, and production guardrails. Then track their time and cost like any other team member.
Frequently Asked Questions
What is an AI agent?
An AI agent is software that uses a large language model to reason about tasks, decide which tools to use, take actions, and iterate until a goal is achieved. Unlike a chatbot, an agent acts autonomously across multiple steps.
How do you build an AI agent from scratch?
Define a specific goal, choose an LLM, define the tools the agent can call, build the agent loop (prompt → LLM → tool call → observe → repeat), add memory for context persistence, and add guardrails for safety and cost control.
What programming language is best for building AI agents?
Python is the most common choice due to its mature ecosystem of LLM libraries, agent frameworks, and community support. Most tutorials and frameworks are Python-first.
What frameworks are available for building AI agents?
Options range from LLM provider SDKs for maximum control, to orchestration frameworks for pre-built components, to agent-specific frameworks for multi-agent collaboration. No-code platforms also exist for non-developers.
How much does it cost to run an AI agent?
Costs depend on the model, token usage, and tool calls. Entry-level models cost fractions of a penny per request. Production agents with complex reasoning and multiple tool calls can cost more, which is why per-task cost tracking is essential.
How do you prevent AI agent hallucinations?
Use chain of verification (the agent questions its own answers), enforce structured output with schema validation, ground responses in retrieved documents, and implement human-in-the-loop approval for high-stakes actions.
How do you track AI agent performance and time?
Log session duration, token usage, API costs, task completion rates, and quality metrics for every agent run. Use observability tools to trace execution step by step. Treat agent time as billable work that needs the same tracking as human hours.
Built an AI agent? Track it like a team member.
Keito tracks time, cost, and output for your AI agents alongside your human team — so you can bill accurately and measure ROI.