An AI agent is a program that lets a language model work rather than just answer: it receives a goal instead of a question, breaks that goal into steps itself, calls tools along the way and checks after each step whether it has made progress. The model makes the decisions, the loop around it turns a single answer into a process. The umbrella term for the same principle is agentic AI.
The word is handed out generously at the moment. Plenty of things calling themselves agents are a chatbot with two extra buttons. So the distinction is worth making.
How it differs from a chatbot
A chatbot gets a question and writes an answer. Then the exchange is over. An agent gets a goal and stays in a loop until it reaches it, gives up, or runs out of budget.
| AI chatbot | AI agent | |
|---|---|---|
| Input | a question | a goal |
| Flow | one answer | several steps in a loop |
| Reach outside | only the context you supplied | tools: search, interfaces, files, databases |
| End | the answer is written | goal reached or run aborted |
| Typical failure | a wrong statement | a wrong action |
The last row matters most. You can read an invented statement and discard it before any harm is done. An action that has been carried out, an email sent or a record overwritten, has already happened. That is the real jump, and it changes how carefully such a system has to be built.
The building blocks
Three things separate an agent from a plain model call. Take one away and you are back to chat.
Tools
A tool is a function the model is allowed to call: a search, a database lookup, a write into the ticket system, a script. The model gets a description of what each tool is for and what inputs it needs, then decides on its own when to reach for it and with which values. How that wiring looks technically has recently converged on a standard, see Model Context Protocol.
The quality of an agent depends surprisingly heavily on the quality of those descriptions. A vaguely described tool gets used wrongly, and in a new way every time.
Memory
A model holds no state. Between two calls it remembers nothing beyond what you hand it again. An agent therefore needs storage of its own: the transcript of the current run, intermediate results, notes on what has already been tried and failed. For durable knowledge a search over your own documents usually joins in, see retrieval augmented generation.
All of that occupies room in the context window, so managing memory on longer runs is not a detail but core work: summarise, discard, keep the right things.
Planning
The agent has to derive steps from the goal and decide again after each one. Some systems write a rough plan up front and work through it, others decide purely step by step. Both work. What matters is less the planning strategy than the feedback: given an honest result after a tool call, an agent often corrects itself. Given nothing but an “OK”, it will happily carry on in the wrong direction.
Where agents work well today
There is a pattern that tells you fairly reliably whether a plan fits: agents are strong wherever the result is cheap to check.
- Research across several sources. Search, read, compare, summarise. The sources sit beside the answer at the end and can be spot-checked.
- Software development. The agent reads the codebase, changes something and runs the tests. The test result is hard feedback it can act on by itself.
- Routine work with a clear rule. Sorting incoming enquiries, pulling data out of documents, reconciling master data. A mistake is annoying but visible and repairable.
- First drafts with sign-off. The agent prepares, a person approves. Unspectacular, and in practice often the setup that actually pays.
Conversely, it gets hard wherever checking the result costs the same effort you were trying to save.
Where it gets difficult
Reliability across many steps
Errors multiply. A step that goes right nine times out of ten is decent on its own. Across twenty steps it becomes a chain that rarely runs clean. So the most important design decision is usually not which model to use but how short the chains are kept and where the checkpoints sit.
Cost per run
A chat costs one model call. An agent run costs as many as it needs steps, and every step drags the previous transcript along. Cost therefore grows not with the size of the task but with how tangled it is. A stop condition and a per-run cap belong in the design from the start, not after the first invoice. Billing is based on tokens.
Permissions
An agent acts with whatever rights you give it. It is also a system that reads text from the outside world, from web pages, emails and documents, and that text can contain instructions, see prompt injection. Put the two together and you have an uncomfortable combination. In practice: its own account rather than an admin login, write access only where it is needed, and human confirmation for anything irreversible.
Frequently asked questions about AI agents
Does every agent need a very large model? No, but the requirements differ from chat. Reliability in calling tools and sticking to formats matters more than breadth of knowledge. Smaller models rarely fail to understand the task, they fail at discipline across many steps.
What is the difference between agentic AI and an AI agent? Practically none. “Agentic AI” is the umbrella term for the way of working, “AI agent” names the concrete system. Marketing sticks both words on anything that uses a language model.
Can an agent work unsupervised in a live system? Technically yes, organisationally it is rarely wise. The usual middle ground is an agent that prepares everything and leaves the final, irreversible click to a person. That costs little speed and removes nearly all of the risk.
Why do agent projects fail in practice? Usually not because of the model. They fail on messy interfaces, missing permissions, data nobody ever tidied up, and tasks whose results nobody can check cheaply. Those are classic integration problems in new clothes.
Conclusion
The jump from chatbot to agent is smaller than it sounds, and its consequences are bigger. Technically you add a loop, a handful of tools and some storage. Organisationally you get a system that acts rather than answers, which shifts the whole question from “is the answer correct” to “who is accountable for the action”. Good agents are therefore an architecture problem more than a model problem: short chains, clear permissions, checkable results. How we build software, and where interfaces, permissions and tests come together while doing it, is described on our page about custom software development. What actually holds up when using AI in development is written down in the article AI in software development. At what point an agent pays off against plain automation is on our page about AI agents, and for anything beyond that there is a free consultation.