How AI Agents Work - The Loop, Tools and Memory

An AI agent is a program that uses a language model to pursue a goal over several steps, deciding for itself what to do next. If a chatbot is a question-and-answer machine, an agent is more like a junior colleague who takes a task, goes away, tries things and comes back with a result. This guide explains the moving parts so you can judge any agent product or build a simple one.

In plain English

Picture asking an assistant to "find out why the printer in accounting stopped working." A chatbot would offer generic advice. An agent would check the printer's status, read a log, try a fix, check again, and report what it found. It acts, looks at what happened, and decides the next move.

The four parts

1. A model: the decision maker. It reads the situation and chooses what to do.
2. Tools: things the agent can do, such as search, read a file, query a database or call a service.
3. A loop: the code that repeats think, act, observe until the job is done.
4. Memory and context: what the agent knows right now.

Everything else, whether a framework, a product name or a feature, is packaging around these four.

The loop

The cycle is the same almost everywhere:

1. The agent receives a goal and the list of tools.
2. The model decides: answer, or request a tool.
3. Your code (or the platform) runs the tool.
4. The result goes back to the model.
5. Repeat until the model gives a final answer or a limit is hit.

The model never runs anything itself. A program does, and that program is where limits, permissions and approvals belong. For a build-it-yourself walkthrough, see our guide on building your first agent.

What "memory" really means

Models do not remember in the human sense. What looks like memory is text that gets put back in front of the model:

• Working memory: the current conversation and tool results, held in the context the model sees right now. It has a size limit.
• Notes or files: the agent writes facts to a file or database and reads them later.
• Retrieved knowledge: documents searched and inserted when relevant. See our RAG guides.
• Saved preferences: a product may store facts about you and feed them back in each time.

If something is not in the context or stored somewhere your code reads, the model does not know it. See our guide on agent memory and context management.

Where agents help

• Tasks with uncertain paths, where the next step depends on what was found.
• Research and information gathering.
• Repetitive investigations that follow a pattern, such as triaging a ticket.
• Coding tasks with tests the agent can run to check its own work.

Where agents struggle

• Long tasks where early errors compound.
• Vague goals, which produce wandering.
• Situations where a single wrong action is costly.
• Tasks that a fixed script can do predictably. A script is cheaper, faster and easier to test.

Failure modes to know

• Looping: repeating the same step. See our debugging guide.
• Overreach: using a tool in a way you did not intend.
• Confident invention: filling gaps with plausible but made-up details.
• Context overflow: forgetting early instructions as the task grows.
• Manipulation: hostile text in a web page or email steering the agent. See our prompt injection guides.

Technical notes

• Most vendors offer some form of tool calling in their API, and many agent frameworks are thin wrappers over the loop above.
• Standards such as the Model Context Protocol make it easier to connect an agent to tools and data sources without custom code for each one. See our guide on MCP in practice.
• Reliability comes from engineering around the model: narrow scope, validated tool inputs, step and cost limits, logging and tests.
• Autonomy is a dial, not a switch. You choose how much the agent may do without asking.

Questions to ask about any agent

1. What can it do, exactly? What can it change?
2. What stops it? Is there a step, time and cost limit?
3. Who approves risky actions?
4. How would I know what it did? Is there a log?
5. How was it tested?

Next steps

• Read our guide on building your first agent.
• Read about giving agents tools safely.
• Sketch an agent for one of your own repetitive tasks and list its tools.
• Decide which of its actions would need a human yes.
• Read about evaluating agents so you can measure results.