AI Agents Explained — How LLMs Take Actions

What is an AI Agent?

An AI agent is a system that uses an LLM as its "brain" to autonomously accomplish tasks. Unlike a simple chatbot that generates text responses, an agent can reason about a problem, decide which actions to take, execute those actions using external tools, observe the results, and iterate until the task is complete.

The key distinction is agency — the ability to take real-world actions. A chatbot can tell you the weather if you ask. An agent can check the weather API, look up your calendar, find a restaurant with outdoor seating, and make a reservation. It does this by combining LLM reasoning with tool use.

Agents vs Chatbots vs Chains

To understand agents, it helps to place them on a spectrum of autonomy:

• Chatbot: Takes a user message, generates a text response. No tool use, no planning. Example: a basic ChatGPT conversation.
• Chain: A fixed pipeline of LLM calls and tool uses, orchestrated by developer code. The workflow is predetermined. Example: a RAG pipeline that always retrieves documents, then generates an answer.
• Agent: The LLM decides the workflow dynamically. Given a goal and a set of tools, the agent figures out which tools to call, in what order, and when to stop. The developer defines the tools, but the LLM controls the flow.

This distinction is crucial. In a chain, the developer is the orchestrator. In an agent, the LLM is the orchestrator.

How Agents Work: Plan, Act, Observe

All agents follow a fundamental loop, regardless of the specific framework or implementation:

The Agent Loop

1. Plan: The LLM analyzes the user's goal and decides what step to take next. It considers what it already knows, what information it needs, and which tools are available.
2. Act: The agent executes the chosen action — calling a tool, running code, querying an API, searching the web, or writing to a file.
3. Observe: The agent receives the result of the action and adds it to its context.
4. Repeat: The agent loops back to planning with the new information. It continues until it has enough information to answer the user or has completed the task.

A Concrete Example

Suppose you ask an agent: "What was the GDP of Japan in 2024, and how does it compare to Germany?"

• Plan 1: I need to find Japan's 2024 GDP. I'll use the web search tool.
• Act 1: Calls search("Japan GDP 2024")
• Observe 1: "Japan's GDP in 2024 was approximately $4.2 trillion."
• Plan 2: Now I need Germany's GDP. I'll search again.
• Act 2: Calls search("Germany GDP 2024")
• Observe 2: "Germany's GDP in 2024 was approximately $4.5 trillion."
• Plan 3: I have both values. I can now compare them and answer.
• Final Response: "Japan's GDP in 2024 was $4.2 trillion, while Germany's was $4.5 trillion. Germany's economy was about 7% larger."

This loop — plan, act, observe, repeat — is the core of every agent architecture.

Tool Use and Function Calling

Tools are what give agents their power. Without tools, an LLM can only generate text. With tools, it can interact with the world.

What Are Tools?

A tool is a function that the agent can call. Each tool has:

• Name: A descriptive identifier (e.g., search_web, read_file)
• Description: Explains what the tool does and when to use it. The LLM reads this to decide when to call the tool.
• Parameters: The inputs the tool expects, with types and descriptions
• Return value: The output the tool produces

Common tool categories include:

• Information retrieval: Web search, database queries, API calls, file reading
• Code execution: Running Python, JavaScript, or shell commands in a sandbox
• Data manipulation: Creating spreadsheets, transforming data, generating charts
• Communication: Sending emails, posting messages, creating tickets
• System interaction: Reading/writing files, managing processes, deploying code

Function Calling

Function calling is the mechanism that enables tool use. When an LLM supports function calling (GPT-4, Claude, Gemini all do), you provide it with a list of available functions and their schemas. Instead of generating free-form text, the LLM can output a structured request to call a specific function with specific arguments.

Here is how it works in practice:

1. You define tools with JSON Schema descriptions
2. You send the user's message plus the tool definitions to the LLM
3. The LLM either responds with text (if no tool is needed) or with a function call request
4. If a function call is returned, your application executes it and sends the result back to the LLM
5. The LLM incorporates the result and either responds to the user or calls another function

The LLM never executes functions directly — it only requests them. Your application is always in control of execution. This is an important safety property.

The ReAct Pattern

The ReAct (Reasoning + Acting) pattern, introduced in a 2022 research paper, is the most widely used agent architecture. It structures the agent loop as a series of interleaved reasoning traces and actions.

How ReAct Works

In each step, the LLM generates a thought (reasoning about what to do) followed by an action (the tool to call). After receiving the observation (tool result), it generates another thought and action, or a final answer.

A ReAct trace looks like this:

Question: Who is the CEO of the company that makes the iPhone?
Thought: The iPhone is made by Apple. I need to find the current CEO of Apple.
Action: search("Apple CEO 2026")
Observation: "Tim Cook has been CEO of Apple since 2011."
Thought: I now have the answer.
Final Answer: The CEO of Apple, the company that makes the iPhone, is Tim Cook.

The key insight is that by making the LLM's reasoning explicit (the "Thought" steps), you get more accurate and debuggable agents. The LLM "thinks out loud" before acting, which reduces errors.

ReAct vs Pure Function Calling

Modern LLM function calling (GPT-4, Claude) essentially implements ReAct implicitly — the model reasons about which function to call and generates structured calls. However, explicit ReAct prompting can still be useful for complex multi-step tasks where you want visible reasoning traces, or when using models that do not support native function calling.

Agent Frameworks

Several frameworks make it easier to build agents:

LangGraph

LangGraph (part of the LangChain ecosystem) models agents as graphs where nodes are actions (LLM calls, tool uses) and edges define the flow. This gives you fine-grained control over the agent's behavior, supports cycles (essential for iterative agent loops), and integrates with LangChain's tool ecosystem. It is the most popular choice for production agents. See our LangChain tutorial for a practical example.

When to Use What

• Simple agent, one LLM: Use native function calling directly (OpenAI, Anthropic SDK)
• Complex workflows, need control: Use LangGraph
• Multi-agent teams: Use CrewAI or AutoGen
• Enterprise Microsoft stack: Use Semantic Kernel

Real-World Agent Applications

Agents are already being used in production across many domains:

• Code assistants: Cursor, GitHub Copilot Workspace, Devin — agents that can write, test, and debug code by reading files, running tests, and iterating.
• Research agents: Systems that search the web, read papers, synthesize information, and produce reports. They can spend minutes researching before answering a question.
• Data analysis: Agents that write SQL queries, execute them, create visualizations, and explain insights — all autonomously.
• Customer support: Agents that look up order information, check inventory, process refunds, and escalate to humans when needed.
• DevOps: Agents that monitor systems, diagnose issues, write fixes, and deploy patches with human approval.
• Personal assistants: Agents that manage calendars, send emails, book travel, and coordinate tasks across multiple services.

The pattern is consistent: agents excel at tasks that require multiple steps, tool use, and information gathering — tasks that would take a human several minutes of clicking and typing.

Limitations and Risks

Agents are powerful but far from perfect. Understanding their limitations is essential for safe deployment.

Reliability

Agents are probabilistic. They can choose the wrong tool, pass incorrect arguments, misunderstand observations, or loop endlessly. Unlike deterministic software, you cannot guarantee an agent will complete a task correctly every time. Production agents need guardrails: maximum iteration limits, human approval for critical actions, and fallback behaviors.

Hallucination in Actions

LLMs can hallucinate tool calls — requesting functions that do not exist, passing fabricated arguments, or claiming to have received results they never got. This is especially dangerous in agents because hallucinated actions can have real-world consequences.

Cost and Latency

Agents make many LLM calls per task (one per loop iteration). A complex agent task might involve 5-20 LLM calls, each costing money and adding latency. Simple tasks that could be handled by a single chain are better served by that chain — do not use agents when a deterministic pipeline works.

Safety

The ability to take actions makes agents inherently more dangerous than text-only LLMs. Key safety measures include:

• Sandboxing: Run code execution in isolated environments (containers, sandboxes)
• Human-in-the-loop: Require human approval for destructive or high-impact actions
• Least privilege: Give agents only the tools and permissions they need
• Monitoring: Log all agent actions for auditing and debugging
• Rate limits: Cap the number of tool calls and total cost per task