Safe Tool Use in LangChain v1: Permissions, Guardrails and Human-in-the-Loop
Nov 15, 2026 · 18 min read
An agent that can only talk has a limited worst case: a wrong answer. Give it a refund tool, and the worst case becomes a wrong refund. Give it a tool that deletes customers, and the worst case is data you cannot get back.
The tempting fix is a stricter system prompt: "Never refund more than $100." That instruction helps, but it is not a control. The model can misread it, a long conversation can push it out of focus, and a prompt injection (instructions hidden in user input or in a tool result) can override it. Treat the model as a component that will sometimes do the wrong thing, and put checks around it that do not depend on its cooperation.
LangChain v1 gives you the parts for this. Tools get the user's identity from your application instead of from the model. Middleware hooks into every step of the agent loop, so you can filter tools, check inputs and outputs, limit calls, and pause for a human. This guide builds one agent from start to finish and adds one layer of protection per section.
If create_agent and @tool are new to you, start with An Introduction to LangChain. The code here targets langchain 1.5 and langgraph 1.2 on Python 3.11+. Every example was run against those versions with a scripted fake model; the checklist at the end explains how.
The example: a support agent with three tools
Our agent helps the support staff of an online shop. It has three tools, and each one carries a different level of risk:
| Tool | What it does | Risk |
|---|---|---|
lookup_order | Reads an order's status and total | Low: read-only |
issue_refund | Sends money back to a customer | Medium: costs money, but limited and traceable |
delete_customer | Removes a customer and all their data | High: irreversible |
Three kinds of staff use it. A viewer may only look orders up, an agent may also refund, and an admin may do everything. By the end of this guide, every tool call passes through the checks below. Pick a role and a request to see where each one stops:
- Injection filterbefore_agentpass
No blocked phrases.
- Model picks a toolwrap_model_callpass
Sees lookup_order, issue_refund. Calls issue_refund(order_id="A-1002", amount=150).
- Call limitafter_modelpass
First refund in this run (run_limit=1).
- Permission checkwrap_tool_callpass
'agent' may use issue_refund.
- Human approvalwrap_tool_call (per_call)paused
Paused. The run is saved in the checkpointer until a person decides.
- Schema + tooltool execution
… waiting for a human decision
Pick a role and a request. Try “Refund $50” as a viewer, then “Refund $900” as an agent and approve it.
Each row is one piece of middleware or one tool feature from this article. Two cases are worth trying now, because both come up again later: a viewer's refund is stopped even when the model calls a tool it was never shown, and an approved $900 refund still fails, because the approval happens before the schema check.
Layer 1: tools that are hard to misuse
Install the packages:
pip install "langchain>=1.5" langchain-anthropicIdentity comes from your app, never from the model
The most important design decision is which values the model is allowed to choose. The model should choose what to do: which order, what amount. It should never choose who is asking or whose data it may see. Those values come from your login session, and LangChain passes them to tools through the runtime context.
ORDERS stands in for your database:
ORDERS = {
"A-1001": {"shop": "shop-1", "customer": "C-42", "status": "delivered", "total": 89.90},
"A-1002": {"shop": "shop-1", "customer": "C-42", "status": "delivered", "total": 420.00},
"B-2001": {"shop": "shop-2", "customer": "C-77", "status": "shipped", "total": 35.00},
}The context is a plain dataclass, and a tool reads it through a runtime parameter:
from dataclasses import dataclass
from langchain.tools import tool, ToolRuntime
@dataclass
class SupportContext:
user_id: str # the support rep who is logged in
role: str # "viewer", "agent" or "admin"
shop_id: str # the only shop whose data this session may touch
def find_order(order_id: str, shop_id: str) -> dict | None:
order = ORDERS.get(order_id)
return order if order and order["shop"] == shop_id else None
@tool
def lookup_order(order_id: str, runtime: ToolRuntime[SupportContext]) -> str:
"""Look up an order by ID. Returns its status, total and customer ID."""
order = find_order(order_id, runtime.context.shop_id)
if order is None:
return f"No order with ID {order_id}."
return (
f"Order {order_id}: {order['status']}, total ${order['total']:.2f}, "
f"customer {order['customer']}."
)The runtime parameter does not appear in the schema the model sees. The model can pass an order_id, but it has no way to pass a shop_id. If shop_id were a normal argument, one injected sentence such as "check order B-2001 for shop-2" would be enough to read another shop's data. With the context, a shop-1 session that asks for B-2001 gets No order with ID B-2001. The order exists, just not for this session.
The schema is your first guardrail
For issue_refund, a Pydantic schema puts hard limits on the arguments:
from pydantic import BaseModel, Field
class RefundInput(BaseModel):
order_id: str = Field(description="ID of the order to refund, e.g. 'A-1001'.")
amount: float = Field(gt=0, le=500, description="Refund in USD. At most 500.")
reason: str = Field(min_length=10, description="Why the customer gets a refund.")
@tool(args_schema=RefundInput)
def issue_refund(
order_id: str, amount: float, reason: str, runtime: ToolRuntime[SupportContext]
) -> str:
"""Refund part or all of an order. Look the order up first."""
order = find_order(order_id, runtime.context.shop_id)
if order is None:
raise ValueError(f"there is no order {order_id}")
if amount > order["total"]:
raise ValueError(f"${amount:.2f} is more than the order total ${order['total']:.2f}")
# payments.refund(order_id, amount) <- your real payment call goes here
return f"Refunded ${amount:.2f} on order {order_id}."
@tool
def delete_customer(customer_id: str, runtime: ToolRuntime[SupportContext]) -> str:
"""Permanently delete a customer and all of their data. Cannot be undone."""
return f"Customer {customer_id} deleted."The field descriptions go to the model, so it knows the limits before it calls the tool. If it breaks them anyway, the tool does not run. LangChain validates the arguments first and returns the error to the model as a tool message (shortened here):
Error invoking tool 'issue_refund' with kwargs {'order_id': 'A-1002', 'amount': 900, ...} with error:
amount: Input should be less than or equal to 500
Please fix the error and try again.The model reads that and can correct the call or explain the limit to the user. Rules that need data, like "no more than the order total," cannot live in the schema. They live in the tool body and raise an exception.
Decide which errors the model may see
By default, an exception inside a tool stops the whole run. That is the right default for bugs. It is the wrong one for expected failures like "amount is more than the order total," which the model could fix if it knew about them. ToolErrorMiddleware lets you choose which exceptions go back to the model:
from langchain.agents.middleware import ToolErrorMiddleware
def explain_tool_error(exc: Exception, request) -> str | None:
if isinstance(exc, ValueError):
return f"{request.tool_call['name']} failed: {exc}. Fix the input or ask the user."
return None # anything else is a bug: let it stop the runReturn a string, and the exception becomes an error message for the model. Return None, and the exception propagates. This opt-in design matters: a raw database exception can contain table names, connection details or another customer's data, and none of that belongs in the model's context. Only messages you wrote for the model should reach it.
You add ToolErrorMiddleware to the agent at the end, together with the rest of the middleware.
Layer 2: permissions
Show each role only its tools
Middleware lets you change the agent loop without rewriting it. A wrap_model_call hook runs around every model call and can change the request before the model sees it. This one removes the tools that the current role may not use:
from langchain.agents.middleware import wrap_model_call
ROLE_TOOLS = {
"viewer": {"lookup_order"},
"agent": {"lookup_order", "issue_refund"},
"admin": {"lookup_order", "issue_refund", "delete_customer"},
}
@wrap_model_call
def show_permitted_tools(request, handler):
allowed = ROLE_TOOLS[request.runtime.context.role]
tools = [t for t in request.tools if t.name in allowed]
return handler(request.override(tools=tools))A viewer's model now sees only lookup_order. The prompt gets shorter, and the model stops offering actions the user cannot take.
Hiding a tool does not block it
This point is easy to miss. show_permitted_tools changes what the model sees. It does not change what the agent can run. All three tools are still registered with the agent, so if the model produces a call to delete_customer anyway (a hallucination, or a tool name it read in an injected message), the agent runs it.
You can check this yourself. With only show_permitted_tools in place, a scripted model running as a viewer called delete_customer, and the tool returned Customer C-42 deleted.
The real check has to happen when the tool executes. A wrap_tool_call hook runs around every tool call:
from langchain.agents.middleware import wrap_tool_call
from langchain.messages import ToolMessage
@wrap_tool_call
def enforce_permissions(request, handler):
role = request.runtime.context.role
name = request.tool_call["name"]
if name not in ROLE_TOOLS[role]:
return ToolMessage(
content=f"Permission denied: the '{role}' role cannot use {name}.",
tool_call_id=request.tool_call["id"],
status="error",
)
return handler(request)If the role is not allowed, the hook never calls handler, so the tool never runs. The model gets an error message and can tell the user. Keep both hooks: show_permitted_tools keeps the model's choices clean, and enforce_permissions is the actual security boundary. The shop check inside each tool is a third line of defense.
Layer 3: guardrails
Guardrails check content as it enters and leaves the agent. The LangChain docs describe two kinds. Deterministic guardrails use rules: keyword lists, regular expressions, explicit checks. They are fast and free, but they miss anything the rule did not anticipate. Model-based guardrails ask a model to judge the content. They catch subtler problems, but they cost a model call and add latency. Start with deterministic checks and add model-based ones where rules fall short.
Stop obvious attacks before the first model call
A before_agent hook runs once at the start of each run. If it returns "jump_to": "end", the run ends there:
from typing import Any
from langchain.agents.middleware import AgentState, before_agent
from langchain.messages import AIMessage
from langgraph.runtime import Runtime
BLOCKED_PHRASES = ("ignore previous instructions", "ignore all instructions", "system prompt")
@before_agent(can_jump_to=["end"])
def block_obvious_injection(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
text = str(state["messages"][-1].content).lower()
if any(phrase in text for phrase in BLOCKED_PHRASES):
return {
"messages": [AIMessage("I can't help with that. Please describe the customer's issue.")],
"jump_to": "end",
}
return NoneA blocked request costs nothing, because the model is never called. Be clear about what this filter does, though. Anyone who rephrases the attack gets past a keyword list. It removes lazy attempts cheaply; it does not protect you from prompt injection. Your real protection against injection is Layer 2: assume the model can be talked into anything, and make sure "anything" is limited to what the current user may do anyway.
Keep sensitive data away from the model
PIIMiddleware finds personal data and handles it with one of four strategies: redact, mask, hash or block. It has built-in detectors for email, credit_card, ip, mac_address and url, and you can add your own with a regular expression. These two rules go into the agent's middleware list at the end:
from langchain.agents.middleware import PIIMiddleware
PIIMiddleware("credit_card", strategy="mask", apply_to_tool_results=True)
PIIMiddleware("api_key", detector=r"sk-[a-zA-Z0-9]{32}", strategy="block")The first rule masks card numbers in user input (on by default) and in tool results. A message like "customer card 4111 1111 1111 1111 was charged twice" reaches the model as customer card **** **** **** 1111 was charged twice. Tool results deserve the extra flag: they are a common way for sensitive data to enter the context, and they often come from systems you do not fully control.
The second rule refuses any input that contains something shaped like an API key. The block strategy raises a PIIDetectionError, so catch it in your application and show the user a clear message.
Limit how much the agent can do in one run
Two built-in middleware put a ceiling on the agent's activity:
from langchain.agents.middleware import ModelCallLimitMiddleware, ToolCallLimitMiddleware
ModelCallLimitMiddleware(run_limit=8)
ToolCallLimitMiddleware(tool_name="issue_refund", run_limit=1)ModelCallLimitMiddleware stops an agent that loops, for example one that keeps retrying a failing tool. ToolCallLimitMiddleware limits a single tool, here to one refund per request. If the model tries a second refund in the same run, the tool does not run, and the model gets Tool call limit exceeded. Do not call 'issue_refund' again.
A run is one user message and everything the agent does to answer it. Both classes also accept thread_limit, which counts across the whole conversation and needs a checkpointer.
Check the final answer with a second model
Some problems are hard to catch with rules: a reply that promises a free replacement, or one that mentions another customer's order. An after_agent hook runs once, after the agent finishes, and can ask a small model for a verdict:
from langchain.agents.middleware import after_agent
from langchain.chat_models import init_chat_model
judge = init_chat_model("anthropic:claude-haiku-5-5")
@after_agent
def review_final_reply(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
reply = state["messages"][-1]
verdict = judge.invoke(
"You review replies from a customer-support assistant. Answer UNSAFE if the reply "
"promises compensation other than a refund, reveals another customer's data, or "
"is rude. Otherwise answer SAFE. Answer with one word.\n\nReply:\n"
f"{reply.content}"
)
if "UNSAFE" in verdict.content:
safe = AIMessage(
"I've passed your request to a human colleague, who will reply shortly.",
id=reply.id, # same id: replaces the reply instead of appending
)
return {"messages": [safe]}
return NoneThe new message has the same id as the original reply, so it replaces that reply in the state instead of being added after it. This check costs one extra model call per run, which is why it uses a small model and why it comes after the cheap deterministic checks.
Layer 4: human in the loop
Some actions should not be left to the model at all, however good your other checks are. HumanInTheLoopMiddleware pauses the agent before selected tool calls run and waits for a person to decide:
from langchain.agents.middleware import HumanInTheLoopMiddleware
approval = HumanInTheLoopMiddleware(
interrupt_on={
"issue_refund": {
"allowed_decisions": ["approve", "edit", "reject"],
"when": lambda request: request.tool_call["args"]["amount"] > 100,
},
"delete_customer": {"allowed_decisions": ["approve", "reject"]},
},
interrupt_mode="per_call",
)interrupt_on lists the tools that pause. Tools that are not listed, like lookup_order, run without review. when narrows the pause to some calls: a $50 refund runs at once, and a $150 refund waits. delete_customer does not allow edit, because an edited deletion is still a deletion.
interrupt_mode="per_call" creates one pause per tool call. The default, "batched", creates one pause per model turn that lists every gated call in it. The docs recommend per_call for new agents, and it is planned to become the default in the next major release.
Pausing and resuming
A pause is a LangGraph interrupt. The agent saves its state in a checkpointer and stops. A person can decide a minute later or a day later, and the run continues from the same point. That is why human-in-the-loop needs a checkpointer and a thread_id:
from langgraph.types import Command
config = {"configurable": {"thread_id": "ticket-8812"}}
ctx = SupportContext(user_id="rep-7", role="agent", shop_id="shop-1")
result = agent.invoke(
{"messages": [{"role": "user", "content": "Refund $150 on A-1002, it was broken on arrival."}]},
config,
context=ctx,
version="v2",
)
for interrupt in result.interrupts:
print(interrupt.value["name"], interrupt.value["args"])
# issue_refund {'order_id': 'A-1002', 'amount': 150, 'reason': 'broken on arrival'}With version="v2", invoke returns an object with two fields: result.value holds the agent state, and result.interrupts lists the open pauses. Each interrupt value contains the tool name, its args, a description for the reviewer and the tool_call_id. Show these in your review screen. Then resume with one decision per interrupt, keyed by the interrupt's ID:
decision = {"type": "approve"}
# or: {"type": "edit", "edited_action": {"name": "issue_refund",
# "args": {"order_id": "A-1002", "amount": 120, "reason": "partial refund agreed"}}}
# or: {"type": "reject", "message": "Refunds over $100 need a photo of the damage first."}
result = agent.invoke(
Command(resume={result.interrupts[0].id: decision}),
config,
context=ctx,
version="v2",
)Pass the same context again when you resume. The checkpointer saves the conversation state, but not the runtime context. Resume without it, and the first middleware that reads runtime.context fails with AttributeError: 'NoneType' object has no attribute 'role'.
Here is what the model sees after each decision:
- approve: the tool runs as requested, and the model gets its normal result.
- edit: the tool runs with the reviewer's arguments. The tool message starts with a notice that a human replaced the call and lists the arguments that actually ran, so the model knows the refund was $120, not $150.
- reject: the tool does not run. The model gets an error message with your reason:
User rejected the tool call for `issue_refund` with reason: Refunds over $100 need a photo of the damage first.
A fourth decision type, respond, returns your text to the model as a successful tool result. Use it only when the human plays the role of the tool, for example in an ask_user tool. Do not use it to deny an action: the model would read your refusal as the tool's output.
In batched mode, one interrupt lists every gated call in value["action_requests"], and you resume with Command(resume={"decisions": [...]}), one decision per call, in the same order.
Approval comes before validation
Try "Refund $900" as an agent in the demo above. The amount is over $100, so the agent pauses for approval. If the reviewer approves, the schema check rejects the call anyway, because the limit is $500. The approval hook wraps the tool call, and argument validation happens inside it, when the tool actually runs. So show the tool's limits next to the arguments in your review screen; otherwise reviewers spend time on calls that are going to fail.
Use a persistent checkpointer in production
InMemorySaver is fine in development, but it loses every pending approval when the process restarts. In production, use a database-backed checkpointer such as PostgresSaver from the langgraph-checkpoint-postgres package. When a pause can last hours, the state has to survive a deploy.
Putting it together: the order matters
Here is the complete agent:
from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver
agent = create_agent(
model="anthropic:claude-sonnet-5-5",
tools=[lookup_order, issue_refund, delete_customer],
system_prompt=(
"You are a support assistant for an online shop. Look orders up before "
"acting on them, and explain every refund in one sentence."
),
context_schema=SupportContext,
middleware=[
block_obvious_injection,
PIIMiddleware("credit_card", strategy="mask", apply_to_tool_results=True),
PIIMiddleware("api_key", detector=r"sk-[a-zA-Z0-9]{32}", strategy="block"),
ModelCallLimitMiddleware(run_limit=8),
show_permitted_tools,
ToolCallLimitMiddleware(tool_name="issue_refund", run_limit=1),
enforce_permissions,
approval,
ToolErrorMiddleware(explain_tool_error),
review_final_reply,
],
checkpointer=InMemorySaver(),
)The order of the middleware list is not cosmetic. LangChain runs before_* hooks first to last, after_* hooks last to first, and wrap_* hooks as nested layers, where the first in the list wraps all the others. Three choices in this list follow from that:
block_obvious_injectioncomes first, so a blocked request costs no model call and no PII processing.enforce_permissionscomes beforeapproval. Inper_callmode, the approval middleware is also awrap_tool_callhook, so list order is nesting order, and a forbidden call is denied before it reaches a person. Swap the two, and a reviewer would be asked to approve a viewer's refund that is then denied anyway. That wastes the reviewer's time and teaches them to click "approve" without reading.ToolErrorMiddlewareis the innermost tool wrapper, so it only sees exceptions from the tool itself, after every check above it has passed.
A production checklist
Before you give an agent tools that change real data:
- Take identity and scope from
runtime.context, never from tool arguments. - Enforce permissions where the tool runs. Hiding a tool from the model is a convenience, not a control.
- Write tool error messages for the model. Do not pass raw exceptions through.
- Limit calls for every tool that costs money, and set a model call limit to stop loops.
- Use a persistent checkpointer for human-in-the-loop, and pass the same context when you resume.
- Log every tool call with the user, the arguments and the outcome. A
wrap_tool_callhook placed first in the list also sees the calls thatenforce_permissionsdenies. - Test your guardrails like code. None of the checks above needs a real model. Every example in this article was verified with
GenericFakeChatModelfromlangchain_core, a fake model that returns scripted messages, including scripted tool calls. The tests cover a viewer calling a hidden tool, a refund above the limit, an approved, an edited and a rejected refund, a second refund in the same run and the injection filter. They run in seconds, they give the same result every time, and they cost nothing. - Trace production runs with LangSmith, so you can see which checks fire and how often reviewers reject the agent's calls.
The pattern is the same at every layer: the model decides what it wants to do, and your code decides what it is allowed to do. The model is good at the first job. Only your code should be trusted with the second.
Want to know when a new article drops?
Get an email whenever I publish something new. No spam, unsubscribe anytime.
Comments
Related articles
An Introduction to LangChain: Chains, RAG & Agents the v1 Way
Aug 24, 2026 · 9 min read
PyTorch Autograd Explained: How loss.backward() Actually Works
Oct 7, 2026 · 18 min read
Working Effectively with Claude Code — Part 2: Agents, Subagents & Multi-Agent Setups
Jul 15, 2026 · 11 min read
BPE (Byte Pair Encoding): What It Is and Why It Matters for LLMs
Jul 1, 2026 · 9 min read
Working Effectively with Claude Code — Part 1: Context, Skills & Prompting
Jul 1, 2026 · 7 min read
Building GPT From Scratch: A Step-by-Step nano-gpt Walkthrough
Jun 27, 2026 · 10 min read
About This Blog
Jun 26, 2026 · 2 min read
What Is a Tensor? Linear Algebra for Deep Learning
Jun 24, 2026 · 5 min read