LiteLLM Guardrails: Setup, Providers, and Akto Integration
Learn how LiteLLM guardrails work, configure native and third-party checks, and integrate Akto for prompt injection detection, PII protection, and AI gateway security.
Arpashree
LiteLLM guardrails sit between a request and the model it's headed to, inspecting traffic at defined points in the call lifecycle and blocking, masking, or logging whatever violates policy. As a multi-provider LLM gateway, LiteLLM ships with fast, deterministic checks out of the box, and supports a long list of third-party providers for everything those checks can't catch. This guide covers LiteLLM proxy guardrails end to end, from native checks to how to configure LiteLLM guardrails for a production stack, and explains why Akto, now a native, officially announced guardrail provider in LiteLLM, is becoming the default model-based layer teams chain on top.
What are LiteLLM Guardrails?
A LiteLLM guardrail is a configured check, deterministic or model-based, that runs against a request or response passing through the LiteLLM proxy. Guardrails are defined under a guardrails block in config.yaml, each with a guardrail_name and a set of litellm_params specifying which provider to use and when it should run.
Pre-Call, During-Call, and Post-Call Hooks: Where Guardrails Sit in the Request Flow
LiteLLM guardrails run at one of three points. pre_call runs before the request reaches the model, checking user input and able to block or modify it before generation starts. during_call runs in parallel with the LLM call itself, checking the same input but without blocking the model from starting, which lowers latency at the cost of not being able to stop the request before it begins. post_call runs after the model responds, checking the full output, and is the only mode with visibility into what the model actually said. Most production setups use pre_call and post_call together, covering input and output separately, since each mode sees a different slice of the conversation. Good LiteLLM security practice generally means never relying on a single hook alone, since a guardrail scoped only to input misses everything a model might generate that wasn't present in the original request.
What LiteLLM Ships With Natively
Deterministic Checks: Regex-Based PII, Secret Scanning, Banned-Word Lists
LiteLLM's built-in guardrails handle fast, pattern-based detection: regex matching for PII like phone numbers and SSNs, secret scanning for API keys and credentials accidentally included in a prompt, and banned-word lists for blocking specific terms outright. These checks run in milliseconds and require no external call, since the logic lives entirely inside the proxy. For teams asking what the best LiteLLM guardrail setup looks like out of the box, these native checks are the right starting point precisely because they are free, fast, and require no external dependency to configure.
Why Deterministic Checks Alone Leave a Gap
A regex can catch a correctly formatted SSN. It cannot catch an attacker describing the same information in natural language to avoid the pattern, and it cannot recognize a prompt injection attempt phrased in a way no one has written a rule for yet. Deterministic checks are fast and cheap precisely because they only look for what they were explicitly told to look for. Anything semantic, anything that requires understanding intent rather than matching a pattern, falls outside what regex and keyword lists can do.
Adding Model-Based Detection: Akto for LiteLLM
Why Akto Runs Natively as a Chained Guardrail in LiteLLM
Akto now runs natively inside the LiteLLM proxy as a chained guardrail, configured as a callback on an existing proxy with no application-level changes required. LiteLLM's built-in guardrails handle the cheap, fast checks. Akto handles the cases that need a model in the loop to classify intent rather than match a pattern. Because the integration runs as a callback rather than a fork or a wrapper around the proxy, upgrading LiteLLM itself does not risk breaking the guardrail chain the way a more invasive integration pattern often would.
Threats Model-Based Detection Can Help Identify Beyond Regex
Akto's model-based detection covers semantic PII that doesn't match a fixed pattern, novel prompt injection phrasing that has never been seen before, and custom policy violations that require understanding context rather than keywords, the exact category of threat deterministic checks were never built to catch.
Verified Integration: Official LiteLLM × Akto Partnership
Akto is a native guardrail provider in LiteLLM, announced via the official LiteLLM × Akto partnership post, with reference architecture published on GitHub at akto-api-security/akto-litellm.
The Two-Entry Pattern: akto-validate (pre_call) and akto-ingest (post_call)
Akto's LiteLLM integration uses two separate guardrail entries rather than one. akto-validate runs in pre_call mode, evaluating the incoming request against Akto's security policies before it reaches the model. akto-ingest runs in post_call mode, sending the completed request and response to Akto for monitoring and analysis regardless of whether the request was blocked. Keeping these as two distinct entries, rather than combining them, is what the current integration expects.
Sync Mode vs. Async Mode
Akto supports sync mode, which blocks the request until the guardrail check completes and before the LLM call proceeds, and async mode, which logs and alerts without adding latency to the request path. Sync mode trades a small amount of latency for the ability to stop a violation before it reaches the model. Async mode adds no delay but can only flag a violation after the fact, which fits teams prioritizing visibility over blocking during an initial rollout.
Quick Start: Adding Akto to Your config.yaml
For the Full Step-by-Step Setup
See How to Add Akto Guardrails to LiteLLM in 5 Minutes for the complete walkthrough, including credential setup and verification steps.
Other Third-Party Guardrail Providers LiteLLM Supports
LiteLLM's guardrail ecosystem extends well beyond any single provider, and a fair comparison is part of making a good decision here.
Bedrock Guardrails, Aporia, Lakera, Presidio, HiddenLayer, IBM, and Others
Bedrock Guardrails integrates via AWS's ApplyGuardrail API for content filtering and PII masking, a natural fit for teams already running models through Bedrock. Aporia detects PII in requests and profanity in responses. Lakera AI focuses on prompt injection and jailbreak detection, though its current integration is limited to the chat completions endpoint. Presidio, Microsoft's open-source PII engine, runs detection and redaction locally without an external API call, which matters for teams that need PII handling to stay entirely within their own infrastructure. HiddenLayer and IBM's FMS Guardrails round out the list with their own detection and content-safety capabilities, each shaped by the specific threat model its parent company originally built it to address. Each provider has a defined scope, and each provider offers different capabilities, so enterprises should evaluate coverage against their specific security requirements.
For a Full Comparison
See Akto vs. Aporia vs. Lakera vs. Bedrock Guardrails for LiteLLM for a detailed, side-by-side breakdown of coverage, latency, and setup complexity across these providers.
Why Chaining Beats Choosing Just One
A Real Reference Pattern: Bedrock Owns Toxicity/PII, Akto Owns Prompt Injection
A documented reference architecture runs Bedrock's guardrail for toxicity and certain PII categories alongside Akto for prompt injection, with Bedrock's own prompt-attack detection explicitly disabled so the two layers don't compete over the same signal. Both guardrails can still independently flag overlapping PII categories, with whichever one classifies the specific entity type reporting it first. This division of labor reflects a sensible general principle: let each provider own the category it was specifically built to detect well, rather than asking every guardrail in the chain to attempt comprehensive coverage and risk each one doing a mediocre job across categories it was never optimized for.
Layering Deterministic and Model-Based Detection for Defense-in-Depth
No single guardrail category covers every threat class well. Deterministic checks are fast and reliable for known patterns. Model-based detection catches what deterministic rules miss but costs more latency and compute per request. Chaining both means fast checks run first and catch the obvious cases cheaply, while model-based detection handles the harder, semantic cases that actually need it. A well-planned LiteLLM guardrails setup treats this layering as the default architecture rather than an afterthought, since neither category alone matches what a defense-in-depth posture actually requires once a deployment handles meaningful production traffic.
Session-Aware Guardrails
Correlating Requests Across a Conversation for Malicious-Session Detection
A single malicious request can look benign in isolation but reveal clear intent when viewed alongside the rest of a conversation. An attacker probing for a jailbreak rarely succeeds on the first message. The pattern typically unfolds across several turns, each individually unremarkable, building toward an eventual request a single-message guardrail would have blocked outright if it had appeared first. Session tracking, enabled by passing a session identifier header on requests to the LiteLLM proxy, lets a guardrail provider correlate activity across a full conversation rather than evaluating each request as an independent, context-free event, which is what actually catches this kind of gradual, multi-turn escalation.
Guardrails for Agentic and Tool-Calling Traffic
The LiteLLM Tool Permission Guardrail
LiteLLM's Tool Permission Guardrail controls which tool calls a model is allowed to invoke, using configurable allow and deny rules enforced at the proxy level rather than left to the model's own judgment about what it should or shouldn't call.
Why Agentic Traffic Needs Broader Coverage Than Chat Completions Alone
An agent's risk surface extends well past the text it generates. Tool calls, MCP server connections, and multi-step action sequences all need coverage that a guardrail built only for chat completions was never designed to provide. Notably, some providers, including Lakera's current v2 integration, explicitly do not yet support MCP or non-chat-completions endpoints, which is worth checking directly against any given provider before assuming agentic coverage is included. A team routing agentic workloads through LiteLLM should treat this as a specific line item to verify during vendor evaluation, rather than an assumption that any provider labeled as a guardrail automatically covers tool-calling traffic the same way it covers a standard chat completion.
Guardrails as Part of Full LiteLLM Gateway Security
Guardrails are one layer of a broader LiteLLM gateway security posture that also includes authentication, rate limiting, and cost control across every model and provider routed through the proxy. Guardrails handle content and behavior; the rest of the gateway's security configuration handles who can call it and how much they can spend doing so. Treating guardrails as the only security control on a LiteLLM deployment leaves the identity, traffic, and cost dimensions of gateway security entirely unaddressed, which is why a complete production setup needs all of these layers working together rather than any single one standing in for the rest.
Why Enterprises Add Akto to LiteLLM Guardrails
Teams running LiteLLM in production increasingly default to Akto as the model-based layer for a straightforward reason: the integration is officially verified rather than a community workaround, it chains onto an existing proxy configuration without requiring application-level rewrites, and it closes specifically the gap deterministic guardrails leave open, semantic PII, novel injection phrasing, and context-dependent policy violations. The low-friction chaining model means adding Akto does not mean replacing whatever deterministic checks or other providers are already in place; An additional model-based detection layer can help address gaps left by deterministic guardrails.
Get Started
See the 5-minute setup guide for the complete implementation walkthrough, or compare Akto to other LiteLLM guardrail providers to see how coverage and setup complexity stack up side by side.
Final Thoughts on LiteLLM Guardrails
LiteLLM's native guardrails handle the fast, deterministic cases well, and the platform's growing list of third-party providers covers most everything else, each with its own specific scope and limitations. Akto's native, officially verified integration adds the model-based layer most deployments are missing, chained onto an existing setup rather than replacing it, closing the specific gap between what a regex can catch and what actually requires understanding intent. The teams getting the most out of LiteLLM's guardrail ecosystem are not the ones searching for a single provider to cover everything. They are the ones deliberately layering fast, deterministic checks with model-based detection, and verifying that whatever they chain together actually extends to the agentic and tool-calling traffic increasingly running through the same proxy.
FAQs: LiteLLM Guardrails
1. What are LiteLLM guardrails?
Configured checks, deterministic or model-based, that run at defined points in the request lifecycle as traffic passes through the LiteLLM proxy, able to block, mask, or log content that violates policy.
2. What's the difference between pre_call, during_call, and post_call guardrail modes?
Pre_call runs before the model is called and can block or modify input beforehand. During_call runs in parallel with the model call without blocking it from starting. Post_call runs after the response and is the only mode that sees the model's actual output.
3. What guardrails does LiteLLM support natively, without a third-party provider?
Regex-based PII detection, secret scanning for exposed credentials, and banned-word lists, all running as deterministic, pattern-based checks with no external API call required.
4. Why do regex-based guardrails miss certain threats?
Regex and keyword matching only catch what they were explicitly written to detect. Semantic PII, novel prompt injection phrasing, and context-dependent policy violations all require understanding intent, which pattern matching cannot do.
5. What is Akto, and how does it integrate with LiteLLM?
Akto is a model-based guardrail provider that runs natively inside the LiteLLM proxy as a chained guardrail, configured as a callback with no application-level changes required.
6. Is Akto an official, verified LiteLLM guardrail provider?
Yes. The integration was announced via an official LiteLLM × Akto partnership post, with Akto's guardrail documented directly in LiteLLM's own docs and a reference architecture published on GitHub.
7. What's the difference between Akto's sync and async modes?
Sync mode blocks a request until the guardrail check completes, stopping a violation before the model is called. Async mode logs and alerts without adding latency, flagging violations after the fact rather than blocking them.
8. Can I chain Akto with Bedrock Guardrails or another provider instead of replacing it?
Yes. A documented reference pattern runs Bedrock for toxicity and certain PII categories alongside Akto for prompt injection, with each provider covering the category it handles best rather than one replacing the other.
9. How do session-aware guardrails work in LiteLLM?
Passing a session identifier header on requests lets a guardrail provider correlate activity across an entire conversation, catching malicious intent that only becomes clear across multiple requests rather than in any single one.
10. Does LiteLLM guardrail support extend to agentic and tool-calling traffic?
Partially, and it varies by provider. LiteLLM's own Tool Permission Guardrail controls which tool calls a model can invoke, but some third-party integrations, including Lakera's current v2 integration, do not yet support MCP or non-chat-completions traffic.
11. How do I add Akto as a guardrail in my LiteLLM config.yaml?
Define two separate guardrail entries, akto-validate in pre_call mode and akto-ingest in post_call mode, both pointing to your Akto Guardrail API base URL and API key, then restart the LiteLLM proxy.
12. Which LiteLLM guardrail provider should I default to for production use?
Most production deployments benefit from chaining rather than choosing one: deterministic checks or a provider like Bedrock or Presidio for fast, known-pattern detection, paired with Akto for the semantic and novel-threat coverage pattern matching cannot provide.
Experience enterprise-grade Agentic Security solution