AI-Aware Gateway & Guardrail Control: Why Neither Works Alone

Modern enterprises seeking to leverage generative AI at scale must grapple with the complex reality of managing tens or hundreds of different models, providers, and applications. By implementing an AI-aware gateway, enterprises can ensure appropriate usage of Large Language Models (LLMs) by directing requests to multiple LLM providers and applying policy controls at the time of traffic routing. Meanwhile, guardrails offer additional security and control by analyzing prompts and responses at the application layer, preventing unauthorized data exposure and other potentially unsafe outcomes. The combination of gateways and guardrails provides enterprises with visibility and control over all AI activity at the edge. This blog explores the combination of Gateways with Guardrails and how they work together to efficiently secure LLMs.

What "AI-Aware" Actually Means for a Gateway

Taking security concerns away for a second, an AI gateway is still a useful concept, as it provides the infrastructure necessary to perform operations with multiple models and providers in a coherent manner.

1. Routing - directing queries to specific models based on cost, latency, type of work, or other variables rather than tying applications to a single provider

2. Fallbacks - automatic switching to another provider if the current one becomes unavailable or rate-limited

3. Logging - auditing of all operations in a machine-readable format, usually including information about what model was used, who used it, how much it cost, and how long it took

4. Provider abstraction - acting as a single point of entry for multiple models with significantly different APIs

5. Model-aware operations - operating at a lower level than a regular API gateway would, handling tokens, context lengths, rate limits, and versioning specific to a model.

All of these features are useful, but none of them have anything to do with keeping third-party models secure. A gateway can route requests correctly, fail over immediately if something goes wrong, and provide extensive logging of all operations without lifting a finger to actually secure anything. It can route malicious requests to databases holding sensitive information and let SQL injection attacks run rampant; it can forward unvalidated user input to models and let them return private or restricted data without any sanitization. A gateway is an operational tool, and just like any other software, it can be insecure in its own way, but its primary responsibility is to make applications easier to develop, not to protect them.

The Blind Spot a Gateway Has Without Guardrails

A gateway makes decisions based on destination, load, and availability, not what’s in the request, which is a problem because the things it can’t see are precisely what prompt injection, jailbreaks, and data exfiltration all have in common: instructions.

Prompt injection

LLMs can’t differentiate between instructions and data since they both come in natural language within the same context window. A router has no reason to look any deeper than the top layer of meaning in any request - and the problem isn’t hypothetical or theoretical: it’s why OWASP ranks it as one of the top ten LLM application risks and why a single injection attempt against Claude Opus 4.6 (GUI agent) managed to achieve meaningful success on the first try in Anthropic’s own system card, 78.6% success rate by the 200th.

Jailbreaks

There’s a reason OWASP lists jailbreaks as a separate category from prompt injection. External actors use jailbreaks to “bypass content filters” while prompt injections focus on “instructing the model to perform a specific action.” Either way, it still starts with a request containing disguised instructions, which is no different from asking ChatGPT to “act as a search engine” - except routing rules can’t inspect the internal context of a request for hidden instructions.

Data exfiltration

An attacker could embed exfiltration instructions in a website, document, or email that the agent can see, which would appear as a harmless request to the gateway. The ability to bypass all content filters, safety checks, and security rules makes injecting a third-party model particularly dangerous, especially when used as an agent.

And then there’s Echo Leak, a zero-click vulnerability in Microsoft 365 Copilot (CVSS 9.3) that was still actively exploited as of June 2025. It worked by simply instructing the agent to perform a task - no additional requests, traffic, or patterns to detect. The same pattern could be used to compromise GitHub Copilot (CVSS 9.6) and Cursor IDE (CVSS 9.8), both of which also allow external access to private code repositories. A similar set of rules would be sufficient to turn any agent-based service into an information disclosure channel, which is why a gateway could potentially satisfy all these conditions and have no idea it’s happening.

What an AI-Aware Gateway and Guardrails Pairing Looks Like In Practice

AI Agent Gateway is situated in the model access path, providing guards both on model input and output, with the ability to enforce policies in both directions, not just one.

Request guardrails (pre-model)

Before the prompt or tool call gets processed by the model, it is evaluated by the AI Agent Gateway against the policies set in Akto, notably the prompt injection, SQL injection, command injection detection and prevention rules, and other organization-specific security policies. Where the system under review uses an agentic paradigm, Akto security policies are not limited to just one prompt - they analyze the model input in the context of the whole conversation plus any tools used, MCP requests, and other means of model access. The reason for this is that in an agentic system, the model has agency and can make decisions on its own, and therefore the attack surface is not limited to just the prompt but includes all available tools. In effect, this means that each request is evaluated against all applicable security policies, preventing potentially dangerous prompts and tool calls from reaching the model and thereby wasting inference capacity.

Response guardrails and redaction (post-model)

Having processed the request, the model's response is then evaluated by the AI Agent Gateway against similar security policies, with the ability to block, redact, or otherwise modify the response before relaying it to the end user. In effect, this means that each response from the model is evaluated for potential security issues and sensitive data content, with three possible outcomes: passing through as-is, being partially redacted, or being blocked completely. The end user therefore sees either the response as written, a notice that it could not be delivered, or a redacted response, depending on the severity of the issues found and the policies in place. Unlike solutions that have the ability to only deny access, however, Akto's ability to redact sensitive data from responses means that potentially useful information can be extracted while still removing any information that the end user should not see, eliminating the need to contact support for every questionable response.

One policy, all apps behind the gateway protected

By virtue of being placed in front of the model, the AI Agent Gateway provides consistent security features to all applications that make use of it, without having to implement additional policy logic in each application. In practice, this means that the policies only have to be configured once on the gateway, and then they are automatically enforced for all applications using it, with no additional effort. This has implications for any system using agentic AI, where even the smallest oversight in security can lead to model abuse and data exposure. By implementing security policies once, in the gateway, Akto provides a comprehensive solution without requiring additional measures per integration. As one partner described it, this is similar to having a secure foundation that does not require additional reinforcement with every new application or service added.

Where This Doesn't Reach

The gateway described here only observes the traffic that flows through it. Three surfaces are not covered, which are typical for developers using AI:

• Employee endpoints: a developer's machine where the AI security tools are used outside of any application that the gateway is in front of.

• Local IDEs: the same AI coding assistants are often used directly in the IDEs, where no application is started.

• Browser-based usage: developers may use a web-based AI assistant directly in the browser, which means that no application is involved in the request/response chain at all

These surfaces are not endpoints that would traditionally be guarded by a gateway - that is, the requests and responses do not pass through a central point where the traditional gateway would be deployed. Atlas provides protections for such cases as well, with the details discussed below.

The same components that perform discovery are used to enforce guardrails: browser extensions for browser-based AI usage, AI Endpoint Shield for local MCP traffic management, and IDE hooks for IDE-based AI assistants. These are the same three surfaces as are managed by the gateway, with the difference that enforcement is done locally for browser- and IDE-based usage, and that AI Endpoint Shield is used for local MCP traffic (instead of the gateway).

In terms of implementation, these components are much closer to the user, and policies are pushed down from Akto's central console and evaluated locally, with no central component involved. Another difference is that there is no need to observe network traffic for enforcement: Akto Atlas discovers MCP servers by observing files and processes, and observes local traffic with user-space proxies, without the need for root. That is why these components are discussed separately: they run on the developer's machines, often in unmanaged environments, and need to be able to operate even when such machines are not connected to the Internet, or are running other tools that the gateway would not see.

See the Endpoints guardrails documentation (Atlas overview and Atlas guardrails) for details on how to deploy the components.

How Akto Implements Gateway and AI Guardrails

Unlike standalone proxy deployments, Akto can also be installed as a guardrail layer into a customer’s existing gateway of choice - the Portkey partnership is one such example of this model.

Setup

• Akto is deployed as a native guardrail layer within Portkey, and policies can be created and edited in minutes via the Guardrails module.

• Policies are created once in the gateway and are then automatically available across all applications using it, without requiring changes to individual services.

• Once installed, Portkey will route all LLM calls to Akto, and teams can decide whether to run guards on input, output, or both.

What gets inspected

• Prompt injection and jailbreak attacks on input are caught and mitigated.

• Malicious or policy-violating outputs are flagged.

• Both input and output can be scanned for PII, credentials, or other context-sensitive information to redact.

• For agent workflows, Akto can scan tool calls, MCP requests, or multi-turn execution context.

Outcomes

For each prompt/response, teams can decide whether to

• Allow the request or reply to proceed.

• Reject it and show a fallback message.

• Redact specific fields from either the input or the output.

This provides visibility and control over both non-critical issues (logging and optional redaction) and critical attacks (hard blocks).

Why it matters

Teams using this solution get the combined benefit of Akto’s powerful security posture and Portkey’s observability dashboard, without needing to change their infrastructure. As a result, engineers can continue to monitor and troubleshoot their applications in the same way as before, while security teams get an additional layer of defense. Meanwhile, because policy logic is placed in the shared gateway rather than in each individual service, deploying guardrails is seamless - the same general rules automatically apply to all services, agents, and models, including customer-facing ones, with no extra proxies or per-application code modifications.

Want to see what Akto can do across your full AI stack? Request an agentic security demo today.

Final Thoughts on AI-Aware Gateway & Guardrail Control

These two guardrails aren’t alternatives, they protect different traffic. The gateway protects what’s built using the applications, agents, and MCP workflows that run through it. The endpoint layer protects what’s used - browsers, IDEs, employee laptops that the gateway never touches. Most organizations will want to run both, and define policies once (rather than in every tool or across every team) and let those policies be applied consistently throughout the infrastructure

Guardrails only enforce what they’re built to enforce - they’re an implementation for consistently applying policy, not a replacement for defining what that policy should be.

FAQs on AI-Aware Gateway & Guardrail Control

1. What does “AI-aware” mean in the context of a gateway?

In short, an AI aware gateway is one that actually looks at the contents of what’s passing through and takes actions in response – as opposed to a generic gateway which usually only handles routing, load balancing, and logging of arbitrary payload. An AI aware gateway offers inline AI-specific protections such as prompt injection or data leakage guardrails, in contrast to having these built into each individual application. So the gateway makes decisions based on the actual content of the prompt, not merely acting as a conduit from caller to model to response.

2. Why isn’t an AI gateway secure by default without guardrails?

Routing, caching, or even observability of the traffic is great, but these are features addressing reliability needs rather than security ones. A gateway on its own would not have visibility into the actions an agent is taking while handling a full conversation lifecycle – it would not be able to detect shadow AI beyond what teams had already deployed, nor would it be able to red-team an agent before any external adversary could. With guardrails attached, the gateway presents a uniform surface to all customers in terms of security capabilities – without guardrails, the gateway is just a highway with no checkpoints.

3. What’s the difference between request guardrails and response guardrails?

Request guardrails are about keeping unwanted content out; they’re about protecting your models and data from being attacked or improperly used. They’re deployed on the request path, between the caller and the model. Response guardrails, on the other hand, inspect responses to detect and remediate unwanted content such as data leakage or security violations before they reach the end user. They operate on the response path, between the model and the caller. Both types can take blocking or non-blocking (redaction) actions, but only response guardrails can choose between the two.

4. What’s the impact on security policy if guardrails are enforced per application rather than at the gateway?

It increases risk surface significantly – the same policy will need to be re-written and re-deployed across all applications that make use of AI, with potential inconsistencies and omissions across different teams. By designating a gateway as the sole entry point to all applications, the policy only needs to be written once, and all applications that use it get visibility and protections by default, with no per-application changes needed beyond adding the gateway. Per-application enforcement of guardrails is similarly error prone when it comes to coverage – a service that forgot to add the guardrails has none of the protections, with no central point to discover the misconfiguration.

5. Does gateway-level enforcement also cover employee endpoints and local IDEs?

Not directly – a gateway only sees traffic that passes through it, meaning that a developer laptop does not see any gateway by default. The same span of coverage applies to browser extensions, local IDE plugins, and other local proxies – those only receive traffic that the local machine routes through them. Coverage for these endpoints and their associated guardrails is handled by an endpoint-installed extension, with browser extensions handling browser traffic, an IDE extension handling in-IDE traffic, and an AI Endpoint Shield installed on the developer machine intercepting AI traffic for processing. See the endpoint guardrails documentation for more information.

6. How does Akto integrate with gateways such as Portkey, TrueFoundry, and Kong?

They are integrated by installing Akto’s guardrail policies as part of the gateway’s native rule set, rather than deploying Akto as a separate proxied service:

• For Portkey, Akto’s policies are installed as a built-in guardrail option – they may be optionally selected and configured to run on either the request, response, or both paths

• TrueFoundry makes use of a similar mechanism, with the four available hooks (beforeRequestHook, afterRequestHook, mcpPreTool, mcpPostTool) declared in a YAML-formatted set of options when defining a new guardrail rule. This covers both general request/response inspection as well as LLM- and MCP-specific tools

• Kong utilizes a separate custom plugin to Kong which proxies API requests to either AI agents or MCP servers, with Akto’s native guardrails service separately deployed

7. What is response redaction, and how is it different from response blocking?

Blocked responses are binary – either they pass, or they are rejected, with no middle ground. With redaction, the sensitive parts of the response are stripped away, with the remainder passed to the caller. In practice, this can result in three possible outcomes depending on context – allowing the response, completely blocking it (with an alternate default response), or redacting specific data fields before passing the response to the next system in the chain. This is beneficial for data leakage scenarios in particular – a response containing one leaked credential does not need to be entirely blocked, only the credential needs to be stripped away.