GenAI Security Framework: Principles, Controls, and Best Practices

Generative AI brings unique security challenges due to prompt injection, data leakage, model subversion, hallucination, and content misuse. The Generative AI Security Framework offers best practices for designing and implementing solutions that detect and prevent such vulnerabilities during data gathering, model training, and operation. Typically, it covers key areas, including prompt validation, input/output sanitization, model integrity, confidential data protection, adversarial training, and robustness to attacks. It helps organizations adopt generative AI models, protect data, maintain regulatory compliance, and ensure safety during the development and deployment of AI-based systems.

In this blog, we explore various Gen AI security frameworks and which one to adopt.

Why “Gen AI Security Framework” Means Different Things to Different Vendors

The term “Gen AI Security Framework” is notably ambiguous, and it ultimately creates expectations management issues for end-users trying to compare solutions and build out their own security programs. In practice, the phrase appears to describe three broad categories of control frameworks that cut across different problem domains.

Scoping/maturity frameworks

These frameworks define what areas of Gen AI security programs should target and provide a sense of the organization’s progression toward stronger security outcomes. Capability maturity is typically evaluated in terms of the presence of certain “visibility” and “control” areas, such as shadow AI discovery and inventory or model input/output logging, and vendors often position their framework solutions around this concept, as it directly informs buyers’ maturity assessments.

Taxonomy frameworks

These frameworks attempt to catalog potential attack surfaces, threats, and failure modes in depth. Attack trees, OWASP’s LLM Top 10, and insecure data practices fall under this category, and the value of such frameworks lies in standardizing the language around potential risks and harms.

Regulatory/compliance frameworks

These frameworks derive from regulators’ expectations, directives, and supporting standards, such as those defined by the EU AI Act or NIST AI RMF, as well as industry-specific requirements represented by ISO/IEC 42001. While not focused on attack surfaces and their taxonomy, such frameworks provide guidance on operational and technical processes that enterprises should implement to meet compliance requirements.

Practically speaking, a vendor looking to sell a “Gen AI security framework” may well be talking about any of the three, and there is an inherent risk of misaligned expectations between buyers and sellers. Organizations must therefore understand what problem domain a given framework addresses and have realistic expectations about their security programs. Most critically, they must recognize that frameworks of all three types are typically needed to address the diverse requirements of enterprises managing large-scale Gen AI capabilities.

AWS’s Generative AI Security Scoping Matrix

AWS developed this framework to help categorize generative AI use cases based on the level of control and ownership of both the model and data. According to the matrix, the use case falls within the scope of consuming a public model, such as ChatGPT, with no control or knowledge of the data and model. It also includes retraining an existing model with a proprietary data set and fine-tuning an existing foundation model. The lowest level is developing and training a model from scratch. The higher the level, the more control and responsibility for AI security the organization has. The matrix arranges the five scopes from lowest to highest in terms of governance and compliance, legal and privacy, risk management, control, and resilience, depending on the level of ownership of the model and data.

How It Was Developed Into the Agentic AI Security Scoping Matrix

The initial version was built around the foundation-model paradigm, where the default is stateless processing of a query/response exchange. By contrast, agentic systems perform extended multi-step reasoning, make autonomous decisions, and directly interact with supporting infrastructure and data. This requires different security controls around memory, tools, identity, and orchestration that the original matrix did not include.

In November 2025, AWS introduced the agentic AI security scoping matrix as an extension to the original framework. It uses two axes to classify various security scenarios – agency (what actions can the system take) and autonomy (can the system make decisions independently). The matrix then identifies four possible scopes of operation:

1. No agency: human-in-the-loop only; no capacity to take any action

2. Prescribed agency: suggested changes but requires explicit approval for all modifications

3. Supervised agency: humans define objectives, but the system plans and executes changes independently

4. Full agency: self-directed systems that continuously observe and adapt with only high-level human oversight

For each scope, the matrix specifies security requirements around identity context, data/memory/state protection, auditing, agent and foundational model controls, agency perimeters, and orchestration needs. In general, each subsequent scope has more stringent requirements across all categories.

Why OWASP and CoSAI Reference It

AWS mentions that the initial Generative AI Scoping Matrix was “widely referenced by organizations such as OWASP, CoSAI, and other industry standards bodies, partners, systems integrators (SIs), analysts, auditors, and more.” One reason is that AWS played a direct role in the establishment of some of these organizations’ work on AI security: co-author Matt Saner is a member of the project governance board and executive steering committee for the Coalition for Secure AI (CoSAI), while also being a distinguished reviewer for OWASP’s Gen AI and Agentic AI security initiatives.

In practice, this means that the matrix helped to fill a genuine need for both coalitions: OWASP’s LLM/Gen AI Top 10 and CoSAI’s secure AI guidance concentrate on potential threats and attacks (prompt injection, data leakage, insecure design of plugins, etc.), whereas the AWS matrix offers a perspective on the degree of responsibility one has for addressing them. That combination appears to have contributed to the document’s popularity among these organizations as a reference framework rather than a competing standard.

Google's Secure AI Framework (SAIF)

Unlike AWS's matrix, which categorized systems based on ownership tiers, Google's SAIF takes the shape of a risk map: a shared taxonomy of particular threats with definitions of their introduction points, exposure, and mitigations across the AI development lifecycle, as opposed to focusing on the relative scope of each system on a ladder.

Overview and structure

Google released the Secure AI Framework (SAIF) in June 2023, organizing potential implementation measures into six principles, although not as a single assessment matrix: extending security foundations to AI, incorporating AI into broader enterprise threat detection and response, automating wherever possible, aligning across platforms, improving controls based on rapid feedback cycles, and situating AI within broader business processes. The framework's primary visual aid, the SAIF Risk Map, structures known security risks based on the AI lifecycle, breaking it down into four areas: Data, Infrastructure, Model, and Applications. Each of the named risks is then placed within these areas according to three axes: introduction points, exposures, and mitigations.

Beyond the general areas of the AI lifecycle, the 2023 version of SAIF lists particular risks, such as Data Poisoning, Model Source Tampering, Model Exfiltration, Prompt Injection, Model Evasion, and Sensitive Data Disclosure, among others. Each of these items follows the same tripartite division, specifying what stage of development or deployment introduces the risk, where it is exposed, and what mitigations are available.

Agents and updates

SAIF 2.0 was released in 2026, with an update that separated agents into a separate domain. While the first version structured potential risks based on Data, Infrastructure, Model, and Applications, the second version introduced agents as a category that included four subcategories: Application & Perception, Reasoning core, Orchestration, and Response rendering. This update added additional risks unique to agents, such as Rogue Actions, which are actions taken by an agent that go beyond what the principal expects. These actions stem from either the reasoning core or poisoned orchestration tools, memory, or retrieved data and are mitigated through input validation, adversarial training of the core, restricted tool access based on policies, and response sanitization. Another risk unique to agents is the misuse of privileged access: agents have access to the user's data, tools, and systems, which opens the possibility for agents to access information the user should not see, such as emails, files, and other data.

Transparency and availability

The way Google has positioned SAIF within the larger Coalition for Secure AI (CoSAI) has demonstrated its commitment to transparency, as both the risk map, assessments, and taxonomy of control-component-risk trios are available for the broader research community to adopt, discuss, and improve. This emphasis on transparency probably underlines the differences between the two frameworks, as AWS's approach remained proprietary, only informing the public about their general strategy and philosophy concerning secure AI systems. Yet, structuring risk around the control-component-risk trios and making them publicly accessible probably helps both SAIF and AWS matrix, as they are both positioned as part of efforts to develop a common set of standards and terminology, as illustrated by OWASP's work in the space.

The OWASP Gen AI Security Project is the Umbrella

Unlike AWS and Google, which offer a unified list of recommendations, OWASP represents the umbrella under which the other two organize their research and development. The Open Worldwide Application Security Project, better known as the OWASP Gen AI Security Project, is an open-source initiative gathering the world’s security community to identify, mitigate, and document security and safety risks associated with large language models, agentic AI, and AI applications. It emerged in 2023 as a response to the identified gaps and grew into a global community of over 600 security professionals. The project now comprises 8 working groups and 14+ sub-projects working on various aspects, ranging from data and red-teaming security to AI governance and incident response.

It is noteworthy that the two most popular reports, namely OWASP Top 10 for LLM Applications and OWASP Top 10 for Agentic Applications, have already been covered by separate posts on this blog. Therefore, this section will focus on highlighting the unique value proposition of the umbrella. In this regard, the emphasis is on the fact that OWASP aims to complement rather than compete with other standards. Even within the Top 10 lists, the LLM applications retain their place as a critical asset, and the project leads actively promote the need to harmonize various standards to facilitate the development of safer and more intelligent systems as agentic applications evolve.

It is also this common sense that enables the collaboration between the three IT security giants, with AWS Agentic AI Scoping Matrix and Google’s SAIF Framework being positioned as complementary to OWASP. In other words, the LLM Top 10 list serves as the reference point, which cross-references industry-standard taxonomies such as NIST, MITRE ATLAS, CWE, OWASP Top 10 for Agentic Applications, and others. As a result, in practice, OWASP rarely competes with other frameworks in the same space but rather supplies the detail-level risk documentation and severity prioritization that higher-level standards lack. Notably, Google has already donated its risk-matrix data to CoSAI, a standardization body that unifies many contributors, including OWASP’s community.

Comparison Table: Framework, Scope, and Best Fit

Here is a comparison table of Gen AI frameworks concerning their type, scope, and what they are best suited for:

Which Framework to Actually Adopt

Realistically, there is no either/or - most comprehensive programs will incorporate multiple frameworks, though there are a few clear priorities depending on which risk domains you're trying to address. If you have to choose one to start with, here's what we recommend based on your priorities:

If you're AWS-native

It's hard to overstate the value of the AWS Generative AI Security Scoping Matrix / Agentic AI Security Scoping Matrix as a starting point for your risk modeling. The matrix does an exceptional job of capturing the AWS reality of shared responsibility while also providing a high-level categorization of different use cases (consumed tools, fine-tuned, fully built) to scope your controls against. In practice, we find that it's most useful to apply the 5 primary risk scopes to any generative AI use case and then overlay the 4 agentic scopes (no agency through full agency) as appropriate to conversations around autonomous systems.

The matrix provides an excellent foundation for an executive-level conversation about what areas you have visibility/control over versus what is the responsibility of AWS. As such, it should be the foundation of any control framework in your AWS-native implementation of generative AI.

If you need to align with regulatory expectations (APAC focus)

If you're working in an APAC-focused environment, the Singapore Model AI Governance Framework for Generative AI is your friend. Specifically, the 9-dimension breakdown of accountability, data, trusted development and deployment, incident reporting, testing and assurance, security, content provenance, safety and alignment research, and AI for social good covers many of the disclosure controls and treatment expectations that you're likely to find in your local regulators' guidance. The MGF for Gen AI, along with its 2026 companion for agentic AI, also provides a credible foundation for the Global AI Assurance Pilot run by IMDA.

The MGF for Gen AI is less a risk framework per se, and more a way to structure your disclosure controls around AI governance topics. As such, it complements the technical risk controls you might have adopted from AWS or OWASP while leaving the implementation and operational details to "industry standards" (in the eyes of the MGF authors).

If you want an actual risk taxonomy

For a concrete list of risks you should think about addressing in your generative AI implementation, you'll want to look at Google's SAIF Risk Map and OWASP's Top 10 lists. We'd argue that the combination of the two provides the most comprehensive view of technical risks while also giving you some context about which risks to prioritize (SAIF lists them in order of impact severity). The risk map breaks down risk sources across the data, infrastructure, model, application, and now agentic (perception, reasoning, orchestration, and response) lifecycle stages with a view into what is introduced, exposed, or mitigated at each step. The OWASP equivalents for LLMs and Agentic AI provide similar lists of risks, but with much broader community endorsement.

In reality, the two resources are supplementary to each other - Google has published its risk map as part of the CoSAI initiative, and SAIF references OWASP's Top 10 lists extensively. Use SAIF if you want a taxonomy of risks introduced or exacerbated by different elements of the generative AI stack, and OWASP if you want a prioritized list of the most pressing concerns to address first.

How Akto.io Maps These Frameworks

Akto’s AI security ecosystem frameworks tackle various angles of the overarching issue - governance, threat modeling, application-specific risks, and control gaps - and suggest combining NIST AI RMF (risk backbone), MITRE ATLAS/OWASP/Cisco (realistic threat modeling), and Google SAIF (secure-by-design culture) into a single system.

It has the most operational coverage for OWASP, which released three distinct AI/ML Top 10 lists this year: for large language models, agentic applications, and managed cloud platforms (MCP). Specifically, Akto’s red-teaming feature uses over 4000 probe library to scan all three areas as a single continuous chain rather than isolated elements. It does the same for attacking skills: for the OWASP Agentic Skills Top 10, it matches each entry with exact detection logic (malicious pattern scanners, privilege analyzers, and real-time exploit blockers).

The same technology base underlies two products: Atlas (covers employee-facing AI/IDE) and Argus (for internal agents and MCPs), which unify posture management across OWASP, NIST, and MITRE ATLAS standards in a single interface.

Explore MCP and Agentic AI Security at Akto → Book a demo today!

Final Thoughts on Gen AI Security Framework

There is no one true framework for answering these questions, as AWS, Google, and OWASP were created with different goals in mind. AWS’s Scoping Matrices define responsibility - which parts of a system are owned and controlled by which actors. Google’s SAIF taxonomy classifies potential failure points and their location within a system. OWASP’s Top 10 list prioritizes items based on the urgency of action needed, informed by the collective input of its 600+ contributors, whom AWS and Google both answer to rather than compete with.

Used in combination, each framework fills gaps left by the others, forming a stack of tools that can realistically help organizations build out their own security posture. In isolation, each is incomplete-AWS offers no technical detail on failure modes, SAIF provides no prioritization, and OWASP delivers no governance or control specification. By combining AWS’s control scoping, Google’s threat modeling, and OWASP’s prioritization, organizations can build a shared security knowledge graph spanning across operations, use cases, and personnel.

This is the space where vendors like Akto aim to provide value directly to the user – by building shared lexicons and frameworks, they enable discovery, adversarial emulation, and enforceability at scale. As agentic AI matures, frameworks will likely consolidate around common standards rather than compete – an outcome beneficial to any organization seeking to develop long-term, sustainable security programs, rather than chase ever-changing third-party certifications.

FAQ’s on Gen AI Security Framework

1. What is the AWS Generative AI Security Scoping Matrix, and how has it changed?

The AWS Generative AI Security Scoping Matrix is an AWS framework that categorizes use cases of generative AI based on five stages of data/model ownership, ranging from using a public tool to building a model from scratch. Each category is then associated with five security disciplines: governance, legal/privacy, risk, controls, and resilience. In November 2025, AWS announced the companion Agentic AI Security Scoping Matrix that uses agency as its organizing principle. Instead of five stages of data/model ownership, the new matrix has four scopes ranging from no agency to full agency.

2. What is Google’s SAIF, and how is it different from AWS’s approach?

SAIF (Secure AI Framework) is Google’s framework for securing generative AI announced in 2023. Unlike AWS’s risk matrix, SAIF takes the form of a heat map that plots named threats (e.g., data poisoning, model exfiltration, prompt injection) across the different components of the data lifecycle: Data, Infrastructure, Model, and Application. SAIF 2.0, announced in 2026, adds an agent-specific heat map.

3. What are the 9 dimensions of Singapore’s Model AI Governance Framework for Generative AI?

Accountability, Data, Trusted Development and Deployment, Incident Reporting, Testing and Assurance, Security, Content Provenance, Safety and Alignment R&D, and AI for Public Good.

4. How does OWASP GenAI Security Project relate to these named frameworks?

The OWASP GenAI Security Project serves as the community governance and ecosystem for the LLM, Agentic, and MCP Top 10 lists. The project has seen the contributions of 600+ community members and has been referenced by AWS, Google, and Singapore as a foundational resource for their respective frameworks. Rather than being a competing governance framework, OWASP GenAI Security Project provides scoping and taxonomy that the other frameworks build upon.

5. Why do OWASP and CoSAI reference AWS’s Scoping Matrix specifically?

Some of the reasoning is likely due to direct contributions: one of the co-authors of this blog is on the board of directors of CoSAI and the OWASP GenAI Security Project. In addition, AWS’s approach to Scoping fills a particular need in the industry: while OWASP/CoSAI define what can go wrong, AWS defines what a particular organization is responsible for.

6. Which framework should an AWS-native organization adopt?

An organization that is native to AWS should begin its security strategy with AWS’s Scoping Matrices since they align with their existing risk management approach (e.g., Bedrock, SageMaker). From there, it can build upon the controls suggested by the OWASP Top 10 and SAIF.

7. Do these frameworks compete with NIST AI RMF and ISO 42001, or do they complement them?

The short answer is that they complement existing risk management frameworks such as NIST AI RMF and ISO 42001. AWS, Google, and OWASP/CoSAI serve to add specificity and detail to the high-level controls suggested by NIST and ISO.

8. How does Akto map its controls to these named GenAI security frameworks?

Akto’s security testing solution covers the OWASP LLM, Agentic, and MCP Top 10 lists through its 4000+ built-in red-team probes. For each of the threats, users can view which of Akto’s detection capabilities (e.g., skills scanning, tool-call authorization) can be used to address the threat. In addition, users can view the security posture of their generative AI application in relation to the OWASP, NIST AI RMF, and MITRE ATLAS frameworks.