Shadow AI Risk Assessment: How to Score and Prioritize Your Exposure

Most companies are aware of the AI tools their employees use or at least think they are. In truth, there is often a much larger ecosystem of unofficial, shadow IT tools in use that pose significantly greater risks. From personal generative AI tool accounts to AI-powered browser extensions, copilots embedded in everyday applications, and agents communicating directly with internal systems via MCP servers, these tools typically do not appear on any corporate software inventory or procurement records.

A shadow AI risk assessment offers a systematic approach to identifying and evaluating such tools. Rather than banning them outright (which may only drive their use underground), companies can use this process to uncover which AI tools are in use, assess them according to specific criteria, and take appropriate remedial actions. This article provides an overview of how to conduct a shadow AI risk assessment.

This blog will discuss how companies can detect shadow AI, identify whether it is dangerous, and effectively govern it.

What is Shadow AI Risk Assessment

A shadow AI risk assessment is a systematic process of identifying, analyzing, and prioritizing AI tools used in the organization that should not be utilized for various reasons. From an IT perspective, the practice includes browsers’ chatbots and AI coding assistants, as well as the application’s AI elements not identified by IT and inherent in the SaaS or autonomous agents on MCP servers.

The key points of assessment are different from traditional IT audits because employees constantly identify and implement new AI tools, and approved vendors also introduce such technologies without IT control. Therefore, a shadow assessment requires continuous monitoring and reanalysis rather than a one-time examination and documentation.

Nevertheless, the practice is less complex than it appears at first glance. In general, shadow AI assessment consists of three stages: identification of tools with sensitive data, prioritization by the value of this data, and the development of particular actions based on this information. One of the critical aspects of such an audit is that it does not aim to eliminate the use of AI technologies in the organization. Instead, it helps ensure that no sensitive information is processed without the knowledge of the IT department.

Why Generic IT Risk Frameworks Are Not Fit for Purpose

Generic IT risk management frameworks are not designed to find or assess shadow AI. Traditional frameworks operate under the assumption of known systems, whereas shadow AI, by definition, flies under the radar. Furthermore, shadow AI operates at the individual user level and not at the enterprise platform level. And finally, the risk profile of shadow AI tools often depends on the data they are trained on, not just their inherent design.

Why the Information Itself May Pose Greater Exposure Risk

The generative AI tool itself is rarely the source of concern; rather, it’s the information that is being fed. A free AI writing assistant is low risk when it is simply polishing a blog, but becomes much more concerning the second one pastes in their client contract or unreleased financial numbers. This is why ranking shadow AI by tools lacks nuance; the value of the information it is touching greatly informs its sensitivity, and therefore effective risk assessment should follow that information through its journey, rather than simply cataloging what assessments have been installed.

What a Shadow AI Risk Assessment Should Measure

A proper assessment would go beyond "which tools are people using" to understand the true exposure, not only how many users are interacting with unapproved AI technologies, but also the sensitivity of the data flowing through these tools, the organization's readiness to govern AI responsibly, and what risky blind spots could be lurking.

Volume and Scope of Unmanaged AI Usage

Beyond just identifying the tools themselves, an assessment should quantify how many AI tools are in use, how many employees are engaging with them, and across how many departments; a small number of users experimenting with a writing assistant is substantially different from hundreds of employees across finance, legal, and engineering routinely using unapproved tools.

Sensitivity of Data Flowing Into AI Tools (PCI, PHI, PII, IP)

An effective assessment will evaluate what types of sensitive data (PCI, PHI, PII, IP) are flowing into the identified AI tools, as different industries have varying regulatory and business-critical requirements for protecting this information.

Governance and Policy Maturity

The assessment should evaluate the organization's current readiness to govern AI technologies, including whether there are established lists of approved vendors, policies around acceptable usage, defined ownership of AI governance, and processes for reviewing new tools.

Visibility Gaps Across Layers (browser, network, identity, SaaS)

Since shadow AI can originate from 4 different sources (browser extensions, network traffic, SSO/identity, and embedded within SaaS applications), a proper assessment should identify which of these layers lack proper monitoring and controls, rather than focusing on a single detection method.

Building a Shadow AI Risk Tiering Model

It is essential to design a risk tiering model for shadow AI, based on which appropriate action can be taken. The model helps prioritize the detected tools according to actual exposure and not waste resources on less important discoveries, such as a grammar checker.

Critical Risk – Exposes Regulated Data to AI

This category includes all shadow AI tools that process regulated data (e.g., PCI, PHI, etc.), regardless of their potential business value. The action taken to address the risk should also depend on whether the application can be governed or must be prohibited and substituted with an approved alternative.

High Risk – Exposes Proprietary Business Data to AI

Shadow AI tools that process non-regulated but still confidential and valuable data, such as code, financial models, contracts, and strategic plans, fall under this category. Even though there is no direct regulatory impact, such information requires protection if there is a possibility of exfiltration and misuse.

Medium Risk – Exposes Internal, Non-Sensitive Data to AI

AI applications that process non-sensitive information are generally safe, so no action is required unless they have the potential to be used for processing more important data.

Low Risk - No Access to Sensitive Data

No action is necessary for shadow AI tools that do not provide access to any data, merely offer public facts or writing assistance. However, such tools should still be monitored to ensure they are not misused for storing or processing sensitive information.

Step-by-Step Process for Conducting a Shadow AI Risk Assessment

Here are the steps to conduct a shadow AI Risk Assessment:

Step 1: Inventory Connected Third-Party Apps and OAuth Grants

The first step in the process is to create a comprehensive list of all third-party applications that have been granted OAuth permissions to your core platforms such as Google Workspace, Microsoft 365, Slack, and others. For many organizations, this proves to be the most straightforward way to begin investigating shadow IT. Most AI tools are simply adopted by clicking a “sign in with Google” button and do not require any formal procurement process. This exercise should provide you with a useful list of tools that were in some way involved in the day-to-day operations of the company.

Step 2: Pull Proxy, DNS, and Gateway logs

The second step is to analyze proxy, DNS, and gateway logs in order to find all AI-related domains that may have been accessed on the network. This information can be compared to the list produced in Step 1 in order to find applications that are being used in conjunction with the core platform but do not have any formal integration.

Step 3: Interview Teams about Real Workflows

At this point, it is crucial to conduct a series of interviews with various teams in order to understand how exactly they perform their daily tasks. This information will greatly augment the technical findings from the previous steps, as some tools may be used on personal accounts or unmanaged workstations and thus may not appear on any technical scans.

Step 4: Classify Findings by Data Sensitivity and Tier

Once all the relevant tools have been discovered and compiled into a single list, it is time to start categorizing them according to your company’s tier system and risk assessment framework. Ideally, you should sort the tools discovered according to the type and sensitivity of data that they process.

Step 5: Score Overall Exposure and Maturity

Finally, use the information compiled in the previous step to calculate the overall exposure of your systems to shadow AI tools. This assessment should be cross-referenced with your maturity framework in order to produce a risk management score that can serve as a crucial metric for future risk assessments.

From Assessment to Action

A risk assessment that results in little more than a spreadsheet of scores is valueless; the true value comes from the next steps, which turn scores into a remediation roadmap and determine what gets turned off and what gets replaced.

Turning Scores Into A Remediation Roadmap

The best assessment actually uses the scores as a springboard to the next steps in a five-step decision tree: accept, enable, assess, restrict, eliminate. This is because, regardless of the policy, employees are going to use AI tools, and each requires an owner, a decision (enable/restrict), and a follow-up, rather than a simple risk category. Automation is critical to making this work: tying the discovery process to actual remediation steps ensures that alerts always go to the right owner and that policy violations (example: non-approved data scanning) are blocked in real-time, rather than relying on annual reviews to notice a problem. The roadmap also has to consider the liabilities around the EU AI Act; even if the company didn't develop the AI, they are responsible for its risks if they deploy or use it.

When To Restrict Versus Provide A Sanctioned Alternative

Few assessments actually address the challenge of balancing the risk of a tool with the business need for it. Simply put, there is often no alternative to a restricted tool, and employees will find ways to use it outside of corporate systems with even less visibility. In one conversation with CIOs about this very issue, two mentioned having a consumer AI tool blocked but an enterprise-class alternative available; despite this, employees used their personal accounts for the disallowed tool because it was faster and less capable than the enterprise-class alternative.

The problem is not with having an alternative, but that in many cases, it is not a competitive alternative. The solution is to pair any restrictions on AI tools with a competitive alternative that has enterprise data-sharing agreements, SSO access, and equal or greater functionality than the restricted tool. In practice, this means that the marketing team member who wants to use an open-source LLM to create copy based on unreleased product data might be directed to an approved internal LLM rather than a public chatbot, preserving their productivity while ensuring data security. Reserved for actual elimination should be tools with unacceptable data-exfiltration risks for which there is no acceptable alternative. Everything else falls somewhere on a spectrum between monitored use and sanctioned features.

How Often To Reassess Shadow AI Risk

A risk score is accurate until you have more information that changes what the score means. AI risk scores are only useful for a short time. This section covers how frequently it's appropriate to redo the assessment on a regular schedule, and which events prompt an immediate reassessment.

Why Your Reassessment Plan Is A Moving Target

The moment you complete your initial shadow AI risk assessment, its value begins to decay. New tools appear every week, and authorized SaaS vendors constantly add new AI features with no change log or IT approval. For instance, a tool that was low risk in January can become high risk by March without any changes to your policies. Most sources will advise a quarterly review of your complete inventory, accepted vendors, and general policy as a baseline frequency, due to the fluid nature of AI innovation and the speed at which businesses adopt these tools.

Beyond general time-based triggers, reassessment should also be triggered by events or discoveries relating to specific vendors. Tools that change their terms of service, add new integrations, grant additional permissions, or begin processing more sensitive categories of information should be immediately re-scored regardless of where they are in the review cycle. Similarly, high-risk tools should have their approvals canceled by default at the beginning of each quarterly review and should only be retained if the responsible owner can demonstrate continued necessity.

The problem of shadow AI grows at such a rapid pace that an annual review is insufficient to keep up. In the words of one Netwrix survey, only 21% of respondents had full insight into which of their sensitive data was funneled through AI tools, while only 11% felt prepared to deal with the situation if they did. It is crucial to treat reassessment as an ongoing process rather than a singular event or project.

How Akto Supports Shadow AI Risk Assessment

Assessing shadow AI risk is not the end. Closing the gaps in your shadow AI risk management assessment is the real challenge that most companies are currently struggling with. Akto provides security teams with the visibility and control needed to both assess and close the gaps in their shadow AI risk management posture.

Continuous Discovery to Fuel Real-Time Risk Scoring

Instead of quarterly sweeps, Akto discovers MCP servers and all associated API calls in user environments on an ongoing basis, eliminating the need for manual enumeration and reducing the time between discovery and remediation. Every discovered endpoint is continuously tested against a documented attack matrix, while agent traffic is inspected for anomalies, ensuring risk scores are always up to date as soon as something changes in the environment.

Coverage of All Agent and MCP Traffic Surfaces

Shadow AI can come from anywhere. Akto discovers all sources of generative AI and autonomous agents, including AI-powered IDEs such as Cursor and GitHub Copilot accessing repositories, browsers using GenAI to summarize internal documents, and autonomous agents calling private APIs through MCP servers with broad permissions. Akto covers the entire agentic attack surface and provides full visibility into the risks posed by all agents, tools, resources, browser usage, endpoints, and connected services, along with shadow AI that the current technology stack was not built to detect.

From Risk Assessment to Guardrails

A list of risks is not enough - Akto's agent graph discovers relationships between agents, tools, prompts, permissions, and resources to understand how risks are interconnected, since each agent risk is often part of a much larger attack chain and ecosystem risk. Every risk is prioritized based on how easily it can be exploited, based on permissions and trust boundaries, as well as the potential blast radius of an exfiltration or breach. Security teams can then remediate sanctioned tools as appropriate while also understanding why unsanctioned tools are a risk. Finally, leadership can have executive visibility into the agentic risk landscape, including sensitive data access events, while also enforcing guardrails across all tools and agents that are aligned with FedRAMP, MITRE ATLAS, and CMMC requirements.

With Akto, companies can achieve true shadow AI risk assessment that goes beyond merely identifying risks to providing concrete guardrails that close the gaps in their Shadow AI risk management assessment.

Final Thoughts on Shadow AI Risk Assessment

The usage of Shadow AI technologies within an organization can either be officially adopted or not. Banning such tools from use in the company cannot eliminate the threats that they pose, but rather ignores them. The proper risk assessment, therefore, should include the identification of all Shadow AI tools, their possible data breaches, and a corresponding risk score. Based on the findings, the organization can choose to mitigate the risks by restricting access to the resources used by these systems or eliminating them as a security risk. Finally, a company should constantly update the risk assessment since the initial value might change with varying levels of access permissions and data sensitivity.

FAQs on Shadow AI Risk Assessment

1. Is a shadow AI risk assessment focused on the risk of chat tools and assistants, or does it also include risks associated with agents, MCP servers, and other tools?

A competent risk assessment targets the entire surface: chat tools, browser extensions, AI features in authorized SaaS applications, AI IDEs, autonomous agents, and MCP servers. Only a comprehensive assessment covers the entire risk surface. An assessment focused solely on chatbots and assistants will miss the traffic between agents and APIs and shadow MCP servers, which is now one of the highest risk surfaces due to the proliferation of permissions and often overly permissive APIs.

2. How frequently should a shadow AI risk assessment be performed?

The general recommendation is every quarter due to the dynamic nature of the field. A tool or AI feature with a relatively low risk profile can appear and proliferate within the organization faster than it can be discovered and assessed. Aside from the regular cadence, always reassess whenever there is a significant change in permissions, data access, vendors, or other factors.

3. What is the difference between a shadow AI risk assessment and a shadow discovery scan?

Discovery focuses on identifying the tools, agents, and MCP servers that exist in the environment. The assessment part takes the findings and evaluates them in terms of data sensitivity, exposure, and other factors. In other words, a shadow discovery scan is the preparation for a risk assessment.

4. How can an organization take the findings of a shadow AI risk assessment and remediate the identified issues?

Each finding should have an owner, a decision (accept, restrict, replace, or eliminate), and a reassessment trigger. For restricted tools, a competent assessment will suggest not blocking but rather replacing with an alternative sanctioned option. This is particularly important for tools that have broad appeal within the organization: blocking them will lead to decreased productivity and increased usage of non-sanctioned alternatives.

5. What regulatory compliance frameworks apply to a shadow AI risk assessment?

The assessment should take into account the NIST AI Risk Management Framework, the NIST 2024 Generative AI Profile, the EU AI Act (which makes organizations liable as deployers of AI systems even if they did not knowingly adopt them), and other relevant regulations such as PCI DSS, HIPAA, and various data privacy laws depending on the data type and territory.