Digital Transformation Stories and Insights

GenAI is no longer an experiment in a single innovation lab. Network telemetry shows enterprise GenAI traffic surged nearly 900% in 2024. Organizations are interacting with dozens of unsanctioned GenAI apps, causing data loss incidents linked to AI to double. Today, they make up 14% of all SaaS Data Loss Prevention (DLP) events. Behind these numbers is a familiar pattern: when vetted AI tools are slowed by pay-per-token budgets or compliance controls that affect output quality, employees turn to personal subscriptions that offer fewer constraints. This is "shadow AI" in practice, and it has rapidly evolved into one of the most expensive drivers of data breaches. IBM’s 2025 Cost of a Data Breach report shows incidents involving shadow AI cost an average of $670,000 more than typical breaches, accounting for roughly 20% of all breaches today. This article outlines the active governance-first blueprint we deploy today, leveraging modern Security Service Edge (SSE) and Identity platforms, to enable rapid AI adoption while strictly containing data leakage and regulatory risk. What's in this article: How unvetted AI tools bypass standard security and inflate data breach costs Why legacy security, such as blocking domains, fails against shadow AI A four-pillar governance framework to map AI usage, enforce risk-based policies, and apply data guardrails without stifling innovation A 90-day execution blueprint to move to a secure, observable AI environment Shadow AI: How We Got Here “Shadow AI” describes AI tools and services used without IT or security teams’ approval. It can be browser plugins, SaaS copilots, GenAI APIs integrated directly by teams, and personal accounts of popular AI platforms. The intent is mostly benign: employees are trying to draft emails faster, summarize documents, debug code, or explore ideas. The problem is what goes into the prompt. Recent studies highlight the scale of this behaviour: Widespread Adoption: The average enterprise interacts with around 66 different GenAI applications, roughly 10% of which are rated high-risk. High-volume Policy violations: Organizations see an average of 223 monthly attempts to paste regulated data, source code, Intellectual Property (IP), and credentials into GenAI prompts. Inadequate Tooling: Only about half of organizations have DLP controls capable of preventing sensitive data from leaking via GenAI apps in real time. In other words, GenAI usage is exploding, sensitive data is flowing through prompts, and technical guardrails are lagging far behind. Training and awareness alone cannot close this gap. How Unvetted AI Creates Costly Data Breaches Traditional shadow IT created isolated islands of unapproved tools. Shadow AI goes further: it creates direct channels for high-value data to leave your organization entirely. Here are the key risk patterns driving this leakage: Prompt-based Exfiltration: Employees paste customer records, strategy decks, or proprietary code into prompts to "get better answers." Today, regulated data and IP make up the largest share of GenAI policy violations. For instance, an engineer pasting proprietary authentication logic into a public AI tool to debug a production outage inadvertently exposes critical IP to the vendor's training dataset. The Long Tail of GenAI Apps: Enterprises often see traffic flowing to dozens of unvetted GenAI tools. Many of these lack mature security, contractual protections, or data handling guarantees. Opaque Data Flows: Over 80% of organizations cannot see where their AI-related data ends up. They are running hundreds of unofficial apps with zero visibility. Expanded Attack Surface: OAuth integrations and AI browser extensions frequently request broad access to mailboxes, files, and code repositories. Left unchecked, they become easy pivot points for attackers. When a breach involves shadow AI, remediation is costlier and slower. Security teams are left completely blind, unsure of which tools were used, what specific data was leaked, or how long the exposure has been occurring. Why Legacy Governance Models Fail Against GenAI Most enterprises today respond to GenAI with one of three approaches: Block Everything: Outbound traffic to popular GenAI domains is blocked. This may reduce leakage in the short term, but employees often route around it via personal devices, mobile hotspots, or lesser-known tools. Awareness-first: Organizations run training, issue AI usage guidelines, and rely on user judgment. Without technical enforcement, this tends to drift back toward convenience under pressure. Tool-centric: Security teams deploy a new “AI security” platform without aligning it to existing data governance, identity, and risk processes. The underlying issue is architectural: AI is treated as a new silo instead of a channel that cuts across data, identity, cloud, and SaaS. Leading guidance from regulators and industry bodies—such as NIST’s AI Risk Management Framework (RMF) and emerging regional AI acts—stresses that AI risk must be integrated into existing governance and assurance structures, not managed in a vacuum. The Four Pillars of Governance-First GenAI Security A governance-first model for GenAI adoption can be framed around four core principles. 1. Inventory Before Innovation You cannot govern what you cannot see. Discover AI Usage: Utilize AI-powered Security Service Edge (SSE) platforms, combining Cloud Access Security Broker (CASB), endpoint DLP, and proxy capabilities, to maintain full visibility into GenAI traffic across both on-premises and remote environments. Map Data Flows: Understand which repositories (CRM, ERP, source code, file shares) are feeding prompts directly or via connectors. Catalog Tools: Establish a living register of approved AI services and explicitly track unsanctioned tools with associated risk levels. 2. Policy and Risk-Based Access Once an inventory exists, governance can move from generic warnings to concrete rules. Define AI Usage Policy: Codify what data classes (for example, payment data, health data) may never be sent to external GenAI, and under what conditions internal AI platforms may consume sensitive data. Risk-based Access Tiers: Establish rules for tools. For example, Tier 1 high-risk tools are blocked; Tier 2 moderate-risk tools allow read-only, anonymized, or synthetic data, and full usage is reserved for managed devices and within specific roles. Align with Regulations: Ensure policy explicitly references applicable frameworks and laws (NIST AI RMF, sectoral regulations, EU AI Act, etc.). 3. Guardrails in the Data Path Operationalize policy via guardrails where data flows. AI-aware DLP and CASB: Extend DLP policies to GenAI endpoints, inspecting prompts and file uploads in real time to block or redact sensitive data before it leaves the organization. Industry data shows that only around half of organizations have this in place today, despite rapidly increasing policy violations. Secure AI Gateways and Proxies: Route GenAI calls, both user and application integration traffic, through gateways that handle authentication, encryption, and logging. For example, in our architecture, integrating identity providers with SSE gateways ensures only authenticated users on compliant devices reach sanctioned GenAI tools. Pattern Libraries for Safe Prompts: Offer approved prompt templates, context windows, and integration patterns (for example, a retrieval-augmented pattern that keeps sensitive data in a private vector store). 4. Continuous Assurance and Monitoring GenAI systems evolve too quickly for a one-time risk assessment. Establish a cross-functional GenAI council to track metrics like blocked uploads and newly discovered apps. Apply risk assessment gates at each stage of the GenAI solution lifecycle, from pilot to production. Aligning AI Accountability: Three Lines of Defense Governance-first security is as much about operating model as it is about technology. A practical structure often mirrors the “three lines of defense” concept adapted for AI: Line of DefenseOwnerKey ResponsibilitiesFirst LineBusiness & Product TeamsIdentify use cases, classify data sent to AI systems, and apply approved templates for safe usage.Second LineCentral AI Governance & SecurityDefine policy/risk appetite, approve or deny GenAI tools, and operate AI gateways, DLP, and CASB controls.Third LineAudit & AssurancePeriodically review control effectiveness, policy adherence, and validate that reported usage aligns with reality. Clear Responsible, Accountable, Consulted, and Informed (RACI) Matrix definitions around GenAI use cases, from idea intake to decommissioning, are critical. Without this, shadow AI fills the vacuum as teams move faster than central functions can respond. A 90-Day Blueprint for Secure AI Adoption Enterprises often ask a simple question: Where do we start? Based on our own phased rollout and internal learnings, the following is a pragmatic 90-day plan to establish a minimum viable governance-first model. Days 1-30: Discovery Run discovery for GenAI traffic across proxies, firewalls, and CASB tools. Identify top GenAI applications in use (sanctioned and unsanctioned). Map which business units and roles are driving the highest usage. Draft an interim AI usage guideline to reduce risky behaviors while governance is being formalized. Days 31-60: Codify Policy and Implement Controls Formally sanction a dedicated GenAI governance working group. Define prohibited and restricted data categories for GenAI usage. Implement or extend DLP and CASB policies specifically for GenAI endpoints. Deploy an AI gateway or secure proxy for at least one sanctioned GenAI platform and route pilot traffic through it. Days 61-90: Operationalize and Scale Document approved GenAI usage patterns and publish them as reference architectures. Extend AI-aware DLP to additional high-risk channels (developer tools, collaboration platforms). Integrate GenAI risk checks into existing change management and solution review processes. Establish monthly metric reporting to the governance forum and quarterly summaries to executive leadership. This is not the final state, but it is sufficient to move to a controlled, observable environment where innovation and security can coexist. Characteristics of a Mature GenAI Governance Program A governance-first GenAI security model should be judged less by the number of blocked prompts and more by its ability to support safe, scalable adoption. Success indicators include: Reduced Surprise: Shadow AI tools and traffic become rare; new tools are surfaced via discovery and quickly triaged through governance. Fewer Risky Prompts: Policy violations tied to GenAI start to plateau and then decline as guardrails and training work together. Improved Breach Economics: Fewer incidents involve GenAI; when they do, logs, policies, and architectures allow faster containment. Regulatory Readiness: The organization can easily prove to auditors that its GenAI usage aligns with global AI risk management frameworks (such as the NIST AI RMF or ISO/IEC 42001) and emerging regulations (like the EU AI Act). Business Momentum: More business units confidently bring GenAI use cases to production through standardized, approved paths rather than going rogue. Ultimately, the aim is not to eliminate shadow AI by decree, but to make the sanctioned path to GenAI clearly safer, easier, and more valuable than the alternatives. Closing Thoughts Shadow AI is the predictable byproduct of powerful tools meeting unmet needs. The response cannot simply be more blocking or another point solution. Enterprises that succeed with GenAI will treat it as a first-class citizen in their governance and security architecture: visible, policy-driven, and controlled at the points where data moves. By starting with inventory, codifying risk-based policies, inserting guardrails across the data path, tracking the provenance of AI-generated assets, and anchoring everything in a clear operating model, organizations can turn GenAI from a leakage multiplier into a governed capability.

As agentic AI takes on more decision-making and execution within enterprise applications, Quality Engineering (QE) teams face a different testing problem. Unlike traditional software, these systems are non-deterministic: the same request may follow different execution paths or generate different responses while still reaching a valid outcome. A generated response may appear correct even when the underlying agent selects the wrong API, omits critical parameters, or hallucinates information. Relying solely on the final output can create a false sense of confidence in system quality. Reliable testing of agentic systems requires QE teams to evaluate more than the final response. They need to understand how the agent reached that response, whether it made the right operational decisions, and whether those decisions were appropriate for the task. Our AI Quality Engineering Framework addresses this challenge by making intermediate agent behavior observable, measurable, and testable throughout the development lifecycle. What’s in this article: Why does agentic AI require a different approach to quality engineering? How can QE teams test the behavior of AI agents? How our AI Quality Engineering Framework evaluates execution, agent decisions, and response quality How structured evaluation helps detect regressions and support release decisions as models, prompts, and tools change Agentic AI Requires a Different QE Approach Traditional software testing typically validates whether a predefined input produces an expected outcome. With agentic AI, QE must extend that validation to the decisions and actions that lead to the outcome. This was a key consideration in our approach to testing an enterprise-grade agentic AI application for a leasing platform. The application coordinated multiple AI agents and invoked backend tools to retrieve property data dynamically. In addition to validating the application’s functional behavior, we designed our QE approach to evaluate how the agents handled each interaction. This meant testing questions that conventional functional checks do not typically address: Was the request routed to the correct specialized agent? Were the appropriate backend tools selected? Were tool parameters interpreted accurately? Was the final response grounded in retrieved data, or did the model hallucinate a plausible-sounding answer? From Observability to Behavior Evaluation Modern observability platforms such as LangSmith provide valuable visibility into execution traces, agent interactions, and tool calls. But observability alone is not enough. A trace can confirm that a tool was invoked; it cannot determine whether that tool was appropriate for the user’s intent or whether the resulting decision was correct. We, therefore, built our AI Quality Engineering Framework around a simple principle: AI behavior must be observable before it can be evaluated, and it must be evaluated before it can be trusted. AI behavior must be observable before it can be evaluated, and it must be evaluated before it can be trusted. The Three-Layer AI Evaluation Framework To move from simply observing AI behavior to actually validating it, we implemented a three-layer evaluation flow in our QE framework. Every user interaction is assessed at the execution, decision, and response-quality levels before it is considered release-ready. 1. Execution Tracing We implemented execution tracing through LangSmith instrumentation in our QE framework, so every conversation turn is captured as a structured runtime trace. For each user request, the framework records agent routing, tool calls, tool arguments, tool outputs, intermediate reasoning checkpoints, and final response metadata in sequence. Inside the framework, this works as a trace-first pipeline: test execution emits interaction events, the events are normalized into a single schema, and the run is persisted as a chronological conversation trace in LangSmith. Because traces are standardized across runs, QE can reliably replay paths, compare behaviors build-to-build, and identify exactly where the agent workflow diverged. While this layer establishes what happened during an interaction, the next layer determines whether those actions were correct. 2. Decision Validation A decision-validation layer on top of trace capture checks whether the system made the right operational choices for user intent. We configured the framework with two types of checks: Standard quality evaluators to monitor hallucination, toxicity, bias, and fairness checks Custom task evaluators tailored to business workflows For the leasing agent, we introduced a custom tool-correctness evaluator to verify that the selected backend tool, input parameters, and invocation order were appropriate for property-related questions. We added this because many failures were not hard crashes; they were decision-quality errors: the system responded fluently but used the wrong tool path or incomplete parameters, which risks inaccurate leasing information. Once we could validate the agent’s decisions, we also needed to assess the quality of the response those decisions produced. 3. LLM-as-a-Judge For response-quality assessment, we operationalized an LLM-as-a-Judge pipeline in the framework. The framework assembles inputs from three sources: Conversation context from trace history Evidence from tool outputs The agent’s final generated response The judge model scores every turn against a fixed evaluation rubric covering factual accuracy, contextual relevance, data grounding, hallucination risk, and overall usefulness, allowing quality to be tracked reliably across runs and releases. The framework aggregates these results at turn-level and conversation-level to produce comparable quality metrics over time. This makes conversational quality regression-testable: model, prompt, or toolchain changes can be measured for quality drift before release. The QE Impact: From Testing to Engineering Trust The three layers work together to give QE a more complete view of an agent interaction. For leasing conversations involving pricing, availability, and property details, failures could now be isolated to the exact layer: Incorrect agent routing Wrong tool choice Incomplete or incorrect tool parameters A mismatch between retrieved data and the generated response Low-quality final responses despite technically successful execution Essentially, our QE approach shifted from: Pass/fail conversation checks to decision-level validation Manual transcript inspection to repeatable evaluator-driven scoring “Response looks reasonable” to evidence-backed grounding and correctness checks Post-release discovery of subtle AI drift to pre-release detection in regression cycles Making AI Quality Actionable Across Teams The framework also made AI quality signals useful to different stakeholders: Quality engineers gain deterministic diagnostics and reproducible failure evidence from traces. Application engineers can identify where an agent workflow failed and which decision needs attention. Product leaders gain measurable quality signals that can inform release decisions. Clients are confident that leasing responses are correct and grounded, not just fluent. This is particularly important for enterprise AI systems where a plausible response is not sufficient evidence of correctness. Teams need a way to understand and measure the behavior behind that response. Governing AI Quality in Production Agentic AI has expanded what QE needs to validate. Functional testing remains essential, but it is only one part of assuring an AI system that can make decisions, invoke tools, and dynamically determine how to complete a task. Our AI Quality Engineering Framework addresses this by treating agent behavior as a testable engineering concern. It captures execution traces, evaluates operational decisions, and measures response quality against defined criteria. This makes AI quality more measurable across development and release cycles. Model changes, prompt changes, and toolchain changes can be evaluated against the same behavioral signals. Regressions can be identified earlier, and release decisions can be supported by evidence. For agentic systems, that is the shift QE needs to make: from asking whether the system produced an acceptable answer to evaluating whether the system behaved appropriately to produce it. Explore our Quality Engineering Services to learn how we can help you build more reliable agentic applications.