[TTD] LiteLLM AI Gateway Command Injection and Provider-Key Exposure

Report Type: Threat-to-Detection Report
Threat Category: AI Gateway Command Injection and Provider-Key Exposure
Assessment Date: June 17, 2026
Primary Impact Domain: AI Infrastructure Integrity
Secondary Impact Domains: Provider-Key Confidentiality; Prompt and Response Privacy; Cloud and Kubernetes Secret Exposure; Model-Provider Abuse; AI Service Availability; Gateway Configuration Trust
Affected Asset Class: LiteLLM AI Gateway, Model Proxy, MCP Preview or Test Endpoint, Provider API Key Store, Cloud or Kubernetes Workload Identity, AI Application Routing Layer
Threat Objective Classification: Command Execution, Credential Access, Provider-Key Exposure, Gateway Configuration Abuse, AI Service Misuse, and Downstream Cloud or Kubernetes Expansion

Published by: CyberDax LLC
Author: Edward “Tony” Dolley
Role: Founder / Principal Threat Researcher, CyberDax LLC
Publication Date: June 17, 2026
Publication Type: Cybersecurity Research Report / White Paper

BLUF

‍  LiteLLM AI Gateway deployments running vulnerable versions are exposed to an actively exploited command-injection path through Model Context Protocol preview or test functionality that can allow commands to execute from the LiteLLM proxy host or container. The executive concern is not only whether LiteLLM has been upgraded, but whether attackers already used gateway access, low-privilege proxy keys, exposed service keys, or an authentication-bypass chain to execute host commands, inspect provider keys, access environment variables, read mounted secrets, alter gateway configuration, abuse model-provider accounts, or expose sensitive AI request and response data before remediation.

Executive Risk Translation

This threat converts a centralized AI gateway into a command-execution and credential-exposure point. For organizations that use LiteLLM to route requests to OpenAI-compatible providers, Anthropic, Google, Azure OpenAI, AWS Bedrock, internal models, agents, RAG workflows, developer tools, or customer-facing AI applications, compromise can create provider-key abuse, unauthorized model usage, sensitive prompt or response exposure, loss of AI access-control trust, cloud or Kubernetes secret exposure, AI-service billing fraud, and emergency restoration burden. Leadership should treat this as an AI infrastructure integrity and credential-validation issue, not as a routine package upgrade.

S5 Executive Risk Summary

Business Risk

LiteLLM command injection can undermine AI gateway integrity, provider-key confidentiality, model-access governance, AI service availability, request-routing trust, prompt and response privacy, cloud workload identity trust, and incident-response confidence. Business impact increases when LiteLLM acts as a shared gateway for multiple teams, agents, internal copilots, customer-facing AI features, developer automation, security workflows, RAG pipelines, or production applications that depend on centralized AI routing and provider credentials.

Technical Cause

The issue involves command injection through LiteLLM MCP preview or test endpoints that accept server configuration data capable of driving subprocess behavior from the proxy runtime. Public reporting identifies vulnerable LiteLLM versions from 1.74.2 through 1.83.6 and remediation in 1.83.7 or later. The durable detection issue is the sequence from suspicious MCP test endpoint activity into LiteLLM process execution, secret or environment access, provider-key exposure, gateway configuration changes, unusual model-provider usage, rare egress, or downstream cloud, Kubernetes, CI/CD, or application impact.

Threat Posture

This is strategically significant for organizations operating internet-facing or broadly reachable LiteLLM gateways because CISA added CVE-2026-42271 to the Known Exploited Vulnerabilities catalog based on active exploitation. Public technical reporting also describes a chain involving CVE-2026-48710 that can remove normal authentication barriers in affected deployment conditions. The threat should be treated as AI gateway command execution and provider-key exposure, not as a narrow version-compliance issue or a single endpoint signature.

Executive Decision Requirement

Leadership should require immediate upgrade to LiteLLM 1.83.7 or later, validation of Starlette dependency exposure where relevant, restriction or disabling of affected MCP preview or test endpoints where upgrade is delayed, preservation of LiteLLM, reverse-proxy, endpoint, container, Kubernetes, cloud, and provider logs before rotation, review of proxy keys and gateway administrators, rotation of provider API keys and reachable secrets where compromise is suspected, validation of model-provider usage, review of prompt and response stores where retained, and deployment of the S25 detection logic across web, endpoint, container, proxy, DNS, cloud, Kubernetes, CI/CD, model-provider, and AI gateway telemetry where available.

S6 Executive Cost Summary

LiteLLM command injection creates financial exposure because the organization must determine whether vulnerable AI gateway infrastructure was only exposed, actively targeted, or already used to execute host commands and access gateway-held credentials. The cost profile is higher than routine patching when LiteLLM centralizes provider keys, serves production AI applications, stores prompts or responses, manages cross-team access, routes customer-facing AI traffic, or runs in cloud, Kubernetes, CI/CD, or developer infrastructure with access to broader secrets.

Response cost is driven by emergency upgrade validation, endpoint restriction, log preservation, LiteLLM key and user review, provider-key rotation, environment and mounted-secret inspection, container or host forensics, cloud and Kubernetes identity validation, model-provider audit review, prompt and response exposure assessment, AI application regression testing, and provider billing or quota abuse review.

Low Impact Scenario

Estimated $20K - $100K.

This scenario applies where vulnerable LiteLLM exposure or suspicious MCP test endpoint requests are identified quickly, exploit attempts are blocked or unsuccessful, and log review confirms no unexpected subprocess execution, provider-key access, secret-file access, gateway configuration change, model-provider abuse, rare egress, cloud identity use, prompt or response exposure, or customer impact.

Moderate Impact Scenario

Estimated $150K - $850K.

This scenario applies where suspicious MCP endpoint activity, unexpected LiteLLM subprocess execution, uncertain provider-key access, abnormal model-provider usage, unusual egress, or incomplete telemetry prevents the organization from quickly ruling out gateway compromise, but customer data, regulated AI content, production cloud credentials, CI/CD secrets, and downstream applications remain intact after validation. Response may require provider-key rotation, LiteLLM proxy-key invalidation, host or container forensics, cloud and Kubernetes review, model-provider audit review, AI application regression testing, and temporary restriction of AI gateway operation.

High Impact Scenario

Estimated $900K - $6.5M+.

This scenario applies where attackers executed commands from the LiteLLM runtime, accessed provider API keys, exposed prompt or response data, modified gateway configuration, created or reused proxy keys, abused model-provider accounts, accessed cloud or Kubernetes secrets, pivoted into adjacent workloads, affected customer-facing AI services, triggered legal review, required customer or regulator notification analysis, generated provider billing abuse, or forced emergency rebuild and restoration of AI gateway trust.

S6A Key Cost Drivers

·        Number and importance of LiteLLM gateway deployments.

·        Whether LiteLLM serves production applications, customer-facing AI workflows, internal copilots, RAG pipelines, agents, developer automation, security workflows, or regulated-data processing.

·        Number of model-provider API keys, cloud credentials, database credentials, Kubernetes secrets, CI/CD secrets, and service-account tokens reachable from the LiteLLM runtime.

·        Whether prompt, response, request metadata, user identifiers, application context, uploaded documents, source code, customer records, or security data are logged or stored by the gateway.

·        Evidence of MCP test endpoint access, subprocess execution, environment inspection, mounted-secret access, provider-key usage anomalies, LiteLLM admin changes, proxy-key changes, model-route changes, outbound callbacks, cloud API activity, Kubernetes API activity, or CI/CD interaction.

·        Availability and retention of LiteLLM logs, reverse-proxy logs, WAF logs, load-balancer logs, endpoint process telemetry, container runtime telemetry, file telemetry, Kubernetes audit logs, cloud audit logs, DNS logs, proxy logs, provider audit logs, and billing records.

·        Completeness of approved LiteLLM administrator, proxy-key owner, service-account, workload identity, model-provider key, approved source, approved scanner, approved maintenance-window, approved deployment-user, and approved egress inventories.

·        Scope of credential rotation for LiteLLM proxy keys, provider API keys, gateway administrator accounts, database credentials, cloud identities, Kubernetes service accounts, CI/CD tokens, secret-manager entries, and application secrets.

·        Business disruption caused by emergency AI gateway restriction, provider-key rotation, model-route validation, AI application regression testing, agent workflow interruption, provider billing investigation, cloud containment, or customer communications.

·        Customer, contractual, regulatory, privacy, or legal exposure if prompt data, response data, user data, customer data, source code, security data, provider keys, or cloud secrets cannot be ruled out as exposed.

S6B Compliance and Risk Context

LiteLLM compromise may create compliance, contractual, privacy, AI governance, third-party-service-management, operational resilience, and customer-trust exposure when attackers can access AI prompts, responses, request metadata, provider credentials, model routes, customer-uploaded content, regulated data, source code, security analysis, internal business records, or cloud secrets.

Vulnerable-version status alone is not sufficient for breach determination. The governance question is whether the organization can prove that the gateway was not used to execute commands, access secrets, copy provider keys, read sensitive AI traffic, alter gateway policy, abuse model-provider accounts, or reach downstream cloud, Kubernetes, CI/CD, or application assets during the exposure window.

Risk Register Entry

Risk Title

LiteLLM AI Gateway Command Injection and Provider-Key Exposure

Risk Description

Attackers may exploit vulnerable LiteLLM MCP preview or test functionality to execute commands from the AI gateway runtime, access provider API keys, inspect environment variables or mounted secrets, modify gateway configuration, abuse model-provider accounts, expose prompt or response data, and force emergency AI infrastructure integrity review.

Likelihood

High for internet-facing or broadly reachable vulnerable LiteLLM deployments, especially where low-privilege proxy keys, exposed service keys, weak network segmentation, or affected authentication-bypass conditions exist.

Impact

Moderate to Severe depending on provider-key concentration, prompt and response sensitivity, customer-facing AI dependency, reachable cloud or Kubernetes secrets, gateway role scope, model-provider billing exposure, and evidence of post-exploitation activity.

Risk Rating

High

Annualized Risk Exposure

Estimated $150K - $900K for organizations with several AI gateway deployments, shared provider keys, incomplete LiteLLM logging, limited endpoint or container telemetry, and incomplete provider-key attribution.

Exposure may exceed $900K - $6.5M+ where compromise affects customer-facing AI applications, regulated prompt or response data, multiple model providers, production cloud credentials, Kubernetes service accounts, CI/CD secrets, legal review, customer communications, provider billing abuse, or public restoration of AI governance trust.

S10 Threat Overview

LiteLLM command injection creates a practical AI infrastructure exploitation risk because vulnerable gateways may be reachable from users, applications, agents, developers, service accounts, CI/CD workflows, internal networks, or the internet. Active exploitation has been confirmed through CISA KEV listing, public analysis describes affected MCP preview or test endpoints, and patching prevents re-entry without proving that previously exposed keys, secrets, gateway configuration, or prompt and response data were not accessed before remediation.

The threat centers on MCP preview or test endpoint behavior that accepts inline server configuration capable of invoking stdio transport command, argument, or environment fields. When abused, the LiteLLM runtime can spawn attacker-controlled subprocess behavior on the proxy host or container. In affected deployment conditions, the command-injection issue may also be chained with authentication-bypass behavior to reduce or remove the need for a valid LiteLLM proxy key.

This TTD treats the activity as AI gateway command execution leading to provider-key exposure and downstream AI infrastructure risk, not as a narrow LiteLLM patching event.

S13 Targets and Exposure Surface

Primary Targets

LiteLLM proxy and AI gateway deployments running vulnerable versions.

LiteLLM MCP preview or test functionality.

LiteLLM endpoints associated with MCP server connection testing and tool-list testing.

LiteLLM proxy keys, internal-user keys, administrator accounts, service keys, team keys, and application keys.

Model-provider API keys for OpenAI-compatible providers, Anthropic, Google, Azure OpenAI, AWS Bedrock, internal model gateways, and other configured providers.

LiteLLM configuration files, environment variables, mounted secrets, database credentials, request logs, prompt logs, response logs, routing tables, and provider configuration.

Cloud workload identities, Kubernetes service accounts, CI/CD tokens, secret-manager paths, container runtime secrets, and adjacent application credentials reachable from the LiteLLM runtime.

AI applications, internal copilots, agents, RAG systems, customer-facing AI features, developer tools, and automation workflows using LiteLLM for routing or policy enforcement.

Higher-Risk Deployment Conditions

LiteLLM is internet facing or reachable from broad internal networks.

LiteLLM is running versions 1.74.2 through 1.83.6.

Affected Starlette dependency conditions exist where the authentication-bypass chain may apply.

MCP preview or test endpoints are exposed through reverse proxy, API gateway, ingress, load balancer, or direct service routing.

Low-privilege LiteLLM proxy keys can reach administrative or test functionality.

Provider API keys are long-lived, over-scoped, shared across applications, or stored in environment variables.

LiteLLM runs in Kubernetes, CI/CD, developer infrastructure, cloud workloads, or shared service environments with access to mounted secrets.

LiteLLM logs prompt and response content, request metadata, user identifiers, provider routing, or application context.

Endpoint, container, file, Kubernetes, cloud, and provider audit telemetry are incomplete or short-lived.

Provider keys are shared across teams or applications without attribution.

Exposure Surface

Public LiteLLM proxy routes.

Internal AI gateway routes.

MCP preview and test endpoints.

Requests to /mcp-rest/test/connection.

Requests to /mcp-rest/test/tools/list.

Reverse-proxy, ingress, API gateway, and load-balancer routes exposing LiteLLM.

LiteLLM proxy API-key verification and role enforcement paths.

LiteLLM configuration and database storage.

Process environment variables.

Mounted secrets and service-account tokens.

Model-provider APIs and billing systems.

Cloud metadata services, secret stores, Kubernetes API, CI/CD platforms, container registries, and adjacent application infrastructure.

S17 MITRE ATT&CK Chain Flow Mapping

Stage 1: AI Gateway Exposure

The attacker reaches an internet-facing or internally reachable LiteLLM gateway running a vulnerable version.

·        T1190 Exploit Public-Facing Application, applied when the LiteLLM gateway is exposed through an internet-facing route or untrusted network path.

Stage 2: Proxy-Key or Authentication-Bypass Access

The attacker uses a valid low-privilege proxy key, exposed service key, compromised internal-user key, or an authentication-bypass chain where deployment conditions allow it.

·        T1078 Valid Accounts, applied when a valid LiteLLM proxy key, service key, or user credential is used.

·        T1190 Exploit Public-Facing Application, applied when unauthenticated access is achieved through exposed application-layer vulnerability chaining.

Stage 3: MCP Preview or Test Endpoint Abuse

The attacker sends MCP test configuration data to LiteLLM endpoint behavior capable of spawning a subprocess from the gateway runtime.

·        T1190 Exploit Public-Facing Application.

·        T1059 Command and Scripting Interpreter, applied when attacker-controlled command or interpreter execution is observed.

Stage 4: Host, Container, or Runtime Command Execution

The LiteLLM process, Python runtime, Uvicorn, Gunicorn, container process, or service context spawns shell, interpreter, file-inspection, networking, or secret-discovery utilities.

·        T1059 Command and Scripting Interpreter.

·        T1105 Ingress Tool Transfer, applied when payload staging, tool download, or remote retrieval is observed.

Stage 5: Provider-Key and Secret Access

The attacker may inspect environment variables, LiteLLM configuration, mounted secrets, cloud credentials, Kubernetes service-account tokens, database credentials, provider API keys, or request logs.

·        T1552 Unsecured Credentials.

·        T1552.001 Unsecured Credentials: Credentials In Files, applied when configuration files, .env files, mounted secret paths, or credential-bearing files are accessed.

·        T1083 File and Directory Discovery.

·        T1005 Data from Local System.

Stage 6: AI Provider Abuse and Downstream Infrastructure Impact

The attacker may use exposed provider keys, alter model routing, create or reuse proxy keys, abuse model-provider accounts, access cloud secrets, pivot into Kubernetes or CI/CD systems, or affect customer-facing AI services.

·        T1098 Account Manipulation, applied when LiteLLM keys, provider credentials, roles, users, teams, or service-account bindings are created or modified.

·        T1041 Exfiltration Over C2 Channel, applied when keys, logs, prompt data, response data, or configuration exports leave the environment.

·        T1496 Resource Hijacking, applied when stolen provider keys are abused for unauthorized model usage, quota exhaustion, or billing impact.

·        T1565.002 Stored Data Manipulation, applied when LiteLLM configuration, model routes, provider mappings, or AI workflow artifacts are modified.

Conditional Technique Notes

T1078 should only be applied where a valid key, service account, user credential, or authenticated LiteLLM identity is observed.

T1059 should only be applied where command, shell, interpreter, subprocess, or scripting behavior is observed.

T1105 should only be applied where download, staging, upload, or transfer behavior is observed.

T1496 should only be applied where unauthorized provider usage, billing abuse, or quota consumption is observed.

T1041 should only be applied where data, keys, logs, prompts, responses, or configuration material are observed leaving the environment.

This TTD should not be mapped to ransomware, malware-specific behavior, or broad cloud-control-plane activity unless supporting telemetry connects those behaviors to the LiteLLM gateway access path.

S18 Attack Path Narrative

An attacker identifies a vulnerable LiteLLM proxy or AI gateway deployment.

The attacker obtains access through a valid low-privilege LiteLLM key, exposed service key, compromised internal-user key, or an authentication-bypass chain where affected conditions exist.

The attacker targets MCP preview or test functionality such as connection testing or tool-list testing.

The attacker supplies MCP server configuration data that causes the LiteLLM runtime to spawn attacker-controlled subprocess behavior.

The command executes with the privileges of the LiteLLM proxy process, container, pod, or service account.

The attacker may inspect environment variables, mounted secrets, LiteLLM configuration files, provider keys, local database material, request logs, prompt and response stores, service-account tokens, cloud metadata, Kubernetes credentials, or CI/CD secrets.

The attacker may use exposed provider keys for unauthorized model access, quota abuse, billing fraud, data retrieval, persistence through gateway configuration changes, or lateral movement into adjacent AI, cloud, Kubernetes, CI/CD, or application infrastructure.

Defenders should detect this through correlated MCP test endpoint requests, unexpected LiteLLM subprocess execution, suspicious command lines, secret-file access, provider-key usage anomalies, LiteLLM administrative changes, rare outbound communication, Kubernetes or cloud identity activity, and model-provider audit records.

S20 TTP Analysis

Initial Access

Attackers abuse reachable LiteLLM gateway exposure through public routes, internal network access, valid proxy keys, exposed service keys, compromised internal-user keys, or authentication-bypass chaining where applicable.

Execution

Execution occurs when LiteLLM MCP preview or test behavior spawns attacker-controlled subprocesses from the proxy runtime, container, pod, or service context.

Persistence

Persistence may occur through new LiteLLM proxy keys, modified users or teams, altered model routes, malicious MCP server definitions, changed provider configuration, new gateway administrators, modified startup commands, container image changes, Kubernetes workload changes, cloud service-account changes, or CI/CD pipeline changes.

Privilege Escalation

Privilege escalation may occur if the LiteLLM runtime can read provider keys, cloud credentials, Kubernetes tokens, database credentials, secret-manager material, CI/CD tokens, or configuration that grants broader access than the original LiteLLM key or network path.

Defense Evasion

Attackers may use legitimate MCP endpoint paths, valid low-privilege proxy keys, normal-looking server configuration names, short-lived subprocesses, benign-looking commands, environment inspection, cleanup commands, cloud-hosted source IPs, or activity that resembles AI platform testing, developer debugging, or gateway validation.

Credential Access

Attackers may access provider API keys, LiteLLM proxy keys, administrator tokens, environment variables, mounted secrets, Kubernetes service-account tokens, database credentials, cloud metadata credentials, CI/CD tokens, source repository tokens, or secret-manager outputs.

Discovery

Attackers may inspect LiteLLM version state, enabled providers, configured models, users, teams, keys, routes, environment variables, mounted secret paths, cloud role identity, Kubernetes namespace context, container metadata, local files, databases, logs, and network reachability.

Command and Control / Tool Transfer

Attackers may download scripts or tools, initiate callbacks, test outbound reachability, access external paste or storage services, or use compromised provider keys from infrastructure outside the organization.

Impact

Attackers may abuse model-provider accounts, exhaust quotas, generate unexpected provider bills, expose sensitive prompts or responses, interrupt AI services, modify model routing, alter policy enforcement, pivot into cloud or Kubernetes environments, or force emergency rebuild and restoration of AI gateway trust.

S20A — Adversary Tradecraft Summary

The durable tradecraft pattern is AI gateway trust conversion: access to LiteLLM MCP preview or test functionality is converted into host or container command execution, which is then converted into provider-key discovery, gateway configuration abuse, AI service misuse, cloud or Kubernetes secret exposure, and downstream AI workflow impact. Detection should focus on that sequence rather than a single CVE label, one endpoint string, one command, one user agent, one source IP, or one provider-key anomaly.

S21 Detection Strategy Overview

Detection Philosophy

Detect LiteLLM exploitation through correlated behavior, not single indicators.

Primary Detection Anchors

Suspicious MCP preview or test endpoint requests, requests to /mcp-rest/test/connection, requests to /mcp-rest/test/tools/list, unusual POST activity from non-administrative sources, low-privilege key activity against test functionality, LiteLLM or Python subprocess creation, shell or interpreter execution, environment-variable inspection, mounted-secret access, provider-key file access, LiteLLM configuration reads or changes, model-provider usage anomalies, rare outbound communication, cloud or Kubernetes identity activity, and AI gateway administrative changes.

Detection Prioritization Model

Prioritize events where LiteLLM MCP endpoint activity is followed by subprocess execution, file or secret access, provider-key usage anomalies, gateway configuration changes, rare egress, cloud API activity, Kubernetes API activity, or provider billing anomalies within a bounded time window.

Correlation Strategy (Strict Enforcement)

Do not promote scanner traffic, one request to an MCP endpoint, one suspicious command, one provider anomaly, or cloud-only activity to high confidence without LiteLLM asset, endpoint, key, user, source, process, file, provider, identity, or time-window correlation.

Telemetry Prioritization

Prioritize LiteLLM access logs, reverse-proxy logs, WAF logs, load-balancer logs, API gateway logs, endpoint process telemetry, container runtime telemetry, file telemetry, Kubernetes audit logs, cloud audit logs, DNS logs, proxy logs, firewall logs, model-provider audit logs, provider billing telemetry, LiteLLM administrative logs, key-management records, and asset inventory.

Detection Design Constraints

Avoid detection designs based only on CVE name, vulnerable version status, single endpoint path, public proof-of-concept user agent, one source IP, one command string, one model-provider anomaly, or isolated scanner traffic.

Baseline and Deployment Requirements

Baseline approved AI platform administrators, approved LiteLLM admin source networks, approved service keys, approved MCP testing workflows, vulnerability scanners, patch-validation tools, deployment users, CI/CD runners, maintenance windows, normal LiteLLM subprocess behavior, expected provider-key usage, expected model providers, expected egress destinations, cloud workload identities, Kubernetes service accounts, and normal model-provider billing patterns.

Variant Resilience Requirements

Rules should remain useful for future LiteLLM, AI gateway, MCP, agent gateway, or model proxy abuse paths that produce the same operational behavior: privileged gateway endpoint access, command execution, provider-key exposure, secret discovery, model-provider abuse, gateway configuration tampering, and downstream cloud or Kubernetes impact.

Operational Detection Model

Run detections in hunt mode first, tune exceptions, validate joins, verify triage fields, confirm LiteLLM and reverse-proxy log retention, confirm provider audit availability, baseline approved AI platform testing, then promote to alert mode.

Explicit Non-Deployment Guardrails

Do not deploy weak WAF-only rules as compromise detection. Do not claim command execution from vulnerable version status alone. Do not claim confirmed compromise from scanner traffic, isolated MCP endpoint access, generic HTTP errors, unrelated subprocess activity, or uncorrelated provider usage. Do not attribute cloud, Kubernetes, CI/CD, or provider anomalies to LiteLLM compromise without LiteLLM host, key, user, identity, source, process, file, provider, or time-window linkage.

Figure

S22 Primary Detection Signals

Primary Detection Signals

POST requests to /mcp-rest/test/connection.

POST requests to /mcp-rest/test/tools/list.

MCP preview or test endpoint access from unauthenticated, unusual, external, non-administrative, low-privilege, or newly observed sources.

LiteLLM proxy keys or internal-user keys invoking MCP test behavior outside approved AI platform workflows.

Unusual Host header patterns where authentication-bypass chain conditions may apply.

LiteLLM, Python, Uvicorn, Gunicorn, or container runtime process spawning shell, scripting, transfer, archive, networking, or file-inspection tools.

Command lines referencing command, args, env, .env, /secrets/, Kubernetes service-account paths, provider-key variables, cloud credential paths, or LiteLLM configuration files.

Access to provider-key material, environment files, mounted secrets, LiteLLM configuration, local databases, prompt logs, response logs, request logs, or credential-bearing files after suspicious MCP endpoint activity.

Provider API usage from unfamiliar source locations, unexpected applications, unexpected models, abnormal request volume, abnormal token usage, or unusual billing patterns.

Supporting Detection Signals

HTTP 200, 400, 401, 403, 404, 422, or 500 response patterns around MCP test endpoint requests.

Repeated failed-to-success patterns involving MCP test endpoints.

Unexpected LiteLLM administrator, user, team, key, provider route, model route, or MCP server definition changes.

Outbound DNS, proxy, firewall, EDR, or container network activity from LiteLLM hosts or pods to newly seen, rare, suspicious, or unapproved destinations.

Cloud metadata, secret-manager, key-vault, Kubernetes API, CI/CD, container registry, or storage access by LiteLLM-linked workload identity after suspicious endpoint activity.

Provider-key usage after key rotation deadlines, patching, gateway restriction, or suspicious MCP activity.

Exploit Attempt and Instability Signals

Repeated requests to MCP test endpoints.

Unusual status-code patterns around MCP endpoint requests.

Unexpected subprocess errors, timeout errors, command-not-found errors, or failed external connectivity after MCP requests.

Scanner-like activity followed by subprocess execution or provider usage.

Requests containing server configuration fields that resemble stdio command execution workflows.

Outbound Communication Signals

DNS, proxy, firewall, EDR, container, or cloud network activity from LiteLLM workloads to rare, newly registered, unknown, suspicious, or unapproved destinations after suspicious MCP endpoint activity or subprocess execution.

Persistence and Post-Exploitation Signals (Conditional)

New or modified LiteLLM proxy keys, users, teams, administrators, provider routes, model aliases, MCP server definitions, startup commands, environment variables, Kubernetes manifests, cloud identities, CI/CD variables, service-account bindings, or scheduled tasks.

Lateral Movement and Expansion Signals (Conditional)

Use of provider API keys, cloud credentials, Kubernetes service-account tokens, database credentials, CI/CD tokens, source repository tokens, or application secrets against adjacent infrastructure after suspicious LiteLLM activity.

Signal Usage Constraints

Do not treat a single signal as compromise confirmation. Promote confidence when signals align by LiteLLM asset, gateway route, source IP, proxy key, user, team, endpoint path, process, command line, file path, provider key, workload identity, destination, and time window.

S23 Telemetry Requirements

Required Telemetry

LiteLLM access logs.

Reverse-proxy logs.

WAF logs.

Load-balancer or API gateway logs where applicable.

LiteLLM asset inventory.

Installed LiteLLM version inventory.

URI and query-string preservation.

Source IP and forwarded source IP fields.

HTTP method, URI path, query string, status code, user agent, request size, response size, virtual host, backend host, and timestamp.

LiteLLM user, team, proxy-key, service-key, API-key, or request identifier where available.

Endpoint or container process telemetry from LiteLLM hosts, containers, pods, or nodes.

Command-line telemetry.

Parent process telemetry.

File telemetry for LiteLLM configuration, environment files, mounted secrets, provider-key stores, local databases, prompt logs, response logs, request logs, and cloud credential paths.

Approved LiteLLM administrator lookup.

Approved administrator source-network lookup.

Approved AI platform testing lookup.

Approved scanner and validation-source lookup.

Approved maintenance-window lookup.

Provider-key ownership and application mapping.

Model-provider audit or billing logs.

Strongly Recommended Telemetry

LiteLLM administrative audit logs.

LiteLLM database change records for users, teams, keys, models, provider routes, and MCP server definitions.

Container runtime telemetry.

Kubernetes audit logs.

Kubernetes pod, namespace, service-account, and image inventory.

Cloud audit logs.

Secret Manager, Key Vault, Parameter Store, or equivalent audit logs.

Cloud metadata access telemetry where available.

DNS logs.

Proxy logs.

Firewall logs.

EDR network telemetry.

CI/CD logs.

Container registry logs.

Source repository audit logs.

Database access logs.

Prompt and response store access logs.

CMDB records.

Cloud workload identity mapping.

Recently seen domain enrichment.

Domain reputation enrichment.

Known-good LiteLLM configuration baseline.

Approved deployment-user lookup.

Approved provider-key usage baseline.

Approved egress-destination lookup.

Local Mapping Required

LiteLLM deployment identifier.

Virtual host.

Backend host.

Container ID.

Pod name.

Namespace.

Node name.

Cloud account, subscription, or project.

Workload identity.

Source IP.

Forwarded source IP.

Authenticated LiteLLM user where available.

LiteLLM proxy-key identifier where available.

LiteLLM team or application identifier where available.

HTTP method.

URI path.

Query string.

Status code.

User agent.

Request ID.

Process name.

Parent process.

Command line.

Process user.

File path.

File extension.

File hash where available.

File owner.

File action.

Provider key identifier.

Model provider.

Model name.

Application identifier.

Destination domain.

Destination IP.

Cloud API action.

Kubernetes API action.

CI/CD user or token.

Maintenance-window status.

Approved administrator status.

Approved scanner status.

Approved AI platform testing status.

Approved deployment-user status.

Approved egress-destination status.

Approved provider-key usage status.

S24 Detection Opportunities and Gaps

Detection Opportunities

Suspicious LiteLLM MCP test endpoint activity can be correlated with subprocess execution, secret access, provider-key usage anomalies, rare egress, gateway configuration changes, and downstream cloud or Kubernetes identity activity.

Endpoint and container telemetry may directly reveal command execution from LiteLLM, Python, Uvicorn, Gunicorn, or service-wrapper process lineage.

File telemetry may reveal access to environment files, mounted secrets, provider-key stores, LiteLLM configuration, local databases, or request and response logs even when HTTP logs are incomplete.

Reverse-proxy, WAF, load-balancer, and API gateway logs may preserve endpoint-path evidence when LiteLLM application logs are incomplete.

Model-provider audit and billing records may reveal unauthorized use of exposed provider keys after gateway compromise.

Kubernetes, cloud, CI/CD, and secret-manager logs may reveal downstream impact from identities reachable by the LiteLLM runtime.

Detection Gaps

LiteLLM logs may not retain user, team, proxy-key, or request-level attribution.

Web logs may rotate quickly.

Reverse proxies may not preserve full path, query, request body, or forwarded source fields.

Request bodies containing MCP configuration may not be logged for privacy or operational reasons.

Endpoint process telemetry may be unavailable inside minimal containers.

Environment-variable access may not generate file telemetry.

Provider keys may be shared across applications, reducing attribution confidence.

Model-provider audit logs may not show internal LiteLLM users or teams.

Cloud, Kubernetes, and provider logs alone cannot prove LiteLLM exploitation.

Prompt and response store review may require legal, privacy, or data governance approval.

Compensating Controls

Use LiteLLM logs, reverse-proxy logs, WAF logs, load-balancer logs, endpoint telemetry, container telemetry, Kubernetes audit logs, cloud audit logs, provider audit logs, billing records, secret-manager logs, gateway configuration review, provider-key rotation evidence, and known-good configuration comparison.

Do not rely only on LiteLLM version validation if suspicious activity occurred during the exposure window.

S25 Ultra-Tuned Detection Engineering Rules

NDR / Network Behavioral Analytics

Detection Viability Assessment

Production-deployable where HTTP request metadata, WAF logs, reverse-proxy logs, API gateway logs, URI paths, response status, virtual host, backend host, source identity, DNS, proxy, provider egress, and asset-enrichment data can be joined to the same LiteLLM gateway asset. Pure NetFlow is not sufficient because this detection requires LiteLLM MCP-specific URI visibility and gateway identity. Use NDR as a correlation layer when it can consume URI-aware gateway telemetry, DNS/proxy telemetry, model-provider destination context, and LiteLLM asset inventory.

Rule

LiteLLM MCP Test Endpoint Request With Rare Egress or Provider Activity Correlation

Rule Format

Production-deployable NDR / WAF / reverse-proxy behavioral correlation pattern requiring local query-language translation.

Detection Purpose

Detect suspicious LiteLLM MCP preview or test endpoint activity that aligns with abnormal source context, suspicious response patterns, rare outbound communication, model-provider API anomalies, or downstream network behavior from the same LiteLLM gateway host, pod, container, backend service, or workload identity.

Detection Logic

Trigger when a LiteLLM gateway receives POST activity involving /mcp-rest/test/connection, /mcp-rest/test/tools/list, or locally confirmed MCP preview or test endpoint variants from a source that is not an approved LiteLLM administrator source, approved AI platform testing workflow, approved vulnerability scanner, approved patch-validation source, or approved deployment workflow.

Assign medium severity when suspicious MCP endpoint activity occurs against a LiteLLM gateway asset outside approved testing or maintenance.

Assign high severity when suspicious MCP endpoint activity is followed within 60 minutes by rare outbound communication from the same LiteLLM host, pod, container, backend host, NAT egress identity, or workload identity.

Assign high severity when suspicious MCP endpoint activity includes unusual source context, unexpected Host header behavior, repeated failure-to-success patterns, suspicious user agent context, abnormal response status sequence, or access by a low-privilege key outside approved testing.

Promote to critical when suspicious MCP endpoint activity is followed within 120 minutes by endpoint-confirmed subprocess execution, secret-file access, provider-key usage anomalies, cloud or Kubernetes identity activity, LiteLLM administrative change, model-route change, proxy-key creation, or confirmed provider-key exposure.

Required Telemetry

WAF logs.

Reverse-proxy logs.

Load-balancer logs.

API gateway logs.

LiteLLM access logs.

HTTP request metadata with URI path preservation.

DNS logs.

Proxy logs.

NDR session telemetry.

Model-provider audit or provider egress logs where available.

LiteLLM gateway asset inventory.

Virtual host to backend host mapping.

Container, pod, namespace, node, or workload mapping where applicable.

Approved LiteLLM administrator source lookup.

Approved AI platform testing lookup.

Approved scanner lookup.

Approved validation-source lookup.

Approved deployment-source lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Approved model-provider domain lookup.

Recently seen destination-domain baseline.

Engineering Implementation Instructions

Map http_method, uri_path, query_string, full_url, source_ip, x_forwarded_for, user_agent, http_status, request_size, response_size, virtual_host, backend_host, backend_ip, litellm_gateway_id, src_host, pod_name, container_id, namespace, node_name, workload_identity, dest_domain, dest_ip, dns_query, proxy_action, provider_domain, and event timestamp fields to the customer’s local schema.

Build the LiteLLM gateway asset group from CMDB records, vulnerability-management inventory, AI platform inventory, DNS records, reverse-proxy routing, load-balancer target groups, Kubernetes labels, cloud tags, container images, deployment manifests, and LiteLLM version records.

Create and validate local lookups for approved LiteLLM administrator sources, approved AI platform testing sources, approved vulnerability scanners, approved validation sources, approved deployment sources, approved maintenance windows, approved LiteLLM egress destinations, approved model-provider domains, approved business destinations, and recently seen destination baselines.

Validate URI visibility before deployment. The rule requires URI-preserving WAF, reverse-proxy, ingress, API gateway, load-balancer, or LiteLLM access logs. Pure NetFlow or encrypted traffic without HTTP metadata is not sufficient.

Do not require request-body logging for production deployment. Request-body inspection may support investigation, but the deployable rule should work from endpoint path, source context, response behavior, gateway identity, egress, provider audit, and host or workload correlation.

Translate the Detection Query Pattern into the target NDR, WAF, proxy, or SIEM language using local lookup, enrichment, asset-group, and time-window functions.

Validate joins between virtual host, backend host, pod, container, node, source IP, forwarded source IP, NAT egress identity, workload identity, provider domain, and LiteLLM gateway identifier before alert-mode deployment.

Run the rule in hunt mode first to baseline approved AI platform testing, scanner activity, deployment workflows, provider validation, patch validation, maintenance activity, and expected model-provider egress.

Treat TLS termination, URI preservation, source-IP preservation, backend-host mapping, container-to-host mapping, workload identity mapping, provider egress baselining, allowlist tuning, false-positive testing, query-performance validation, SOC triage fields, and alert-routing ownership as required local deployment work.

DRI Assessment

High resilience where URI visibility, LiteLLM gateway asset inventory, virtual-host mapping, container or pod mapping, DNS/proxy telemetry, model-provider destination context, approved-source enrichment, and rare-destination baselining are available.

DRI

8.6 / 10

TCR Assessment

Operational confidence is moderate for standalone suspicious MCP endpoint activity and high when MCP endpoint activity aligns with rare egress, provider API anomalies, endpoint execution, LiteLLM configuration changes, secret access, or cloud and Kubernetes workload activity.

Operational TCR

8.0 / 10

Full-Telemetry TCR

9.2 / 10

Limitations

Encrypted traffic without WAF, reverse-proxy, ingress, API gateway, or server-side HTTP logging may hide LiteLLM URI paths. Request bodies may not be logged or may be intentionally excluded for privacy and performance reasons. Legitimate AI platform testing, emergency remediation, patch validation, or scanner workflows may touch MCP test endpoints. Critical promotion requires local correlation to process telemetry, file telemetry, provider telemetry, cloud telemetry, Kubernetes telemetry, LiteLLM administrative state, or confirmed secret exposure.

Detection Query Pattern

Production-deployable behavioral correlation pattern pending local query-language translation:

DATASETS:
ENV_WAF
ENV_REVERSE_PROXY
ENV_LOAD_BALANCER
ENV_API_GATEWAY
ENV_LITELLM_ACCESS
ENV_DNS
ENV_PROXY
ENV_FIREWALL
ENV_NDR
ENV_PROVIDER_AUDIT

SCOPE:
normalized_gateway IN ASSET_GROUP("litellm_gateways")

NORMALIZATION:
normalized_gateway =
coalesce(
litellm_gateway_id,
backend_host,
virtual_host,
src_host,
pod_name,
container_id,
workload_identity,
nat_egress_identity
)

WINDOW:
60 minutes for MCP endpoint, rare egress, and provider-usage correlation.
120 minutes for confirmed provider-key exposure, LiteLLM administrative change, cloud identity activity, Kubernetes identity activity, or endpoint-confirmed execution correlation.

SIGNAL suspicious_litellm_mcp_test:
event_dataset IN (
"ENV_WAF",
"ENV_REVERSE_PROXY",
"ENV_LOAD_BALANCER",
"ENV_API_GATEWAY",
"ENV_LITELLM_ACCESS"
)
AND normalized_gateway IN ASSET_GROUP("litellm_gateways")
AND http_method = "POST"
AND (
uri_path CONTAINS "/mcp-rest/test/connection"
OR uri_path CONTAINS "/mcp-rest/test/tools/list"
OR full_url CONTAINS "/mcp-rest/test/connection"
OR full_url CONTAINS "/mcp-rest/test/tools/list"
OR uri_path MATCHES ".*mcp.test."
OR full_url MATCHES ".*mcp.test."
)
AND source_ip NOT IN LOOKUP("APPROVED_LITELLM_ADMIN_SOURCES")
AND source_ip NOT IN LOOKUP("APPROVED_AI_PLATFORM_TESTING_SOURCES")
AND source_ip NOT IN LOOKUP("APPROVED_LITELLM_SCANNERS")
AND source_ip NOT IN LOOKUP("APPROVED_LITELLM_VALIDATION_SOURCES")
AND source_ip NOT IN LOOKUP("APPROVED_LITELLM_DEPLOYMENT_SOURCES")
AND maintenance_window = false

SIGNAL suspicious_mcp_access_context:
event_dataset IN (
"ENV_WAF",
"ENV_REVERSE_PROXY",
"ENV_LOAD_BALANCER",
"ENV_API_GATEWAY",
"ENV_LITELLM_ACCESS"
)
AND normalized_gateway IN ASSET_GROUP("litellm_gateways")
AND http_method = "POST"
AND (
uri_path CONTAINS "/mcp-rest/test/connection"
OR uri_path CONTAINS "/mcp-rest/test/tools/list"
OR full_url CONTAINS "/mcp-rest/test/connection"
OR full_url CONTAINS "/mcp-rest/test/tools/list"
OR uri_path MATCHES ".*mcp.test."
OR full_url MATCHES ".*mcp.test."
)
AND (
repeated_failure_to_success = true
OR source_ip_first_seen_for_gateway = true
OR host_header_anomaly = true
OR proxy_key_id NOT IN LOOKUP("APPROVED_LITELLM_MCP_TESTING_KEYS")
OR litellm_user NOT IN LOOKUP("APPROVED_LITELLM_MCP_TESTING_USERS")
OR user_agent IN LOOKUP("UNUSUAL_OR_AUTOMATION_USER_AGENTS")
OR request_count_by_source_gateway_10m > ENV_MCP_TEST_REQUEST_COUNT_THRESHOLD
)
AND maintenance_window = false

SIGNAL rare_litellm_gateway_egress:
event_dataset IN (
"ENV_DNS",
"ENV_PROXY",
"ENV_FIREWALL",
"ENV_NDR"
)
AND normalized_gateway IN ASSET_GROUP("litellm_gateways")
AND dest_domain NOT IN LOOKUP("APPROVED_LITELLM_EGRESS_DESTINATIONS")
AND dest_domain NOT IN LOOKUP("APPROVED_MODEL_PROVIDER_DOMAINS")
AND dest_domain NOT IN LOOKUP("APPROVED_BUSINESS_DOMAINS")
AND (
dest_domain_age_days < ENV_NEW_DOMAIN_AGE_DAYS
OR domain_reputation IN ("unknown","suspicious","malicious")
OR dest_country NOT IN LOOKUP("APPROVED_EGRESS_COUNTRIES")
OR dest_domain_first_seen_for_gateway = true
)
AND network_action IN ("allowed","proxied","connected")

SIGNAL unusual_provider_key_activity:
event_dataset = "ENV_PROVIDER_AUDIT"
AND provider_key_id IN LOOKUP("LITELLM_PROVIDER_KEYS")
AND (
provider_source_ip NOT IN LOOKUP("APPROVED_PROVIDER_KEY_SOURCE_IPS")
OR model_name NOT IN LOOKUP("APPROVED_PROVIDER_MODELS")
OR provider_account NOT IN LOOKUP("APPROVED_PROVIDER_ACCOUNTS")
OR request_volume_ratio > ENV_PROVIDER_USAGE_RATIO_THRESHOLD
OR token_usage_ratio > ENV_PROVIDER_TOKEN_RATIO_THRESHOLD
OR provider_geo NOT IN LOOKUP("APPROVED_PROVIDER_GEOS")
OR provider_key_first_seen_from_source = true
)

SIGNAL confirmed_downstream_impact:
locally_enriched_confirmed_provider_key_exposure = true
OR locally_enriched_confirmed_litellm_admin_change = true
OR locally_enriched_confirmed_proxy_key_creation = true
OR locally_enriched_confirmed_model_route_change = true
OR locally_enriched_confirmed_cloud_identity_activity = true
OR locally_enriched_confirmed_kubernetes_identity_activity = true
OR locally_enriched_confirmed_endpoint_process_execution = true

CORRELATION high_confidence_network_pattern:
suspicious_litellm_mcp_test NEAR (
suspicious_mcp_access_context
OR rare_litellm_gateway_egress
OR unusual_provider_key_activity
)
BY normalized_gateway
WITHIN 60 minutes

CORRELATION critical_network_pattern:
suspicious_litellm_mcp_test NEAR confirmed_downstream_impact
BY normalized_gateway
WITHIN 120 minutes

SEVERITY:
medium WHEN suspicious_litellm_mcp_test = true
AND high_confidence_network_pattern = false
AND critical_network_pattern = false

high WHEN high_confidence_network_pattern = true

critical WHEN critical_network_pattern = true

OUTPUT:
severity
normalized_gateway
litellm_gateway_id
backend_host
virtual_host
src_host
pod_name
container_id
namespace
node_name
workload_identity
nat_egress_identity
source_ip
x_forwarded_for
user_agent
host_header
uri_path
query_string
http_method
http_status
request_size
response_size
proxy_key_id
litellm_user
litellm_team
dest_domain
dest_ip
dest_country
domain_reputation
dest_domain_age_days
provider_key_id
provider_account
model_name
provider_source_ip
request_volume_ratio
token_usage_ratio
first_seen
last_seen
correlation_window
matched_signals

SentinelOne

Detection Viability Assessment

Production-deployable where SentinelOne covers LiteLLM hosts, container nodes, or gateway workloads and captures process creation, parent process, command line, process user, file telemetry, and network telemetry. This rule is strongest on self-managed LiteLLM hosts, dedicated AI gateway servers, and Kubernetes nodes where LiteLLM process behavior can be separated from approved deployment, debugging, provider validation, and AI platform maintenance.

Rule

LiteLLM Runtime Service Context Execution With Secret Access or Rare Egress

Rule Format

SentinelOne Deep Visibility query pattern for LiteLLM host behavior requiring local field validation.

Detection Purpose

Detect suspicious LiteLLM service-context execution, command-line behavior, secret-file access, provider-key discovery, environment inspection, or outbound tooling that may indicate command execution after MCP test endpoint abuse.

Detection Logic

Trigger when a LiteLLM host, container, or runtime records LiteLLM, Python, Uvicorn, Gunicorn, container runtime, or service-wrapper process activity aligned with shell invocation, interpreter execution, transfer-tool use, environment inspection, archive-tool use, credential-file access, mounted-secret access, or rare outbound network activity.

Assign medium severity for suspicious LiteLLM subprocess execution on a LiteLLM gateway asset.

Assign high severity when suspicious subprocess execution aligns with provider-key variable references, mounted-secret access, environment inspection, cloud credential paths, Kubernetes service-account access, LiteLLM configuration reads, or unapproved outbound connection.

Promote to critical when correlated with MCP test endpoint web activity, known vulnerable LiteLLM version, provider-key usage anomaly, LiteLLM configuration change, cloud identity activity, Kubernetes API activity, or confirmed provider-key exposure.

Required Telemetry

SentinelOne process telemetry.

SentinelOne command-line telemetry.

SentinelOne parent process telemetry.

SentinelOne file telemetry.

SentinelOne network telemetry.

LiteLLM gateway endpoint group.

LiteLLM runtime process mapping.

LiteLLM service account mapping.

Container or Kubernetes node mapping where applicable.

Approved deployment-user lookup.

Approved deployment-command lookup.

Approved AI platform testing lookup.

Approved provider-validation lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Engineering Implementation Instructions

Map EndpointName, EndpointId, AgentUuid, ProcessName, ProcessImagePath, ParentProcessName, ParentProcessImagePath, ProcessUser, CmdLine, FilePath, FileEventType, DstIp, DstPort, DstDomain, EventTime, container identifier, pod name, namespace, and node name where available.

Create local asset groups for LiteLLM gateways, LiteLLM runtime processes, LiteLLM configuration paths, LiteLLM secret paths, LiteLLM service users, LiteLLM container nodes, Kubernetes worker nodes hosting LiteLLM, and approved AI gateway maintenance systems.

Validate local LiteLLM runtime process names and service wrappers, including litellm, python, python3, uvicorn, gunicorn, containerd-shim, docker, Kubernetes runtime wrappers, and systemd service paths.

Create exceptions for approved deployments, container image pulls, AI platform testing, MCP validation, provider validation, patching, emergency remediation, vulnerability validation, backup jobs, authorized debugging, key rotation, and approved incident-response cleanup.

Do not suppress all Python, Uvicorn, Gunicorn, shell, or curl behavior globally. Scope suppression to approved command patterns, approved users, approved maintenance windows, approved deployment sources, approved provider validation, and approved LiteLLM operational workflows.

Validate that SentinelOne covers the actual LiteLLM runtime layer. For containerized deployments, confirm whether telemetry is collected from the container, the node, the runtime shim, or the host process namespace, and tune process lineage accordingly.

Validate file telemetry for .env files, mounted secrets, Kubernetes service-account token paths, LiteLLM configuration files, provider-key files, local database files, credential files, prompt logs, response logs, and request logs.

Validate network telemetry for rare egress from LiteLLM hosts, pods, containers, or nodes to destinations outside approved model providers and approved business destinations.

Run in hunt mode before alert mode to baseline normal LiteLLM startup, deployments, provider validation, key rotation, application testing, debugging, backup activity, and gateway administration.

Treat endpoint grouping, runtime process mapping, service-account mapping, container-node mapping, approved command exceptions, maintenance-window suppression, secret-path validation, egress baseline validation, false-positive testing, and SOC alert routing as required local deployment work.

DRI Assessment

High where LiteLLM hosts or container nodes have SentinelOne coverage with process lineage, command-line, file, and network telemetry enabled.

DRI

8.8 / 10

TCR Assessment

Strong for suspicious LiteLLM runtime execution and secret access when scoped to AI gateway hosts. Strongest when correlated with MCP endpoint web activity, provider-key anomalies, LiteLLM administrative state, or cloud and Kubernetes telemetry.

Operational TCR

8.4 / 10

Full-Telemetry TCR

9.3 / 10

Limitations

Legitimate LiteLLM deployments, MCP testing, provider validation, debugging, patching, container updates, and emergency remediation may spawn interpreters, read configuration, or test outbound provider connectivity. Container-only deployments may require node-level or runtime telemetry. This rule must be scoped to LiteLLM assets and tuned against approved AI platform workflows before alert-mode deployment.

Detection Query Pattern

SentinelOne Deep Visibility / STAR translation pattern with local field mapping, asset scoping, and suppression implementation required:

SCOPE:
EndpointName IN ENV_LITELLM_GATEWAYS
OR EndpointId IN ENV_LITELLM_GATEWAY_ENDPOINT_IDS
OR AgentUuid IN ENV_LITELLM_GATEWAY_AGENT_UUIDS
OR PodName IN ENV_LITELLM_GATEWAY_PODS
OR ContainerId IN ENV_LITELLM_GATEWAY_CONTAINERS
OR KubernetesNamespace IN ENV_LITELLM_GATEWAY_NAMESPACES

NORMALIZATION:
normalized_gateway =
coalesce(
LiteLLMGatewayId,
EndpointName,
EndpointId,
AgentUuid,
PodName,
ContainerId,
KubernetesNamespace,
NodeName
)

child_process_name =
coalesce(
TgtProcName,
ProcessName,
IndicatorName,
ChildProcessName,
TargetProcessName
)

SIGNAL litellm_runtime_context:
(
ProcessUser IN ENV_LITELLM_SERVICE_USERS
OR SrcProcName IN ("litellm","python","python3","uvicorn","gunicorn")
OR SrcProcCmdLine CONTAINS "litellm"
OR SrcProcCmdLine CONTAINS "proxy_server"
OR SrcProcCmdLine CONTAINS "litellm.proxy"
OR ParentProcessName IN ("litellm","python","python3","uvicorn","gunicorn")
OR ParentProcessCmdLine CONTAINS "litellm"
OR ParentProcessCmdLine CONTAINS "proxy_server"
OR ContainerImage CONTAINS "litellm"
OR KubernetesLabels CONTAINS "litellm"
)

SIGNAL suspicious_child_process:
litellm_runtime_context = true
AND child_process_name IN (
"sh","bash","dash","zsh","python","python3","perl","ruby","node","curl","wget","nc","ncat","socat","openssl","env","printenv","cat","grep","awk","sed","id","whoami","uname","tar","zip","7z"
)
AND CmdLine NOT MATCHES LOOKUP("APPROVED_LITELLM_DEPLOYMENT_COMMAND_PATTERNS")
AND CmdLine NOT MATCHES LOOKUP("APPROVED_LITELLM_TESTING_COMMAND_PATTERNS")
AND CmdLine NOT MATCHES LOOKUP("APPROVED_LITELLM_PROVIDER_VALIDATION_PATTERNS")
AND ProcessUser NOT IN LOOKUP("APPROVED_LITELLM_DEPLOYMENT_USERS")
AND maintenance_window = false

SIGNAL suspicious_command_content:
litellm_runtime_context = true
AND (
CmdLine CONTAINS ".env"
OR CmdLine CONTAINS "/secrets/"
OR CmdLine CONTAINS "/var/run/secrets/kubernetes.io/serviceaccount/"
OR CmdLine CONTAINS "serviceaccount"
OR CmdLine CONTAINS "OPENAI_API_KEY"
OR CmdLine CONTAINS "ANTHROPIC_API_KEY"
OR CmdLine CONTAINS "AZURE_OPENAI"
OR CmdLine CONTAINS "AWS_ACCESS_KEY"
OR CmdLine CONTAINS "GOOGLE_APPLICATION_CREDENTIALS"
OR CmdLine CONTAINS "printenv"
OR CmdLine CONTAINS "env "
OR CmdLine CONTAINS "bash -c"
OR CmdLine CONTAINS "/bin/sh -c"
OR CmdLine CONTAINS "curl "
OR CmdLine CONTAINS "wget "
OR CmdLine CONTAINS "169.254.169.254"
OR CmdLine CONTAINS "metadata.google.internal"
OR CmdLine CONTAINS "sts.amazonaws.com"
)
AND CmdLine NOT MATCHES LOOKUP("APPROVED_LITELLM_COMMAND_PATTERNS")
AND ProcessUser NOT IN LOOKUP("APPROVED_LITELLM_DEPLOYMENT_USERS")
AND maintenance_window = false

SIGNAL suspicious_secret_or_config_file_access:
(
EndpointName IN ENV_LITELLM_GATEWAYS
OR EndpointId IN ENV_LITELLM_GATEWAY_ENDPOINT_IDS
OR AgentUuid IN ENV_LITELLM_GATEWAY_AGENT_UUIDS
OR PodName IN ENV_LITELLM_GATEWAY_PODS
OR ContainerId IN ENV_LITELLM_GATEWAY_CONTAINERS
)
AND (
litellm_runtime_context = true
OR ProcessName IN LOOKUP("APPROVED_LITELLM_RUNTIME_PROCESS_NAMES")
OR ProcessUser IN ENV_LITELLM_SERVICE_USERS
)
AND FilePath MATCHES ".*(/\.env$|/secrets/|/var/run/secrets/kubernetes.io/serviceaccount/|litellm.*config|provider.*key|api.key|database|credentials)."
AND FileEventType IN ("opened","read","created","modified","renamed","written")
AND ProcessUser NOT IN LOOKUP("APPROVED_LITELLM_DEPLOYMENT_USERS")
AND maintenance_window = false

SIGNAL rare_litellm_runtime_egress:
litellm_runtime_context = true
AND DstDomain NOT IN LOOKUP("APPROVED_LITELLM_EGRESS_DESTINATIONS")
AND DstDomain NOT IN LOOKUP("APPROVED_MODEL_PROVIDER_DOMAINS")
AND DstDomain NOT IN LOOKUP("APPROVED_BUSINESS_DOMAINS")
AND (
DstDomainFirstSeenForGateway = true
OR DstDomainReputation IN ("unknown","suspicious","malicious")
OR DstCountry NOT IN LOOKUP("APPROVED_EGRESS_COUNTRIES")
)
AND NetworkAction IN ("allowed","connected","proxied")
AND maintenance_window = false

CORRELATION high_confidence_endpoint_pattern:
(
suspicious_child_process = true
OR suspicious_command_content = true
OR suspicious_secret_or_config_file_access = true
OR rare_litellm_runtime_egress = true
)
BY normalized_gateway
WITHIN 60 minutes

CORRELATION critical_endpoint_pattern:
high_confidence_endpoint_pattern = true
AND (
locally_enriched_suspicious_mcp_endpoint_activity = true
OR locally_enriched_confirmed_provider_key_exposure = true
OR locally_enriched_provider_key_usage_anomaly = true
OR locally_enriched_litellm_admin_change = true
OR locally_enriched_cloud_identity_activity = true
OR locally_enriched_kubernetes_identity_activity = true
)
BY normalized_gateway
WITHIN 120 minutes

SEVERITY:
medium WHEN (
suspicious_child_process = true
OR suspicious_secret_or_config_file_access = true
)
AND suspicious_command_content = false
AND rare_litellm_runtime_egress = false
AND critical_endpoint_pattern = false

high WHEN high_confidence_endpoint_pattern = true
AND critical_endpoint_pattern = false
AND (
suspicious_command_content = true
OR suspicious_secret_or_config_file_access = true
OR rare_litellm_runtime_egress = true
)

critical WHEN critical_endpoint_pattern = true

OUTPUT:
severity
normalized_gateway
EndpointName
EndpointId
AgentUuid
ProcessUser
SrcProcName
SrcProcCmdLine
child_process_name
CmdLine
ParentProcessName
ParentProcessCmdLine
FilePath
FileEventType
DstIp
DstPort
DstDomain
DstCountry
DstDomainReputation
PodName
ContainerId
KubernetesNamespace
NodeName
ContainerImage
KubernetesLabels
EventTime
matched_signals

Splunk

Detection Viability Assessment

Production-deployable where Splunk ingests URI-preserving LiteLLM, WAF, reverse-proxy, endpoint process, endpoint file, container, DNS/proxy, Kubernetes, cloud, and model-provider audit logs. Use normalized fields, lookups, bounded windows, and macros. Avoid broad raw searches across all HTTP, process, provider, or cloud logs and avoid unbounded joins.

Rule

LiteLLM MCP Test Endpoint Followed by Runtime Execution or Provider-Key Exposure Evidence

Rule Format

Splunk SPL production correlation search with local index, sourcetype, macro, and lookup mapping.

Detection Purpose

Detect suspicious LiteLLM MCP test endpoint activity followed by subprocess execution, secret access, provider-key access, unusual provider use, rare egress, LiteLLM administrative change, or cloud and Kubernetes activity tied to the same gateway deployment.

Detection Logic

Trigger when suspicious MCP test endpoint activity occurs and is followed within 60 minutes by one or more execution or compromise-validation signals on the same LiteLLM gateway, virtual host, backend host, pod, container, workload identity, or mapped asset.

Assign high severity when suspicious MCP endpoint activity aligns with LiteLLM subprocess execution, secret-file access, provider-key usage anomaly, rare egress, LiteLLM admin change, Kubernetes API activity, or cloud identity activity.

Promote to critical when confirmed provider-key exposure, prompt or response access, cloud secret access, Kubernetes service-account abuse, CI/CD token access, unauthorized provider billing activity, malicious gateway configuration change, or multiple-application impact is observed.

Required Telemetry

LiteLLM access logs.

WAF logs.

Reverse-proxy logs.

Endpoint process logs.

Endpoint file logs.

Container runtime logs where available.

DNS logs.

Proxy logs.

Kubernetes audit logs where available.

Cloud audit logs where available.

LiteLLM administrative logs or database change records where available.

Model-provider audit or billing logs.

Approved LiteLLM administrator lookup.

Approved administrator source lookup.

Approved AI platform testing lookup.

Approved scanner lookup.

Approved deployment-user lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Approved provider-key usage lookup.

LiteLLM gateway asset lookup.

Engineering Implementation Instructions

Map indexes and sourcetypes for LiteLLM access logs, web logs, WAF logs, reverse-proxy logs, EDR process logs, EDR file logs, container logs, DNS logs, proxy logs, Kubernetes audit logs, cloud audit logs, provider audit logs, provider billing logs, and LiteLLM administrative records.

Normalize litellm_gateway_id, virtual_host, backend_host, dest_host, pod_name, namespace, container_id, node_name, workload_identity, uri_path, query_string, full_url, http_method, src_ip, x_forwarded_for, user_agent, status, request_id, litellm_user, litellm_team, proxy_key_id, provider_key_id, model_provider, model_name, file_path, file_action, process_name, parent_process_name, process_user, command_line, dest_domain, dest_ip, cloud_action, kubernetes_action, and _time.

Create lookups for LiteLLM assets, approved administrators, approved administrator sources, approved AI platform testing sources, approved scanners, approved deployment users, approved deployment sources, approved maintenance windows, approved command patterns, approved egress destinations, approved model-provider domains, approved provider-key sources, approved provider-key ownership, approved workload identities, and approved business destinations.

Create macros for local field normalization, LiteLLM asset scoping, MCP endpoint detection, runtime process detection, secret-path detection, provider-usage anomaly detection, rare-egress detection, maintenance-window evaluation, approved-source suppression, and output-field formatting.

Avoid broad raw joins, unrestricted subsearches, unbounded append, repeated mvexpand, and direct high-cardinality correlation across all web, process, provider, and cloud logs.

Use staged candidate searches, accelerated datasets, summary indexes, data models, or scheduled intermediate lookups where available. Correlate candidates by normalized LiteLLM gateway identity, backend host, virtual host, pod, container, workload identity, provider key, source, and bounded time window.

Validate that LiteLLM, WAF, reverse-proxy, ingress, load-balancer, or API gateway logs retain URI paths and that virtual hosts can join reliably to backend hosts, endpoint hostnames, container IDs, pod names, workload identities, provider telemetry, and cloud telemetry.

Validate macro expansion, lookup return fields, lookup key names, lookup output fields, lookup suppression behavior, time-zone handling, clock skew, event delay, field cardinality, and query run time before alert-mode deployment.

Run in hunt mode against at least 14 to 30 days of history to tune scanners, AI platform testing, LiteLLM administration, provider validation, patch validation, deployments, container updates, key rotations, vulnerability validation, emergency remediation, and normal provider egress.

Treat local index mapping, field normalization, lookup creation, macro creation, staged-correlation design, false-positive baselining, query-performance testing, SOC workflow routing, and hunt-to-alert promotion as required local deployment work.

DRI Assessment

High where Splunk can join LiteLLM HTTP activity, WAF, endpoint, container, DNS/proxy, provider audit, cloud, Kubernetes, and administrative telemetry by gateway, virtual host, backend host, pod, container, source, file path, process, provider key, destination, and time window.

DRI

9.0 / 10

TCR Assessment

High when MCP endpoint activity is followed by LiteLLM runtime execution or secret access. Highest when provider audit, LiteLLM administrative state, Kubernetes, and cloud telemetry are available.

Operational TCR

8.6 / 10

Full-Telemetry TCR

9.5 / 10

Limitations

Effectiveness depends on URI logging, asset mapping, endpoint or container telemetry, provider-key attribution, and provider audit availability. Legitimate AI platform testing, patch validation, admin debugging, key rotation, provider validation, container updates, and emergency remediation may require exceptions. Critical promotion requires stronger evidence than request telemetry alone.

Splunk

Detection Query Pattern

Splunk SPL production correlation pattern requiring local macro, lookup, index, sourcetype, field-mapping, summary-index, and performance validation:

SEARCH DESIGN:
Use staged candidate datasets rather than broad raw joins, unrestricted subsearches, unbounded append, or high-cardinality cross-source correlation. Where possible, materialize each candidate signal into a summary index, accelerated dataset, or scheduled lookup before running the final correlation search.

CANDIDATE SEARCH 1:
Suspicious LiteLLM MCP test endpoint activity.

index=ENV_LITELLM_OR_WEB_INDEX sourcetype IN (ENV_LITELLM_OR_WEB_SOURCETYPES)
| litellm_web_field_normalization
| litellm_gateway_asset_scope
| eval http_method=upper(coalesce(http_method,method))
| eval request_all=coalesce(full_url,url,request,"")
| eval uri_path=coalesce(uri_path,url_path,request_path,"")
| eval src_ip=coalesce(src_ip,source_ip,client_ip,c_ip)
| eval normalized_gateway=coalesce(litellm_gateway_id,virtual_host,backend_host,dest_host,pod_name,container_id,workload_identity)
| where http_method="POST"
| where (
like(uri_path,"%/mcp-rest/test/connection%")
OR like(uri_path,"%/mcp-rest/test/tools/list%")
OR like(request_all,"%/mcp-rest/test/connection%")
OR like(request_all,"%/mcp-rest/test/tools/list%")
OR match(uri_path,"(?i).*mcp.test.")
OR match(request_all,"(?i).*mcp.test.")
)
| lookup APPROVED_LITELLM_ADMIN_SOURCES src_ip OUTPUT src_ip AS approved_admin_source
| lookup APPROVED_AI_PLATFORM_TESTING_SOURCES src_ip OUTPUT src_ip AS approved_ai_testing_source
| lookup APPROVED_LITELLM_SCANNERS src_ip OUTPUT src_ip AS approved_scanner_source
| lookup APPROVED_LITELLM_DEPLOYMENT_SOURCES src_ip OUTPUT src_ip AS approved_deployment_source
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| where isnull(approved_admin_source)
AND isnull(approved_ai_testing_source)
AND isnull(approved_scanner_source)
AND isnull(approved_deployment_source)
AND (isnull(maintenance_window) OR maintenance_window!="true")
| eval signal="suspicious_litellm_mcp_test"
| eval signal_time=_time
| eval mcp_window_start=_time-300
| eval mcp_window_end=_time+3600
| table signal_time mcp_window_start mcp_window_end normalized_gateway litellm_gateway_id virtual_host backend_host dest_host pod_name container_id namespace workload_identity src_ip x_forwarded_for user_agent uri_path query_string request_all http_method status litellm_user litellm_team proxy_key_id request_id signal

CANDIDATE SEARCH 2:
LiteLLM runtime execution or secret-discovery behavior.

index=ENV_EDR_PROCESS_INDEX sourcetype=ENV_EDR_PROCESS_SOURCETYPE
| litellm_process_field_normalization
| litellm_host_scope
| eval process_name=coalesce(process_name,process,Image,process_exec)
| eval parent_process_name=coalesce(parent_process_name,parent_process,ParentImage,parent_name)
| eval command_line=coalesce(command_line,cmdline,process_command_line,CommandLine,"")
| eval process_user=coalesce(process_user,user,User)
| eval process_basename=lower(replace(process_name,"^.[\\/]",""))
| eval parent_process_basename=lower(replace(parent_process_name,"^.[\\/]",""))
| eval normalized_gateway=coalesce(litellm_gateway_id,host,dest_host,pod_name,container_id,workload_identity)
| where (
parent_process_basename IN ("litellm","python","python3","uvicorn","gunicorn")
OR like(command_line,"%litellm%")
OR like(command_line,"%proxy_server%")
OR like(command_line,"%litellm.proxy%")
OR process_user IN ("litellm","app","www-data","python")
)
| where (
process_basename IN ("sh","bash","dash","zsh","python","python3","perl","ruby","node","curl","wget","nc","ncat","socat","openssl","env","printenv","cat","grep","awk","sed","id","whoami","uname","tar","zip","7z")
OR match(command_line,"(?i)(\.env|/secrets/|/var/run/secrets/kubernetes.io/serviceaccount/|serviceaccount|OPENAI_API_KEY|ANTHROPIC_API_KEY|AZURE_OPENAI|AWS_ACCESS_KEY|GOOGLE_APPLICATION_CREDENTIALS|printenv|/bin/sh -c|bash -c|curl |wget |169\.254\.169\.254|metadata\.google\.internal|sts\.amazonaws\.com)")
)
| litellm_approved_command_suppression
| lookup APPROVED_LITELLM_DEPLOYMENT_USERS process_user OUTPUT process_user AS approved_deployment_user
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| where (isnull(approved_command) OR approved_command!="true")
AND isnull(approved_deployment_user)
AND (isnull(maintenance_window) OR maintenance_window!="true")
| eval signal="litellm_runtime_execution_or_secret_discovery"
| eval signal_time=_time
| table signal_time normalized_gateway host dest_host pod_name container_id namespace workload_identity process_user process_name parent_process_name command_line signal

CANDIDATE SEARCH 3:
LiteLLM secret, credential, configuration, or prompt/response file access.

index=ENV_EDR_FILE_INDEX sourcetype=ENV_EDR_FILE_SOURCETYPE
| litellm_file_field_normalization
| litellm_host_scope
| eval file_path=coalesce(file_path,path,target_file_path,file_name,"")
| eval process_user=coalesce(process_user,user,User)
| eval normalized_gateway=coalesce(litellm_gateway_id,host,dest_host,pod_name,container_id,workload_identity)
| where match(file_path,"(?i)(/\.env$|/secrets/|/var/run/secrets/kubernetes.io/serviceaccount/|litellm.*config|provider.*key|api.*key|database|credentials|prompt.*log|response.*log|request.*log)")
| search file_action IN ("created","modified","renamed","read","opened","write","written")
| lookup APPROVED_LITELLM_DEPLOYMENT_USERS process_user OUTPUT process_user AS approved_deployment_user
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| where isnull(approved_deployment_user)
AND (isnull(maintenance_window) OR maintenance_window!="true")
| eval signal="litellm_secret_or_config_access"
| eval signal_time=_time
| table signal_time normalized_gateway host dest_host pod_name container_id namespace workload_identity file_path file_action process_name process_user signal

CANDIDATE SEARCH 4:
LiteLLM provider-key usage anomaly.

index=ENV_PROVIDER_AUDIT_INDEX sourcetype IN (ENV_PROVIDER_AUDIT_SOURCETYPES)
| litellm_provider_audit_normalization
| lookup LITELLM_PROVIDER_KEYS provider_key_id OUTPUT litellm_gateway_id owner_application
| where isnotnull(litellm_gateway_id)
| eval normalized_gateway=coalesce(litellm_gateway_id,provider_key_id)
| lookup APPROVED_PROVIDER_KEY_SOURCES provider_key_id provider_source_ip OUTPUT approved_provider_source
| lookup APPROVED_PROVIDER_KEY_MODELS provider_key_id model_name OUTPUT approved_provider_model
| lookup APPROVED_PROVIDER_KEY_ACCOUNTS provider_key_id provider_account OUTPUT approved_provider_account
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| eval provider_anomaly=if(
isnull(approved_provider_source)
OR isnull(approved_provider_model)
OR isnull(approved_provider_account)
OR request_volume_ratio>ENV_PROVIDER_USAGE_RATIO_THRESHOLD
OR token_usage_ratio>ENV_PROVIDER_TOKEN_RATIO_THRESHOLD,
"true","false")
| where provider_anomaly="true"
AND (isnull(maintenance_window) OR maintenance_window!="true")
| eval signal="provider_key_usage_anomaly"
| eval signal_time=_time
| table signal_time normalized_gateway provider_key_id model_provider model_name provider_account provider_source_ip request_volume token_usage provider_anomaly signal

CANDIDATE SEARCH 5:
Rare LiteLLM gateway egress.

index=ENV_DNS_OR_PROXY_INDEX sourcetype IN (ENV_DNS_OR_PROXY_SOURCETYPES)
| litellm_egress_field_normalization
| litellm_egress_scope
| eval dest_domain=coalesce(dest_domain,query,domain,url_domain,dns_query)
| eval src_host=coalesce(src_host,host,src,dest_host)
| eval normalized_gateway=coalesce(litellm_gateway_id,src_host,host,pod_name,container_id,workload_identity)
| lookup APPROVED_LITELLM_EGRESS_DESTINATIONS dest_domain OUTPUT dest_domain AS approved_litellm_egress_destination
| lookup APPROVED_MODEL_PROVIDER_DOMAINS dest_domain OUTPUT dest_domain AS approved_model_provider_domain
| lookup APPROVED_BUSINESS_DOMAINS dest_domain OUTPUT dest_domain AS approved_business_domain
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| where isnotnull(dest_domain)
AND isnull(approved_litellm_egress_destination)
AND isnull(approved_model_provider_domain)
AND isnull(approved_business_domain)
AND (isnull(maintenance_window) OR maintenance_window!="true")
| eval signal="rare_litellm_gateway_egress"
| eval signal_time=_time
| table signal_time normalized_gateway src_host host pod_name container_id namespace workload_identity dest_domain dest_ip signal

FINAL CORRELATION SEARCH:
Run against materialized candidate output from the five staged searches.

index=ENV_LITELLM_DETECTION_SUMMARY_INDEX sourcetype=ENV_LITELLM_CANDIDATE_SIGNALS
| where signal IN (
"suspicious_litellm_mcp_test",
"litellm_runtime_execution_or_secret_discovery",
"litellm_secret_or_config_access",
"provider_key_usage_anomaly",
"rare_litellm_gateway_egress"
)
| eval evidence_time=signal_time
| eval mcp_time=if(signal="suspicious_litellm_mcp_test",signal_time,null())
| eventstats values(mcp_time) AS mcp_times BY normalized_gateway
| mvexpand mcp_times
| where isnotnull(mcp_times)
| where evidence_time>=mcp_times-300
AND evidence_time<=mcp_times+3600
| stats
values(signal) AS signals
values(src_ip) AS src_ips
values(x_forwarded_for) AS x_forwarded_for
values(user_agent) AS user_agents
values(uri_path) AS uri_paths
values(query_string) AS query_strings
values(request_all) AS request_urls
values(status) AS statuses
values(proxy_key_id) AS proxy_key_ids
values(litellm_user) AS litellm_users
values(litellm_team) AS litellm_teams
values(file_path) AS file_paths
values(file_action) AS file_actions
values(process_name) AS processes
values(parent_process_name) AS parent_processes
values(command_line) AS command_lines
values(dest_domain) AS dest_domains
values(dest_ip) AS dest_ips
values(provider_key_id) AS provider_key_ids
values(model_provider) AS model_providers
values(model_name) AS models
values(provider_account) AS provider_accounts
min(evidence_time) AS first_seen
max(evidence_time) AS last_seen
BY normalized_gateway mcp_times
| eval has_mcp=if(mvfind(signals,"suspicious_litellm_mcp_test")>=0,1,0)
| eval has_execution_or_exposure_evidence=if(
mvfind(signals,"litellm_runtime_execution_or_secret_discovery")>=0
OR mvfind(signals,"litellm_secret_or_config_access")>=0
OR mvfind(signals,"provider_key_usage_anomaly")>=0
OR mvfind(signals,"rare_litellm_gateway_egress")>=0,
1,0
)
| where has_mcp=1 AND has_execution_or_exposure_evidence=1
| lookup APPROVED_LITELLM_MAINTENANCE_WINDOWS normalized_gateway OUTPUT maintenance_window
| where isnull(maintenance_window) OR maintenance_window!="true"
| eval severity=case(
mvfind(signals,"provider_key_usage_anomaly")>=0 AND mvfind(signals,"litellm_secret_or_config_access")>=0,"critical",
mvfind(signals,"litellm_runtime_execution_or_secret_discovery")>=0 AND mvfind(signals,"provider_key_usage_anomaly")>=0,"critical",
mvfind(signals,"litellm_runtime_execution_or_secret_discovery")>=0,"high",
mvfind(signals,"litellm_secret_or_config_access")>=0,"high",
mvfind(signals,"provider_key_usage_anomaly")>=0,"high",
mvfind(signals,"rare_litellm_gateway_egress")>=0,"high",
true(),"medium"
)
| eval cyberdax_rule="LiteLLM MCP Test Endpoint Followed by Runtime Execution or Provider-Key Exposure Evidence"
| table first_seen last_seen normalized_gateway severity signals src_ips x_forwarded_for user_agents uri_paths query_strings request_urls statuses proxy_key_ids litellm_users litellm_teams file_paths file_actions processes parent_processes command_lines dest_domains dest_ips provider_key_ids model_providers models provider_accounts cyberdax_rule

Elastic

Detection Viability Assessment

Production-deployable after local ECS, value-list, exception-list, transform, and rule-type validation where Elastic has URI-preserving LiteLLM or reverse-proxy events, endpoint process events, endpoint file events, container events, DNS/proxy events, provider audit events, and enrichment linking LiteLLM gateways to host, pod, container, and workload identity. Exception logic must be implemented using Elastic exception lists, value lists, event filters, transforms, or equivalent local rule exceptions.

Rule

LiteLLM MCP Test to Runtime Execution or Secret Access Sequence

Rule Format

Elastic EQL production sequence with Elastic exception lists, value lists, transforms, and local ECS mapping.

Detection Purpose

Detect suspicious LiteLLM MCP test endpoint activity followed by runtime subprocess execution, secret access, provider-key discovery, rare outbound communication, or provider-key usage anomaly.

Detection Logic

Trigger when suspicious MCP endpoint activity is followed within 60 minutes by suspicious LiteLLM runtime process execution, secret-file access, provider-key material access, or rare outbound communication on the same LiteLLM host, container, pod, or mapped gateway identity.

Assign high severity when the sequence occurs on a LiteLLM gateway asset outside approved maintenance.

Promote to critical when the same host, pod, container, gateway, or provider key is tied to confirmed provider-key exposure, LiteLLM configuration changes, unauthorized provider usage, cloud secret access, Kubernetes service-account abuse, CI/CD token access, or prompt and response exposure.

Required Telemetry

Reverse-proxy events.

WAF events.

API gateway events.

LiteLLM access events.

Endpoint process events.

Endpoint file events.

Container runtime events.

DNS or proxy events.

Provider audit events.

LiteLLM gateway asset enrichment.

Elastic value list for LiteLLM gateways.

Elastic value list for approved LiteLLM administrator sources.

Elastic value list for approved AI platform testing sources.

Elastic value list for approved scanners.

Elastic value list for approved deployment users.

Elastic value list for approved egress destinations.

Elastic value list for approved model-provider domains.

Elastic exception list for approved maintenance windows.

Elastic exception list for approved command patterns.

Engineering Implementation Instructions

Map event.dataset, event.category, event.action, url.path, url.query, url.full, http.request.method, http.response.status_code, source.ip, user_agent.original, host.name, litellm.gateway.id, litellm.user.id, litellm.team.id, litellm.proxy_key.id, container.id, kubernetes.pod.name, kubernetes.namespace, kubernetes.node.name, process.name, process.command_line, process.parent.name, user.name, file.path, destination.domain, destination.ip, provider.key.id, model.provider, model.name, and @timestamp.

Create Elastic value lists for LiteLLM gateways, approved administrator CIDRs, approved AI platform testing sources, approved scanners, approved deployment users, approved egress destinations, approved model-provider domains, approved secret paths, approved command patterns, approved provider-key sources, approved workload identities, and approved LiteLLM maintenance sources.

Create Elastic exception lists for approved maintenance windows, approved AI platform testing, approved provider validation, approved deployment workflows, approved key rotation, approved vulnerability validation, approved debugging, approved incident-response cleanup, and approved command patterns.

Do not assume inline pattern exceptions such as not process.command_line in $APPROVED_LITELLM_COMMAND_PATTERNS will work reliably in EQL. Implement command suppression through exception lists, value lists, transforms, enrichment policies, event filters, or post-rule exception logic.

Use transforms or enrichment policies to attach LiteLLM gateway identity, host identity, container identity, pod identity, namespace, workload identity, provider-key ownership, approved-source context, and maintenance-window status before the EQL sequence executes.

Validate join keys between proxy/WAF events, endpoint process events, endpoint file events, container events, DNS/proxy events, provider audit events, cloud events, Kubernetes events, and LiteLLM asset enrichment before alert mode.

Validate EQL sequence behavior with representative historical data, including event ordering, timestamp consistency, delayed ingestion, cross-index field consistency, and maxspan behavior.

Do not use broad raw cross-index correlation without gateway, host, pod, container, workload identity, or provider-key join keys.

Run in hunt mode first to tune AI platform testing, provider validation, deployments, patching, key rotation, debugging, vulnerability validation, and emergency remediation activity.

Treat local ECS mapping, value-list creation, exception-list creation, transform configuration, enrichment policy validation, join-key validation, alert severity tuning, historical baselining, rule-action routing, and hunt-to-alert promotion as required local deployment work.

DRI Assessment

High where LiteLLM web, endpoint, file, container, provider, and network events share reliable gateway, host, pod, or container identity.

DRI

8.8 / 10

TCR Assessment

Strong when suspicious MCP endpoint activity is followed by LiteLLM runtime execution, secret access, or provider-key anomaly. Weaker where endpoint, container, or provider telemetry is absent.

Operational TCR

8.3 / 10

Full-Telemetry TCR

9.3 / 10

Limitations

EQL sequence quality depends on reliable join keys. URI visibility is required for strong request detection. Legitimate AI platform testing, provider validation, deployment, patching, key rotation, debugging, or emergency remediation activity must be exceptioned. Critical promotion requires provider, file, endpoint, container, cloud, Kubernetes, or LiteLLM administrative evidence.

Detection Query Pattern

Elastic EQL sequence rule requiring local ECS mapping, LiteLLM gateway enrichment, Elastic exception lists, value lists, event filters, and rule-action validation.

RULE TYPE:
Elastic EQL sequence rule.

LOCAL ENRICHMENT REQUIREMENT:
The field litellm.gateway.id must be populated before rule execution through ingest pipelines, transforms, enrichment policies, or asset-mapping jobs. This field should map each LiteLLM-related reverse-proxy, WAF, API gateway, LiteLLM access, process, file, container, DNS, proxy, or network event to the same normalized LiteLLM gateway identity.

DO NOT PLACE APPROVED-ACTIVITY LISTS DIRECTLY INSIDE THE EQL BODY:
The following controls must be implemented as Elastic rule exceptions, value lists, event filters, transforms, enrichment fields, or post-rule suppression logic:

APPROVED_LITELLM_ADMIN_SOURCES
APPROVED_AI_PLATFORM_TESTING_SOURCES
APPROVED_LITELLM_SCANNERS
APPROVED_LITELLM_DEPLOYMENT_USERS
APPROVED_LITELLM_RUNTIME_USERS
APPROVED_LITELLM_COMMAND_PATTERNS
APPROVED_LITELLM_EGRESS_DESTINATIONS
APPROVED_MODEL_PROVIDER_DOMAINS
APPROVED_LITELLM_MAINTENANCE_WINDOWS

EQL QUERY:
sequence by litellm.gateway.id with maxspan=60m

[any where
litellm.gateway.id != null and
event.dataset in (
"ENV_REVERSE_PROXY_DATASET",
"ENV_WAF_DATASET",
"ENV_API_GATEWAY_DATASET",
"ENV_LITELLM_ACCESS_DATASET"
) and
http.request.method == "POST" and
(
url.path : "/mcp-rest/test/connection" or
url.path : "/mcp-rest/test/tools/list" or
url.full : "/mcp-rest/test/connection" or
url.full : "/mcp-rest/test/tools/list" or
url.path : "mcptest*" or
url.full : "mcptest*"
)
]

[any where
litellm.gateway.id != null and
(
(
event.category == "process" and
(
process.parent.name in ("litellm", "python", "python3", "uvicorn", "gunicorn") or
process.parent.command_line : "litellm" or
process.parent.command_line : "proxy_server" or
process.parent.command_line : "litellm.proxy" or
process.command_line : "litellm" or
user.name in ("ENV_LITELLM_SERVICE_USER_1", "ENV_LITELLM_SERVICE_USER_2")
) and
(
process.name in (
"sh", "bash", "dash", "zsh", "python", "python3", "perl", "ruby", "node", "curl", "wget", "nc", "ncat", "socat", "openssl", "env", "printenv", "cat", "grep", "awk", "sed", "id", "whoami", "uname", "tar", "zip", "7z"
) or
process.command_line : ".env" or
process.command_line : "/secrets/" or
process.command_line : "/var/run/secrets/kubernetes.io/serviceaccount/" or
process.command_line : "serviceaccount" or
process.command_line : "OPENAI_API_KEY" or
process.command_line : "ANTHROPIC_API_KEY" or
process.command_line : "AZURE_OPENAI" or
process.command_line : "AWS_ACCESS_KEY" or
process.command_line : "GOOGLE_APPLICATION_CREDENTIALS" or
process.command_line : "printenv" or
process.command_line : "env " or
process.command_line : "bash -c" or
process.command_line : "/bin/sh -c" or
process.command_line : "curl " or
process.command_line : "wget " or
process.command_line : "169.254.169.254" or
process.command_line : "metadata.google.internal" or
process.command_line : "sts.amazonaws.com"
)
) or
(
event.category == "file" and
(
file.path : "/.env" or
file.path : "/.env." or
file.path : "/secrets/" or
file.path : "/var/run/secrets/kubernetes.io/serviceaccount/" or
file.path : "litellmconfig" or
file.path : "providerkey*" or
file.path : "apikey*" or
file.path : "database" or
file.path : "credentials" or
file.path : "promptlog*" or
file.path : "responselog*" or
file.path : "requestlog*"
) and
event.action in ("open", "read", "modification", "creation", "rename", "write")
) or
(
event.category == "network" and
destination.domain != null and
destination.domain != "" and
not destination.domain : ".openai.com" and
not destination.domain : ".anthropic.com" and
not destination.domain : ".googleapis.com" and
not destination.domain : ".google.com" and
not destination.domain : ".azure.com" and
not destination.domain : ".microsoft.com" and
not destination.domain : "*.amazonaws.com"
)
)
]

SEVERITY:
medium when suspicious MCP endpoint activity is followed by LiteLLM runtime process execution without secret access, provider-key indicators, rare egress, or downstream enrichment.

high when suspicious MCP endpoint activity is followed by suspicious command content, secret or configuration file access, or rare egress on the same litellm.gateway.id within 60 minutes.

critical when the same litellm.gateway.id is also enriched with confirmed provider-key exposure, provider-key usage anomaly, LiteLLM administrative change, cloud identity activity, Kubernetes identity activity, CI/CD token access, prompt or response exposure, or confirmed gateway configuration tampering within the local correlation layer.

REQUIRED ELASTIC RULE EXCEPTIONS:
Apply Elastic rule exceptions for approved LiteLLM deployments, approved AI platform testing, approved MCP validation, approved provider validation, approved key rotation, approved patch validation, approved vulnerability validation, approved emergency remediation, approved debugging, approved maintenance windows, approved command patterns, approved service users, approved egress destinations, and approved model-provider domains.

OUTPUT FIELDS:
@timestamp
event.dataset
event.category
event.action
litellm.gateway.id
host.name
container.id
kubernetes.pod.name
kubernetes.namespace
kubernetes.node.name
source.ip
user_agent.original
url.path
url.query
url.full
http.request.method
http.response.status_code
process.name
process.command_line
process.parent.name
process.parent.command_line
user.name
file.path
destination.domain
destination.ip
model.provider
model.name
provider.key.id
matched_signals

QRadar

Detection Viability Assessment

Production-deployable where QRadar parses LiteLLM, web, WAF, reverse-proxy, EDR, container, DNS/proxy, provider audit, cloud, Kubernetes, and LiteLLM administrative events into reliable custom properties. This should be implemented as building blocks with an offense rule. The AQL patterns below are validation searches for building-block logic, not standalone production alerts.

Rule

LiteLLM MCP Test Abuse With Runtime Execution or Provider-Key Exposure Offense

Rule Format

QRadar building-block and offense-rule implementation pattern with AQL validation searches.

Detection Purpose

Correlate suspicious LiteLLM MCP test endpoint activity with runtime subprocess execution, secret-file access, rare egress, provider-key usage anomalies, LiteLLM administrative changes, cloud activity, Kubernetes activity, or model-provider abuse.

Detection Logic

Trigger building block one when a LiteLLM gateway asset receives suspicious POST activity involving /mcp-rest/test/connection, /mcp-rest/test/tools/list, or confirmed MCP test endpoint variants.

Trigger building block two when endpoint, container, or host telemetry shows LiteLLM-associated subprocess execution, shell invocation, environment inspection, credential discovery, secret-file access, or suspicious command behavior.

Trigger building block three when DNS/proxy, provider, cloud, Kubernetes, or LiteLLM administrative telemetry shows rare egress, provider-key anomaly, secret-manager access, Kubernetes API activity, gateway configuration change, new proxy key, altered provider route, or unusual model-provider usage.

Create a high-severity offense when building block one and either building block two or building block three occur on the same LiteLLM gateway, virtual host, backend host, pod, container, workload identity, provider key, or mapped endpoint within 60 minutes.

Promote to critical when confirmed provider-key exposure, cloud secret access, Kubernetes token access, CI/CD token access, prompt or response exposure, unauthorized provider billing activity, malicious route change, or multiple-application impact occurs within 120 minutes.

Required Telemetry

LiteLLM access logs.

Web access logs.

WAF logs.

Reverse-proxy logs.

Endpoint process logs.

Endpoint file logs.

Container runtime logs where applicable.

DNS logs.

Proxy logs.

Provider audit or billing logs.

Cloud audit logs where applicable.

Kubernetes audit logs where applicable.

LiteLLM administrative or database events where available.

LiteLLM gateway reference set.

Approved administrator source reference set.

Approved AI platform testing source reference set.

Approved scanner reference set.

Approved maintenance-window reference set.

Approved egress-destination reference set.

Approved deployment-user reference set.

Approved provider-key source reference set.

Engineering Implementation Instructions

Create custom properties for LITELLM_GATEWAY, VIRTUALHOST, BACKENDHOST, PODNAME, CONTAINERID, NAMESPACE, NODENAME, URLPATH, QUERYSTRING, FULLURL, HTTPMETHOD, SOURCEIP, XFORWARDEDFOR, USERAGENT, HTTPSTATUS, PROXYKEYID, LITELLMUSER, LITELLMTEAM, FILEPATH, FILEACTION, PROCESSNAME, PARENTPROCESSNAME, COMMANDLINE, PROCESSUSER, DESTINATIONDOMAIN, DESTINATIONIP, PROVIDERKEYID, MODELPROVIDER, MODELNAME, CLOUDIDENTITY, KUBERNETESACTION, CLOUDACTION, REQUESTID, and WORKLOADIDENTITY.

Create reference sets for LiteLLM gateways, approved administrators, approved administrator sources, approved AI platform testing sources, approved scanners, approved maintenance windows, approved deployment users, approved command patterns, approved egress destinations, approved model-provider domains, approved provider-key sources, approved workload identities, approved provider validation workflows, and approved business destinations.

Validate DSM parsing for LiteLLM logs, WAF logs, reverse-proxy logs, API gateway logs, endpoint process logs, endpoint file logs, container runtime logs, DNS logs, proxy logs, provider audit logs, cloud audit logs, Kubernetes audit logs, and LiteLLM administrative records before enabling offense correlation.

Implement the detection as staged building blocks with bounded offense correlation. Do not implement it as a single broad raw AQL correlation across all event sources.

Test each building block independently with historical data before enabling the offense rule. Confirm that MCP endpoint activity, runtime process activity, secret-file access, rare egress, provider-key anomaly, cloud activity, Kubernetes activity, and LiteLLM administrative changes are parsed into the expected custom properties.

Validate correlation keys across LiteLLM gateway identity, virtual host, backend host, pod, container, workload identity, provider key, endpoint host, and source IP before assigning high or critical offense magnitude.

Tune reference sets for scanners, AI platform testing, provider validation, deployment workflows, maintenance windows, incident response activity, and approved egress before alert-mode deployment.

Keep AQL examples as validation searches for building blocks. Use QRadar CRE rules, building blocks, reference sets, reference maps, custom properties, offense correlation, and bounded time windows for production deployment.

Treat DSM parsing, custom property creation, reference-set loading, reference-map validation, building-block testing, offense magnitude tuning, ownership routing, suppression workflow, and SOC triage instructions as required local deployment work.

DRI Assessment

Moderate to high where QRadar custom properties and reference sets are reliable.

DRI

8.3 / 10

TCR Assessment

Strong when LiteLLM MCP endpoint activity, runtime process behavior, provider audit data, and endpoint or container behavior can be joined by gateway, backend host, pod, container, provider key, or workload identity. Lower when URI parsing, process telemetry, or provider-key attribution is incomplete.

Operational TCR

8.0 / 10

Full-Telemetry TCR

9.1 / 10

Limitations

QRadar effectiveness depends on DSM parsing quality, custom property accuracy, reference-set hygiene, and offense-rule correlation. Weak URI parsing or missing process/provider telemetry will materially reduce confidence. Shared provider keys and shared NAT egress may weaken attribution. Production confidence requires more than a single MCP endpoint event.

Detection Query Pattern

QRadar CRE building-block and offense-rule implementation pattern with AQL validation searches. The AQL below is for validating building-block logic and parsed custom properties, not for deployment as a single broad production alert.

IMPLEMENTATION MODEL:
Use QRadar custom properties, reference sets, reference maps, building blocks, and CRE offense correlation. Do not implement this detection as one monolithic AQL query across all event sources.

REQUIRED CUSTOM PROPERTIES:
LITELLM_GATEWAY
NORMALIZED_GATEWAY
VIRTUALHOST
BACKENDHOST
PODNAME
CONTAINERID
NAMESPACE
NODENAME
URLPATH
QUERYSTRING
FULLURL
HTTPMETHOD
SOURCEIP
XFORWARDEDFOR
USERAGENT
HTTPSTATUS
PROXYKEYID
LITELLMUSER
LITELLMTEAM
FILEPATH
FILEACTION
PROCESSNAME
PARENTPROCESSNAME
COMMANDLINE
PROCESSUSER
DESTINATIONDOMAIN
DESTINATIONIP
PROVIDERKEYID
PROVIDERACCOUNT
MODELPROVIDER
MODELNAME
CLOUDIDENTITY
KUBERNETESACTION
CLOUDACTION
REQUESTID
WORKLOADIDENTITY

REQUIRED REFERENCE SETS AND MAPS:
ENV_LITELLM_GATEWAYS
ENV_APPROVED_LITELLM_ADMIN_SOURCES
ENV_APPROVED_AI_PLATFORM_TESTING_SOURCES
ENV_APPROVED_LITELLM_SCANNERS
ENV_APPROVED_LITELLM_DEPLOYMENT_SOURCES
ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS
ENV_APPROVED_LITELLM_DEPLOYMENT_USERS
ENV_APPROVED_LITELLM_EGRESS_DESTINATIONS
ENV_APPROVED_MODEL_PROVIDER_DOMAINS
ENV_APPROVED_PROVIDER_KEY_SOURCES
ENV_APPROVED_PROVIDER_KEY_MODELS
ENV_APPROVED_PROVIDER_KEY_ACCOUNTS
ENV_LITELLM_PROVIDER_KEYS
ENV_LITELLM_GATEWAY_IDENTITY_MAP

BUILDING BLOCK ONE:
Suspicious LiteLLM MCP test endpoint activity.

Building block logic:

·        Event matches a LiteLLM gateway asset or normalized gateway.

·        HTTP method is POST.

·        URL path or full URL contains /mcp-rest/test/connection, /mcp-rest/test/tools/list, or a locally confirmed MCP test endpoint variant.

·        Source IP is not an approved LiteLLM administrator source, AI platform testing source, scanner, deployment source, or validation source.

·        Event is outside approved maintenance.

AQL validation search:

SELECT
QIDNAME(qid) AS event_name,
NORMALIZED_GATEWAY,
LITELLM_GATEWAY,
VIRTUALHOST,
BACKENDHOST,
PODNAME,
CONTAINERID,
NAMESPACE,
URLPATH,
QUERYSTRING,
FULLURL,
HTTPMETHOD,
SOURCEIP,
XFORWARDEDFOR,
USERAGENT,
HTTPSTATUS,
PROXYKEYID,
LITELLMUSER,
LITELLMTEAM,
REQUESTID,
starttime
FROM events
WHERE
REFERENCESETCONTAINS('ENV_LITELLM_GATEWAYS', NORMALIZED_GATEWAY)
AND HTTPMETHOD = 'POST'
AND (
LOWER(URLPATH) LIKE '%/mcp-rest/test/connection%'
OR LOWER(URLPATH) LIKE '%/mcp-rest/test/tools/list%'
OR LOWER(FULLURL) LIKE '%/mcp-rest/test/connection%'
OR LOWER(FULLURL) LIKE '%/mcp-rest/test/tools/list%'
OR LOWER(URLPATH) LIKE '%mcp%test%'
OR LOWER(FULLURL) LIKE '%mcp%test%'
)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_ADMIN_SOURCES', SOURCEIP)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_AI_PLATFORM_TESTING_SOURCES', SOURCEIP)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_SCANNERS', SOURCEIP)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_DEPLOYMENT_SOURCES', SOURCEIP)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS', NORMALIZED_GATEWAY)
LAST 60 MINUTES

BUILDING BLOCK TWO:
LiteLLM runtime execution, command content, or secret-discovery behavior.

Building block logic:

·        Event matches a LiteLLM gateway, host, pod, container, node, workload identity, or mapped normalized gateway.

·        Process telemetry shows LiteLLM-associated process lineage, LiteLLM service-user context, or LiteLLM runtime command context.

·        The process name or command line indicates shell execution, interpreter execution, transfer tooling, environment inspection, metadata access, mounted-secret access, provider-key discovery, or credential-file discovery.

·        Event is not from an approved deployment user or approved maintenance workflow.

AQL validation search:

SELECT
QIDNAME(qid) AS event_name,
NORMALIZED_GATEWAY,
LITELLM_GATEWAY,
BACKENDHOST,
PODNAME,
CONTAINERID,
NAMESPACE,
NODENAME,
PROCESSNAME,
PARENTPROCESSNAME,
COMMANDLINE,
PROCESSUSER,
FILEPATH,
FILEACTION,
starttime
FROM events
WHERE
REFERENCESETCONTAINS('ENV_LITELLM_GATEWAYS', NORMALIZED_GATEWAY)
AND (
LOWER(PARENTPROCESSNAME) IN ('litellm','python','python3','uvicorn','gunicorn')
OR LOWER(COMMANDLINE) LIKE '%litellm%'
OR LOWER(COMMANDLINE) LIKE '%proxy_server%'
OR LOWER(COMMANDLINE) LIKE '%litellm.proxy%'
OR LOWER(PROCESSUSER) IN ('litellm','app','www-data','python')
)
AND (
LOWER(PROCESSNAME) IN (
'sh','bash','dash','zsh','python','python3','perl','ruby','node','curl','wget','nc','ncat','socat','openssl','env','printenv','cat','grep','awk','sed','id','whoami','uname','tar','zip','7z'
)
OR LOWER(COMMANDLINE) LIKE '%.env%'
OR LOWER(COMMANDLINE) LIKE '%/secrets/%'
OR LOWER(COMMANDLINE) LIKE '%/var/run/secrets/kubernetes.io/serviceaccount/%'
OR LOWER(COMMANDLINE) LIKE '%serviceaccount%'
OR LOWER(COMMANDLINE) LIKE '%openai_api_key%'
OR LOWER(COMMANDLINE) LIKE '%anthropic_api_key%'
OR LOWER(COMMANDLINE) LIKE '%azure_openai%'
OR LOWER(COMMANDLINE) LIKE '%aws_access_key%'
OR LOWER(COMMANDLINE) LIKE '%google_application_credentials%'
OR LOWER(COMMANDLINE) LIKE '%printenv%'
OR LOWER(COMMANDLINE) LIKE '%/bin/sh -c%'
OR LOWER(COMMANDLINE) LIKE '%bash -c%'
OR LOWER(COMMANDLINE) LIKE '%curl %'
OR LOWER(COMMANDLINE) LIKE '%wget %'
OR LOWER(COMMANDLINE) LIKE '%169.254.169.254%'
OR LOWER(COMMANDLINE) LIKE '%metadata.google.internal%'
OR LOWER(COMMANDLINE) LIKE '%sts.amazonaws.com%'
OR LOWER(FILEPATH) LIKE '%.env'
OR LOWER(FILEPATH) LIKE '%/secrets/%'
OR LOWER(FILEPATH) LIKE '%/var/run/secrets/kubernetes.io/serviceaccount/%'
OR LOWER(FILEPATH) LIKE '%litellm%config%'
OR LOWER(FILEPATH) LIKE '%provider%key%'
OR LOWER(FILEPATH) LIKE '%api%key%'
OR LOWER(FILEPATH) LIKE '%database%'
OR LOWER(FILEPATH) LIKE '%credentials%'
)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_DEPLOYMENT_USERS', PROCESSUSER)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS', NORMALIZED_GATEWAY)
LAST 60 MINUTES

BUILDING BLOCK THREE:
Rare egress, provider-key anomaly, cloud activity, Kubernetes activity, or LiteLLM administrative change.

Building block logic:

·        Event matches a LiteLLM gateway, normalized gateway, provider key, workload identity, cloud identity, Kubernetes identity, or mapped asset.

·        Event shows rare egress, provider-key use outside approved source/model/account patterns, cloud secret access, Kubernetes secret access, LiteLLM administrative change, route change, key creation, or gateway configuration change.

·        Event is outside approved maintenance.

AQL validation search:

SELECT
QIDNAME(qid) AS event_name,
NORMALIZED_GATEWAY,
LITELLM_GATEWAY,
BACKENDHOST,
PODNAME,
CONTAINERID,
NAMESPACE,
SOURCEIP,
DESTINATIONDOMAIN,
DESTINATIONIP,
PROVIDERKEYID,
PROVIDERACCOUNT,
MODELPROVIDER,
MODELNAME,
CLOUDIDENTITY,
KUBERNETESACTION,
CLOUDACTION,
WORKLOADIDENTITY,
REQUESTID,
starttime
FROM events
WHERE
(
REFERENCESETCONTAINS('ENV_LITELLM_GATEWAYS', NORMALIZED_GATEWAY)
OR REFERENCESETCONTAINS('ENV_LITELLM_PROVIDER_KEYS', PROVIDERKEYID)
)
AND (
(
DESTINATIONDOMAIN IS NOT NULL
AND DESTINATIONDOMAIN <> ''
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_EGRESS_DESTINATIONS', DESTINATIONDOMAIN)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_MODEL_PROVIDER_DOMAINS', DESTINATIONDOMAIN)
)
OR (
PROVIDERKEYID IS NOT NULL
AND PROVIDERKEYID <> ''
AND (
NOT REFERENCESETCONTAINS('ENV_APPROVED_PROVIDER_KEY_SOURCES', SOURCEIP)
OR NOT REFERENCESETCONTAINS('ENV_APPROVED_PROVIDER_KEY_MODELS', MODELNAME)
OR NOT REFERENCESETCONTAINS('ENV_APPROVED_PROVIDER_KEY_ACCOUNTS', PROVIDERACCOUNT)
)
)
OR LOWER(CLOUDACTION) IN (
'getsecretvalue','getparameter','getparameters','accesssecretversion','getobject','putobject','setiampolicy','createaccesskey','assumerole','updatefunctioncode','updatestack','createstack','deletestack'
)
OR LOWER(KUBERNETESACTION) IN (
'get secrets','list secrets','create secret','patch deployment','update deployment','create pod','create serviceaccountbinding'
)
)
AND NOT REFERENCESETCONTAINS('ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS', NORMALIZED_GATEWAY)
LAST 60 MINUTES

OFFENSE RULE CONDITION:
Create a high-severity offense when Building Block One and either Building Block Two or Building Block Three occur within 60 minutes on the same NORMALIZED_GATEWAY, LITELLM_GATEWAY, VIRTUALHOST, BACKENDHOST, PODNAME, CONTAINERID, WORKLOADIDENTITY, or PROVIDERKEYID.

CRITICAL PROMOTION CONDITION:
Promote to critical when the high-severity offense is joined within 120 minutes to confirmed provider-key exposure, unauthorized provider use, cloud secret access, Kubernetes service-account abuse, CI/CD token access, prompt or response exposure, malicious LiteLLM configuration change, proxy-key creation, model-route change, multiple-application impact, or confirmed endpoint command execution.

OFFENSE OUTPUT:
NORMALIZED_GATEWAY
LITELLM_GATEWAY
VIRTUALHOST
BACKENDHOST
PODNAME
CONTAINERID
NAMESPACE
WORKLOADIDENTITY
PROVIDERKEYID
PROVIDERACCOUNT
SOURCEIP
XFORWARDEDFOR
USERAGENT
URLPATH
FULLURL
HTTPMETHOD
HTTPSTATUS
PROCESSNAME
PARENTPROCESSNAME
COMMANDLINE
PROCESSUSER
FILEPATH
FILEACTION
DESTINATIONDOMAIN
DESTINATIONIP
MODELPROVIDER
MODELNAME
CLOUDIDENTITY
KUBERNETESACTION
CLOUDACTION
REQUESTID
matched_building_blocks
first_seen
last_seen
offense_magnitude

SIGMA

Detection Viability Assessment

Production-deployable after conversion and local enrichment where the target SIEM can map LiteLLM gateway web, reverse-proxy, WAF, ingress, API gateway, load-balancer, or application access telemetry into SIGMA-compatible fields. SIGMA is appropriate here as a portable event-rule template for identifying LiteLLM MCP test endpoint activity. It should not be treated as full compromise detection by itself. Higher-confidence alerting requires the converted rule to be correlated in the target SIEM with LiteLLM runtime execution, secret or file access, provider-key anomaly, rare egress, cloud activity, Kubernetes activity, LiteLLM administrative changes, or confirmed gateway state changes.

Rule

Suspicious LiteLLM MCP Test Endpoint Activity

Rule Format

SIGMA portable webserver or gateway access event-rule template requiring target-SIEM conversion, LiteLLM gateway enrichment, local field mapping, approved-activity exceptions, and SIEM-native correlation.

Detection Purpose

Detect POST requests to LiteLLM MCP test endpoints or locally mapped MCP test-endpoint variants on LiteLLM gateway telemetry. The rule identifies an event-level candidate that should be correlated with runtime execution, file access, provider-key anomaly, rare egress, cloud activity, Kubernetes activity, or LiteLLM administrative change for higher-confidence alerting.

Detection Logic

Trigger when webserver, reverse-proxy, WAF, ingress, API gateway, load-balancer, or LiteLLM access telemetry records POST activity to /mcp-rest/test/connection, /mcp-rest/test/tools/list, or locally confirmed MCP test endpoint variants on a known LiteLLM gateway.

Assign medium severity for standalone converted rule matches because MCP test endpoint activity can occur during approved LiteLLM validation, AI platform testing, vulnerability validation, deployment checks, scanner activity, emergency remediation, or patch validation.

Promote to high severity in the target SIEM when the event is associated with an unusual source, non-administrative source, newly observed source, unexpected LiteLLM key context, abnormal response sequence, suspicious user agent, repeated failure-to-success pattern, rare egress, provider-key anomaly, or LiteLLM gateway route not normally used for MCP validation.

Promote to critical only when the same LiteLLM gateway, host, pod, container, workload identity, proxy key, source, or bounded time window is joined to endpoint-confirmed subprocess execution, secret-file access, provider-key exposure, LiteLLM administrative change, proxy-key creation, model-route modification, cloud identity activity, Kubernetes identity activity, CI/CD token access, prompt or response exposure, or confirmed gateway configuration tampering.

Required Telemetry

Webserver access telemetry.

Reverse-proxy telemetry.

WAF telemetry.

Ingress telemetry.

API gateway telemetry.

Load-balancer telemetry.

LiteLLM access telemetry where available.

HTTP method field.

URL path or full URL field.

Source IP field.

User agent field.

HTTP status field.

LiteLLM gateway identifier or gateway asset enrichment.

Container, pod, namespace, or workload identity enrichment where available.

Approved LiteLLM administrator source exceptions.

Approved AI platform testing exceptions.

Approved vulnerability validation exceptions.

Approved deployment-check exceptions.

Approved scanner exceptions.

Approved emergency remediation or patch-validation exceptions.

Engineering Implementation Instructions

Convert the SIGMA rule to the target SIEM before deployment and validate all backend-specific field mappings.

Map http.request.method, url.path, url.full, source.ip, user_agent.original, http.response.status_code, event.dataset, container.id, kubernetes.pod.name, kubernetes.namespace, normalized gateway identity, and LiteLLM gateway identifier to the target backend.

Add LiteLLM gateway enrichment after conversion. The rule should only alert on telemetry that can be mapped to known LiteLLM gateway assets, LiteLLM reverse-proxy routes, LiteLLM ingress routes, LiteLLM API gateway routes, or locally confirmed LiteLLM MCP endpoint variants.

Add approved-source and approved-workflow exception logic after conversion for approved LiteLLM administrators, approved MCP validation, approved AI platform testing, approved vulnerability validation, approved deployment checks, approved scanner activity, approved emergency remediation, and approved patch validation.

Do not add fake temporal joins, backend-specific correlation operators, reference-map syntax, cloud correlation syntax, process correlation syntax, provider-audit joins, or Kubernetes correlation syntax inside the SIGMA rule. SIGMA should remain an event-rule template.

Perform sequence correlation in the target SIEM after conversion if the organization wants high-confidence alerting from MCP endpoint activity to runtime execution, secret access, provider-key anomaly, rare egress, cloud activity, Kubernetes activity, or LiteLLM administrative change.

Validate YAML indentation, backend conversion output, field coverage, false-positive handling, severity handling, tag mapping, event routing, and alert ownership before alert-mode deployment.

Do not deploy the SIGMA event rule as high-confidence compromise detection by itself. Treat standalone matches as LiteLLM MCP endpoint activity indicators requiring correlation with LiteLLM web, endpoint, file, provider, cloud, Kubernetes, DNS, proxy, or administrative telemetry.

Treat target-SIEM conversion, field mapping, LiteLLM gateway enrichment, local exception logic, YAML validation, backend conversion testing, and correlation-layer promotion as required local deployment work.

DRI Assessment

Medium to high after conversion and LiteLLM gateway enrichment. Resilience improves when the target SIEM can reliably distinguish approved MCP validation, AI platform testing, scanner traffic, patch validation, and emergency remediation from unusual LiteLLM MCP endpoint activity.

DRI

8.0 / 10

TCR Assessment

Good for portable LiteLLM MCP endpoint candidate detection. Stronger when joined with LiteLLM runtime execution, file telemetry, provider audit logs, DNS, proxy, WAF, cloud, Kubernetes, or gateway administrative telemetry.

Operational TCR

7.8 / 10

Full-Telemetry TCR

8.9 / 10

Limitations

Legitimate LiteLLM administration, MCP validation, AI platform testing, vulnerability validation, scanner activity, deployment checks, emergency remediation, and patch validation may generate similar endpoint activity. SIGMA cannot express the full MCP endpoint to provider-key exposure or downstream cloud-impact sequence by itself. Host, provider, cloud, Kubernetes, and administrative correlation must occur in the target SIEM after conversion.

Detection Query Pattern

title: Suspicious LiteLLM MCP Test Endpoint Activity

id: 89e6b2d4-6d67-43f8-bd26-3f8c77885d41

status: test

description: Detects POST requests to LiteLLM MCP test endpoints or locally mapped MCP test-endpoint variants on LiteLLM gateway telemetry. This SIGMA rule produces an event-level candidate that must be correlated in the target SIEM with runtime execution, file access, provider-key anomaly, rare egress, cloud activity, Kubernetes activity, or LiteLLM administrative activity for higher-confidence alerting.

references:

  - https://github.com/BerriAI/litellm

author: CyberDax

date: 2026-06-17

tags:

  - attack.initial_access

  - attack.t1190

  - attack.execution

logsource:

  category: webserver

detection:

  selection_gateway:

    litellm_gateway_id|exists: true

  selection_method:

    http.request.method: POST

  selection_endpoint_path:

    url.path|contains:

      - /mcp-rest/test/connection

      - /mcp-rest/test/tools/list

  selection_endpoint_url:

    url.full|contains:

      - /mcp-rest/test/connection

      - /mcp-rest/test/tools/list

  selection_generic_mcp_test_path:

    url.path|contains|all:

      - mcp

      - test

  selection_generic_mcp_test_url:

    url.full|contains|all:

      - mcp

      - test

  condition: selection_gateway and selection_method and (selection_endpoint_path or selection_endpoint_url or selection_generic_mcp_test_path or selection_generic_mcp_test_url)

fields:

  - litellm_gateway_id

  - normalized_gateway

  - event.dataset

  - http.request.method

  - url.path

  - url.full

  - source.ip

  - user_agent.original

  - http.response.status_code

  - container.id

  - kubernetes.pod.name

  - kubernetes.namespace

falsepositives:

  - Approved LiteLLM MCP validation.

  - Approved AI platform testing.

  - Approved vulnerability validation.

  - Approved deployment checks.

  - Internal scanner activity.

  - Emergency remediation or patch validation.

level: medium

AWS

Detection Viability Assessment

Production-deployable only when LiteLLM is hosted on AWS and AWS telemetry can be joined with application-layer logs, WAF or ALB logs, endpoint or container telemetry, asset mapping, provider audit records, and workload identity. AWS control-plane logs alone are not sufficient to detect LiteLLM exploitation or command execution.

Rule

AWS Hosted LiteLLM MCP Test With Linked Egress Secret or Workload Activity

Rule Format

AWS Athena / CloudTrail / ALB / WAF / Route 53 / VPC Flow correlation pattern requiring LiteLLM asset and identity mapping.

Detection Purpose

Detect AWS-hosted LiteLLM MCP test endpoint activity that aligns with rare egress, endpoint-confirmed execution, provider-key anomaly, or suspicious AWS activity from a mapped LiteLLM workload.

Detection Logic

Trigger when an AWS-hosted LiteLLM asset receives suspicious MCP test endpoint activity and the same LiteLLM host private IP, EC2 instance, ECS task, EKS pod, target group, instance profile, task role, or LiteLLM-linked workload identity performs rare egress or high-risk AWS activity within 120 minutes.

Assign medium severity for rare egress from AWS-hosted LiteLLM workloads.

Assign high severity when rare egress or high-risk AWS activity aligns with suspicious MCP endpoint activity on a mapped LiteLLM asset.

Promote to critical when the LiteLLM-linked identity accesses Secrets Manager, Parameter Store, S3 prompt or response stores, model data, ECR images, CloudFormation, IAM, production storage, CI/CD secrets, or other sensitive services outside approved maintenance and with linkage to the affected LiteLLM host.

Required Telemetry

AWS asset inventory.

EC2 tags.

ECS or EKS workload labels where applicable.

ALB access logs.

CloudFront logs where applicable.

AWS WAF logs.

VPC Flow Logs.

Route 53 Resolver query logs.

CloudTrail.

EDR or container telemetry from LiteLLM hosts.

LiteLLM logs or reverse-proxy logs.

Model-provider audit logs where available.

Approved LiteLLM IAM role lookup.

Approved LiteLLM instance profile lookup.

Approved LiteLLM task role lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Approved model-provider domain lookup.

Approved LiteLLM asset lookup.

Engineering Implementation Instructions

Map LiteLLM assets to EC2 instance IDs, private IPs, ECS tasks, EKS pods, target groups, ALB targets, CloudFront distributions, WAF web ACLs, IAM roles, instance profiles, task roles, security groups, NAT gateways, Route 53 Resolver sources, hosted zones, provider keys, model-provider destinations, account IDs, and gateway identifiers.

Validate that ALB, CloudFront, WAF, API Gateway, reverse-proxy, or LiteLLM logs preserve URI paths before enabling high-severity logic.

Create local lookups for approved LiteLLM IAM roles, approved LiteLLM instance profiles, approved ECS task roles, approved EKS service accounts, approved maintenance windows, approved deployment actions, approved AI platform testing sources, approved egress destinations, approved model-provider domains, approved production accounts, approved LiteLLM assets, approved provider-key sources, and approved workload identities.

Join AWS telemetry to LiteLLM identity through asset mapping, private IP, target group, instance ID, ECS task, EKS pod, role ARN, instance profile, task role, service account, and NAT identity where available.

Do not deploy an AWS control-plane-only version of this rule. CloudTrail, VPC Flow Logs, Route 53 Resolver logs, or AWS identity activity cannot prove LiteLLM exploitation without application-layer, host, container, provider, workload identity, or time-window correlation.

Use the cloud query as a cloud-hosted correlation template, not as an independently sufficient compromise rule. High or critical severity requires linkage to LiteLLM web activity, endpoint or container execution, provider-key activity, secret access, workload identity activity, or confirmed gateway state change.

Validate CloudTrail identity resolution for assumed roles, task roles, instance profiles, IRSA, federated identities, and service-linked roles before alert-mode deployment.

Validate VPC Flow Log directionality, NAT attribution, target-group mapping, private-IP reuse, ECS task churn, EKS pod churn, log latency, and cross-account visibility before production use.

Run in hunt mode first to baseline deployments, patching, provider validation, model-provider egress, AI platform testing, autoscaling, image pulls, CloudFormation updates, secret reads, and approved incident-response activity.

Treat AWS asset tagging, log-source joins, CloudTrail identity mapping, URI-log validation, endpoint telemetry correlation, identity-to-host mapping, provider-key mapping, maintenance-window tuning, false-positive testing, query-performance validation, and SOC routing as required local deployment work.

DRI Assessment

Moderate to high with ALB/WAF URI logs, LiteLLM logs, endpoint telemetry, Route 53 logs, VPC Flow Logs, asset mapping, provider audit records, and CloudTrail identity mapping.

DRI

7.8 / 10

TCR Assessment

Moderate operational confidence for AWS-only egress. High confidence when LiteLLM web activity joins to endpoint execution, provider-key activity, sensitive AWS API use, or workload identity activity linked to the affected host.

Operational TCR

7.4 / 10

Full-Telemetry TCR

8.9 / 10

Limitations

VPC Flow Logs do not show URI paths or process context. CloudTrail does not show local LiteLLM request handling or subprocess execution. Production use requires application-layer and host or container telemetry; AWS control-plane, flow, DNS, or identity activity should not be treated as proof of LiteLLM compromise without web, host, workload, identity, file, provider, or time-window correlation. Critical severity requires LiteLLM-linked identity or host-to-identity mapping, not time proximity alone.

Detection Query Pattern

WITH approved_sources AS (

  SELECT approved_source_ip

  FROM ENV_APPROVED_LITELLM_ADMIN_SOURCES

  UNION

  SELECT approved_source_ip

  FROM ENV_APPROVED_AI_PLATFORM_TESTING_SOURCES

  UNION

  SELECT approved_source_ip

  FROM ENV_APPROVED_LITELLM_SCANNERS

),

approved_maintenance AS (

  SELECT normalized_gateway

  FROM ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS

),

mcp_endpoint_candidates AS (

  SELECT

    event_time AS mcp_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      virtual_host,

      backend_host,

      pod_name,

      container_id,

      workload_identity

    ) AS normalized_gateway,

    litellm_gateway_id,

    virtual_host,

    backend_host,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip,

    x_forwarded_for,

    user_agent,

    http_method,

    url_path,

    url_full,

    http_status,

    request_id

  FROM ENV_AWS_WEB_OR_GATEWAY_ACCESS_LOGS

  WHERE

    event_time >= current_timestamp - INTERVAL '60' MINUTE

    AND upper(http_method) = 'POST'

    AND (

      lower(url_path) LIKE '%/mcp-rest/test/connection%'

      OR lower(url_path) LIKE '%/mcp-rest/test/tools/list%'

      OR lower(url_full) LIKE '%/mcp-rest/test/connection%'

      OR lower(url_full) LIKE '%/mcp-rest/test/tools/list%'

      OR lower(url_path) LIKE '%mcp%test%'

      OR lower(url_full) LIKE '%mcp%test%'

    )

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      virtual_host,

      backend_host,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

mcp_endpoint_filtered AS (

  SELECT

    m.*

  FROM mcp_endpoint_candidates m

  LEFT JOIN approved_sources s

    ON m.source_ip = s.approved_source_ip

  LEFT JOIN approved_maintenance mw

    ON m.normalized_gateway = mw.normalized_gateway

  WHERE

    s.approved_source_ip IS NULL

    AND mw.normalized_gateway IS NULL

),

aws_secret_identity_activity AS (

  SELECT

    event_time AS aws_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      workload_identity

    ) AS normalized_gateway,

    litellm_gateway_id,

    workload_identity,

    source_ip,

    user_agent,

    event_source,

    event_name,

    user_identity_arn,

    user_identity_principal_id,

    request_parameters,

    response_elements,

    request_id

  FROM ENV_AWS_CLOUDTRAIL_NORMALIZED_EVENTS

  WHERE

    event_time >= current_timestamp - INTERVAL '120' MINUTE

    AND (

      (

        event_source IN (

          'secretsmanager.amazonaws.com',

          'ssm.amazonaws.com',

          'kms.amazonaws.com'

        )

        AND event_name IN (

          'GetSecretValue',

          'DescribeSecret',

          'GetParameter',

          'GetParameters',

          'GetParametersByPath',

          'Decrypt'

        )

      )

      OR (

        event_source = 'sts.amazonaws.com'

        AND event_name IN (

          'AssumeRole',

          'AssumeRoleWithWebIdentity',

          'GetCallerIdentity'

        )

      )

      OR (

        event_source = 'iam.amazonaws.com'

        AND event_name IN (

          'CreateAccessKey',

          'PutUserPolicy',

          'AttachUserPolicy',

          'UpdateAssumeRolePolicy'

        )

      )

      OR (

        event_source IN (

          'eks.amazonaws.com',

          'ecs.amazonaws.com',

          'lambda.amazonaws.com',

          'ecr.amazonaws.com',

          's3.amazonaws.com'

        )

        AND event_name IN (

          'DescribeCluster',

          'UpdateFunctionCode',

          'UpdateService',

          'RegisterTaskDefinition',

          'GetAuthorizationToken',

          'GetObject',

          'PutObject',

          'ListBucket'

        )

      )

    )

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      workload_identity

    ) IS NOT NULL

),

provider_key_activity AS (

  SELECT

    event_time AS provider_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) AS normalized_gateway,

    litellm_gateway_id,

    provider_key_id,

    provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id

  FROM ENV_AWS_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_EVENTS

  WHERE

    event_time >= current_timestamp - INTERVAL '120' MINUTE

    AND provider_key_id IS NOT NULL

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

provider_key_anomalies AS (

  SELECT

    p.*

  FROM provider_key_activity p

  LEFT JOIN ENV_APPROVED_PROVIDER_KEY_SOURCES ks

    ON p.provider_key_id = ks.provider_key_id

    AND p.source_ip = ks.approved_source_ip

  LEFT JOIN ENV_APPROVED_PROVIDER_KEY_MODELS km

    ON p.provider_key_id = km.provider_key_id

    AND p.model_name = km.approved_model_name

  LEFT JOIN ENV_APPROVED_PROVIDER_KEY_ACCOUNTS ka

    ON p.provider_key_id = ka.provider_key_id

    AND p.provider_account = ka.approved_provider_account

  WHERE

    ks.provider_key_id IS NULL

    OR km.provider_key_id IS NULL

    OR ka.provider_key_id IS NULL

),

egress_activity AS (

  SELECT

    event_time AS egress_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) AS normalized_gateway,

    litellm_gateway_id,

    CAST(NULL AS VARCHAR) AS provider_key_id,

    CAST(NULL AS VARCHAR) AS provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id

  FROM ENV_AWS_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_EVENTS

  WHERE

    event_time >= current_timestamp - INTERVAL '120' MINUTE

    AND destination_domain IS NOT NULL

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

egress_anomalies AS (

  SELECT

    e.*

  FROM egress_activity e

  LEFT JOIN ENV_APPROVED_LITELLM_EGRESS_DESTINATIONS ad

    ON e.destination_domain = ad.approved_destination_domain

  LEFT JOIN ENV_APPROVED_MODEL_PROVIDER_DOMAINS pd

    ON e.destination_domain = pd.approved_provider_domain

  WHERE

    ad.approved_destination_domain IS NULL

    AND pd.approved_provider_domain IS NULL

),

provider_or_egress_anomalies AS (

  SELECT

    provider_event_time AS event_time,

    normalized_gateway,

    litellm_gateway_id,

    provider_key_id,

    provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id,

    'provider_key_anomaly' AS anomaly_type

  FROM provider_key_anomalies

  UNION ALL

  SELECT

    egress_event_time AS event_time,

    normalized_gateway,

    litellm_gateway_id,

    provider_key_id,

    provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id,

    'rare_egress' AS anomaly_type

  FROM egress_anomalies

),

correlated_activity AS (

  SELECT

    m.mcp_time,

    COALESCE(a.aws_event_time, p.event_time) AS correlated_event_time,

    m.normalized_gateway,

    m.litellm_gateway_id,

    m.virtual_host,

    m.backend_host,

    m.pod_name,

    m.container_id,

    m.namespace,

    m.workload_identity,

    m.source_ip AS mcp_source_ip,

    m.x_forwarded_for,

    m.user_agent AS mcp_user_agent,

    m.http_method,

    m.url_path,

    m.url_full,

    m.http_status,

    COALESCE(a.source_ip, p.source_ip) AS downstream_source_ip,

    a.user_agent AS downstream_user_agent,

    COALESCE(a.event_source, p.event_source) AS downstream_event_source,

    COALESCE(a.event_name, p.event_name) AS downstream_event_name,

    a.user_identity_arn,

    a.user_identity_principal_id,

    p.provider_key_id,

    p.provider_account,

    p.model_provider,

    p.model_name,

    p.destination_domain,

    p.destination_ip,

    p.anomaly_type,

    CASE

      WHEN a.event_source IN (

        'secretsmanager.amazonaws.com',

        'ssm.amazonaws.com',

        'kms.amazonaws.com',

        'sts.amazonaws.com',

        'iam.amazonaws.com'

      )

      AND (

        a.normalized_gateway = m.normalized_gateway

        OR (

          m.litellm_gateway_id IS NOT NULL

          AND a.litellm_gateway_id = m.litellm_gateway_id

        )

        OR (

          m.workload_identity IS NOT NULL

          AND a.workload_identity = m.workload_identity

        )

      )

      THEN 'critical'

      WHEN p.anomaly_type = 'provider_key_anomaly'

      AND (

        p.normalized_gateway = m.normalized_gateway

        OR (

          m.litellm_gateway_id IS NOT NULL

          AND p.litellm_gateway_id = m.litellm_gateway_id

        )

        OR (

          m.workload_identity IS NOT NULL

          AND p.workload_identity = m.workload_identity

        )

      )

      THEN 'high'

      WHEN p.anomaly_type = 'rare_egress'

      AND (

        p.normalized_gateway = m.normalized_gateway

        OR (

          m.litellm_gateway_id IS NOT NULL

          AND p.litellm_gateway_id = m.litellm_gateway_id

        )

        OR (

          m.workload_identity IS NOT NULL

          AND p.workload_identity = m.workload_identity

        )

      )

      THEN 'high'

      ELSE 'medium'

    END AS severity,

    'LiteLLM MCP test endpoint activity followed by AWS secret, identity, provider-key, or egress evidence' AS detection_name

  FROM mcp_endpoint_filtered m

  LEFT JOIN aws_secret_identity_activity a

    ON (

      m.normalized_gateway = a.normalized_gateway

      OR (

        m.litellm_gateway_id IS NOT NULL

        AND m.litellm_gateway_id = a.litellm_gateway_id

      )

      OR (

        m.workload_identity IS NOT NULL

        AND m.workload_identity = a.workload_identity

      )

    )

    AND a.aws_event_time BETWEEN m.mcp_time AND m.mcp_time + INTERVAL '120' MINUTE

  LEFT JOIN provider_or_egress_anomalies p

    ON (

      m.normalized_gateway = p.normalized_gateway

      OR (

        m.litellm_gateway_id IS NOT NULL

        AND m.litellm_gateway_id = p.litellm_gateway_id

      )

      OR (

        m.workload_identity IS NOT NULL

        AND m.workload_identity = p.workload_identity

      )

    )

    AND p.event_time BETWEEN m.mcp_time AND m.mcp_time + INTERVAL '120' MINUTE

)

SELECT

  mcp_time,

  correlated_event_time,

  severity,

  detection_name,

  normalized_gateway,

  litellm_gateway_id,

  virtual_host,

  backend_host,

  pod_name,

  container_id,

  namespace,

  workload_identity,

  mcp_source_ip,

  x_forwarded_for,

  mcp_user_agent,

  http_method,

  url_path,

  url_full,

  http_status,

  downstream_source_ip,

  downstream_user_agent,

  downstream_event_source,

  downstream_event_name,

  user_identity_arn,

  user_identity_principal_id,

  provider_key_id,

  provider_account,

  model_provider,

  model_name,

  destination_domain,

  destination_ip,

  anomaly_type

FROM correlated_activity

WHERE

  correlated_event_time IS NOT NULL;

Azure

Detection Viability Assessment

Production-deployable only when LiteLLM is hosted on Azure and Azure telemetry can be joined with application-layer logs, WAF or Application Gateway logs, endpoint or container telemetry, identity logs, provider audit records, and workload mapping. Azure Activity Logs alone are not sufficient.

Rule

Azure Hosted LiteLLM MCP Test With Linked Egress Key Vault or Workload Activity

Rule Format

Azure Monitor / Log Analytics KQL correlation pattern requiring LiteLLM asset and identity mapping.

Detection Purpose

Detect Azure-hosted LiteLLM MCP test endpoint activity that aligns with rare egress or LiteLLM-linked identity activity involving Key Vault, storage, App Service, AKS, container registry, infrastructure modification, provider-key exposure, or production-impacting resources.

Detection Logic

Trigger when an Azure-hosted LiteLLM asset shows suspicious MCP endpoint activity and the same VM, workload, private IP, managed identity, service principal, AKS workload identity, or LiteLLM-linked identity shows rare egress or high-risk Azure activity within 120 minutes.

Assign medium severity for rare egress from Azure-hosted LiteLLM assets.

Assign high severity when rare egress or high-risk Azure activity aligns with suspicious MCP endpoint activity on a mapped LiteLLM asset.

Promote to critical when LiteLLM-linked identities access Key Vault secrets, alter production resources, modify web app settings, access storage backups, push container images, assign roles, access provider-key stores, or interact with AI service infrastructure outside approved maintenance.

Required Telemetry

Application Gateway access logs.

Azure WAF logs.

API Management logs where applicable.

Reverse-proxy logs where applicable.

LiteLLM access logs.

Defender for Endpoint telemetry from LiteLLM hosts.

Container or AKS telemetry where applicable.

NSG Flow Logs.

Azure DNS or proxy logs.

Azure Activity Logs.

Microsoft Entra ID logs.

Key Vault audit logs.

Storage audit logs where applicable.

Container Registry logs where applicable.

Azure OpenAI or model-provider audit logs where available.

Approved LiteLLM service principal lookup.

Approved LiteLLM managed identity lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Approved model-provider destination lookup.

Approved LiteLLM asset lookup.

Engineering Implementation Instructions

Map LiteLLM assets to Azure VMs, VMSS instances, App Service instances, AKS workloads, private IPs, Application Gateway backend pools, WAF policies, API Management routes, managed identities, service principals, Key Vault access policies, storage accounts, container registries, Azure OpenAI resources, model-provider destinations, subscriptions, resource groups, and deployment targets.

Validate URI visibility in Application Gateway, WAF, API Management, reverse-proxy, ingress, or LiteLLM logs before enabling high-severity logic.

Create local lookups for approved LiteLLM managed identities, approved service principals, approved AKS workload identities, approved maintenance windows, approved egress destinations, approved model-provider destinations, approved resource groups, approved subscriptions, approved Key Vault resources, approved storage accounts, approved container registries, approved deployment sources, approved LiteLLM assets, and approved provider-key sources.

Join Azure telemetry to LiteLLM identity through private IP, backend pool, hostname, VM, VMSS instance, App Service instance, AKS pod, namespace, managed identity, service principal, subscription, resource group, and workload identity where available.

Do not deploy an Azure Activity Log-only version of this rule. Azure Activity Logs, Entra ID logs, NSG Flow Logs, or Key Vault events cannot prove LiteLLM exploitation without application-layer, host, container, provider, workload identity, or time-window correlation.

Use the cloud query as a cloud-hosted correlation template, not as an independently sufficient compromise rule. High or critical severity requires linkage to LiteLLM web activity, endpoint or container execution, provider-key activity, secret access, workload identity activity, or confirmed gateway state change.

Validate KQL operator precedence with parentheses around fallback MCP matching logic, especially where or is used after URI path conditions.

Validate identity mapping for managed identities, service principals, AKS workload identity, federated credentials, App Service identities, and deployment identities before alert-mode deployment.

Validate NSG Flow Log directionality, private-IP reuse, backend-pool mapping, AKS pod churn, log latency, cross-subscription visibility, and Application Gateway field consistency before production use.

Run in hunt mode first to baseline deployments, patching, provider validation, AI platform testing, autoscaling, image pulls, Key Vault reads, storage access, App Service updates, AKS changes, and approved incident-response activity.

Treat Azure asset mapping, identity mapping, WAF/Application Gateway field validation, endpoint telemetry joins, provider-key mapping, maintenance-window tuning, identity-to-host mapping, KQL syntax validation, false-positive testing, query-performance validation, and SOC routing as required local deployment work.

DRI Assessment

Moderate with Azure-hosted LiteLLM asset mapping and application-layer telemetry. Low for Azure-native control-plane-only telemetry.

DRI

7.6 / 10

TCR Assessment

Moderate operational confidence when Azure logs show suspicious downstream activity. High confidence requires LiteLLM web, endpoint, provider, and identity correlation linked to the affected host or LiteLLM identity.

Operational TCR

7.3 / 10

Full-Telemetry TCR

8.8 / 10

Limitations

Azure Activity Logs do not show LiteLLM request handling, MCP endpoint behavior, or local subprocess execution. Application-layer and endpoint telemetry are required for production confidence; Azure control-plane, identity, network, or resource activity should not be treated as proof of LiteLLM compromise without web, host, workload, identity, file, provider, or time-window correlation. Critical severity requires LiteLLM-linked identity or host-to-identity correlation, not time proximity alone.

Detection Query Pattern

let lookback_mcp = 60m;

let lookback_correlation = 120m;

let approved_sources =

    union isfuzzy=true

        ENV_APPROVED_LITELLM_ADMIN_SOURCES,

        ENV_APPROVED_AI_PLATFORM_TESTING_SOURCES,

        ENV_APPROVED_LITELLM_SCANNERS

    | project approved_source_ip = tostring(approved_source_ip);

let approved_maintenance =

    ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS

    | project normalized_gateway = tostring(normalized_gateway);

let approved_provider_sources =

    ENV_APPROVED_PROVIDER_KEY_SOURCES

    | project provider_source_key = strcat(tostring(provider_key_id), "|", tostring(approved_source_ip)), source_approved = true;

let approved_provider_models =

    ENV_APPROVED_PROVIDER_KEY_MODELS

    | project provider_model_key = strcat(tostring(provider_key_id), "|", tostring(approved_model_name)), model_approved = true;

let approved_provider_accounts =

    ENV_APPROVED_PROVIDER_KEY_ACCOUNTS

    | project provider_account_key = strcat(tostring(provider_key_id), "|", tostring(approved_provider_account)), account_approved = true;

let mcp_endpoint_candidates =

    ENV_AZURE_WEB_OR_GATEWAY_ACCESS_LOGS

    | where TimeGenerated >= ago(lookback_mcp)

    | extend http_method = toupper(tostring(http_method))

    | extend normalized_gateway = coalesce(

        tostring(litellm_gateway_id),

        tostring(normalized_gateway),

        tostring(host_name),

        tostring(backend_host),

        tostring(pod_name),

        tostring(container_id),

        tostring(workload_identity)

      )

    | where isnotempty(normalized_gateway)

    | where http_method == "POST"

    | where tostring(url_path) has "/mcp-rest/test/connection"

        or tostring(url_path) has "/mcp-rest/test/tools/list"

        or tostring(url_full) has "/mcp-rest/test/connection"

        or tostring(url_full) has "/mcp-rest/test/tools/list"

        or (tostring(url_path) has "mcp" and tostring(url_path) has "test")

        or (tostring(url_full) has "mcp" and tostring(url_full) has "test")

    | project

        mcp_time = TimeGenerated,

        normalized_gateway,

        litellm_gateway_id = tostring(litellm_gateway_id),

        host_name = tostring(host_name),

        backend_host = tostring(backend_host),

        pod_name = tostring(pod_name),

        container_id = tostring(container_id),

        namespace = tostring(namespace),

        workload_identity = tostring(workload_identity),

        mcp_source_ip = tostring(source_ip),

        x_forwarded_for = tostring(x_forwarded_for),

        mcp_user_agent = tostring(user_agent),

        http_method,

        url_path = tostring(url_path),

        url_full = tostring(url_full),

        http_status = tostring(http_status),

        request_id = tostring(request_id);

let mcp_endpoint_filtered =

    mcp_endpoint_candidates

    | join kind=leftanti approved_sources on $left.mcp_source_ip == $right.approved_source_ip

    | join kind=leftanti approved_maintenance on normalized_gateway;

let mcp_correlation_keys =

    mcp_endpoint_filtered

    | extend correlation_keys = pack_array(normalized_gateway, litellm_gateway_id, workload_identity)

    | mv-expand correlation_key = correlation_keys to typeof(string)

    | where isnotempty(correlation_key);

let azure_secret_identity_activity =

    ENV_AZURE_ACTIVITY_OR_SIGNIN_NORMALIZED_LOGS

    | where TimeGenerated >= ago(lookback_correlation)

    | extend azure_normalized_gateway = coalesce(

        tostring(litellm_gateway_id),

        tostring(normalized_gateway),

        tostring(workload_identity)

      )

    | where isnotempty(azure_normalized_gateway)

    | where

        (

            ResourceProviderValue in~ (

                "MICROSOFT.KEYVAULT",

                "MICROSOFT.MANAGEDIDENTITY",

                "MICROSOFT.AUTHORIZATION",

                "MICROSOFT.CONTAINERSERVICE",

                "MICROSOFT.WEB",

                "MICROSOFT.CONTAINERREGISTRY",

                "MICROSOFT.STORAGE"

            )

            and OperationNameValue in~ (

                "MICROSOFT.KEYVAULT/VAULTS/SECRETS/READ",

                "MICROSOFT.KEYVAULT/VAULTS/KEYS/READ",

                "MICROSOFT.KEYVAULT/VAULTS/DECRYPT/ACTION",

                "MICROSOFT.MANAGEDIDENTITY/USERASSIGNEDIDENTITIES/ASSIGN/ACTION",

                "MICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE",

                "MICROSOFT.AUTHORIZATION/ROLEDEFINITIONS/WRITE",

                "MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/LISTCLUSTERUSERCREDENTIAL/ACTION",

                "MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/LISTCLUSTERADMINCREDENTIAL/ACTION",

                "MICROSOFT.WEB/SITES/CONFIG/WRITE",

                "MICROSOFT.WEB/SITES/SOURCECONTROLS/WRITE",

                "MICROSOFT.CONTAINERREGISTRY/REGISTRIES/PULL/READ",

                "MICROSOFT.STORAGE/STORAGEACCOUNTS/LISTKEYS/ACTION"

            )

        )

        or (

            EventSource =~ "SigninLogs"

            and (

                tostring(ResultType) == "0"

                or tostring(ResultType) == "success"

            )

            and (

                tostring(ServicePrincipalId) != ""

                or tostring(AppId) != ""

                or tostring(ManagedIdentityId) != ""

            )

        )

    | project

        downstream_event_time = TimeGenerated,

        azure_normalized_gateway,

        azure_litellm_gateway_id = tostring(litellm_gateway_id),

        azure_workload_identity = tostring(workload_identity),

        downstream_source_ip = tostring(IPAddress),

        downstream_user_agent = tostring(UserAgent),

        resource_provider = tostring(ResourceProviderValue),

        operation_name = tostring(OperationNameValue),

        caller = tostring(Caller),

        identity = tostring(Identity),

        service_principal_id = tostring(ServicePrincipalId),

        app_id = tostring(AppId),

        managed_identity_id = tostring(ManagedIdentityId),

        result_type = tostring(ResultType),

        result_description = tostring(ResultDescription),

        correlation_id = tostring(CorrelationId),

        anomaly_type = "azure_secret_or_identity_activity";

let azure_secret_identity_keys =

    azure_secret_identity_activity

    | extend correlation_keys = pack_array(azure_normalized_gateway, azure_litellm_gateway_id, azure_workload_identity)

    | mv-expand correlation_key = correlation_keys to typeof(string)

    | where isnotempty(correlation_key);

let provider_key_activity =

    ENV_AZURE_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_LOGS

    | where TimeGenerated >= ago(lookback_correlation)

    | extend provider_normalized_gateway = coalesce(

        tostring(litellm_gateway_id),

        tostring(normalized_gateway),

        tostring(provider_key_id),

        tostring(pod_name),

        tostring(container_id),

        tostring(workload_identity)

      )

    | where isnotempty(provider_normalized_gateway)

    | where isnotempty(provider_key_id)

    | project

        provider_event_time = TimeGenerated,

        provider_normalized_gateway,

        provider_litellm_gateway_id = tostring(litellm_gateway_id),

        provider_key_id = tostring(provider_key_id),

        provider_account = tostring(provider_account),

        model_provider = tostring(model_provider),

        model_name = tostring(model_name),

        provider_pod_name = tostring(pod_name),

        provider_container_id = tostring(container_id),

        provider_namespace = tostring(namespace),

        provider_workload_identity = tostring(workload_identity),

        provider_source_ip = tostring(source_ip),

        destination_ip = tostring(destination_ip),

        destination_domain = tostring(destination_domain),

        provider_event_source = tostring(event_source),

        provider_event_name = tostring(event_name),

        provider_request_id = tostring(request_id),

        provider_source_key = strcat(tostring(provider_key_id), "|", tostring(source_ip)),

        provider_model_key = strcat(tostring(provider_key_id), "|", tostring(model_name)),

        provider_account_key = strcat(tostring(provider_key_id), "|", tostring(provider_account));

let provider_key_anomalies =

    provider_key_activity

    | join kind=leftouter approved_provider_sources on provider_source_key

    | join kind=leftouter approved_provider_models on provider_model_key

    | join kind=leftouter approved_provider_accounts on provider_account_key

    | extend source_approved = coalesce(source_approved, false)

    | extend model_approved = coalesce(model_approved, false)

    | extend account_approved = coalesce(account_approved, false)

    | where source_approved == false

        or model_approved == false

        or account_approved == false

    | extend anomaly_type = "provider_key_anomaly"

    | project

        anomaly_event_time = provider_event_time,

        anomaly_normalized_gateway = provider_normalized_gateway,

        anomaly_litellm_gateway_id = provider_litellm_gateway_id,

        anomaly_workload_identity = provider_workload_identity,

        provider_key_id,

        provider_account,

        model_provider,

        model_name,

        pod_name = provider_pod_name,

        container_id = provider_container_id,

        namespace = provider_namespace,

        downstream_source_ip = provider_source_ip,

        downstream_user_agent = "",

        destination_ip,

        destination_domain,

        event_source = provider_event_source,

        event_name = provider_event_name,

        request_id = provider_request_id,

        anomaly_type;

let egress_activity =

    ENV_AZURE_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_LOGS

    | where TimeGenerated >= ago(lookback_correlation)

    | extend egress_normalized_gateway = coalesce(

        tostring(litellm_gateway_id),

        tostring(normalized_gateway),

        tostring(pod_name),

        tostring(container_id),

        tostring(workload_identity)

      )

    | where isnotempty(egress_normalized_gateway)

    | where isnotempty(destination_domain)

    | project

        egress_event_time = TimeGenerated,

        egress_normalized_gateway,

        egress_litellm_gateway_id = tostring(litellm_gateway_id),

        egress_workload_identity = tostring(workload_identity),

        provider_key_id = "",

        provider_account = "",

        model_provider = tostring(model_provider),

        model_name = tostring(model_name),

        pod_name = tostring(pod_name),

        container_id = tostring(container_id),

        namespace = tostring(namespace),

        downstream_source_ip = tostring(source_ip),

        downstream_user_agent = "",

        destination_ip = tostring(destination_ip),

        destination_domain = tostring(destination_domain),

        event_source = tostring(event_source),

        event_name = tostring(event_name),

        request_id = tostring(request_id);

let egress_anomalies =

    egress_activity

    | join kind=leftanti ENV_APPROVED_LITELLM_EGRESS_DESTINATIONS on $left.destination_domain == $right.approved_destination_domain

    | join kind=leftanti ENV_APPROVED_MODEL_PROVIDER_DOMAINS on $left.destination_domain == $right.approved_provider_domain

    | extend anomaly_type = "rare_egress"

    | project

        anomaly_event_time = egress_event_time,

        anomaly_normalized_gateway = egress_normalized_gateway,

        anomaly_litellm_gateway_id = egress_litellm_gateway_id,

        anomaly_workload_identity = egress_workload_identity,

        provider_key_id,

        provider_account,

        model_provider,

        model_name,

        pod_name,

        container_id,

        namespace,

        downstream_source_ip,

        downstream_user_agent,

        destination_ip,

        destination_domain,

        event_source,

        event_name,

        request_id,

        anomaly_type;

let provider_or_egress_anomalies =

    union provider_key_anomalies, egress_anomalies;

let provider_or_egress_keys =

    provider_or_egress_anomalies

    | extend correlation_keys = pack_array(anomaly_normalized_gateway, anomaly_litellm_gateway_id, anomaly_workload_identity)

    | mv-expand correlation_key = correlation_keys to typeof(string)

    | where isnotempty(correlation_key);

let azure_correlated =

    mcp_correlation_keys

    | join kind=inner azure_secret_identity_keys on correlation_key

    | where downstream_event_time between (mcp_time .. mcp_time + lookback_correlation)

    | extend severity_rank = case(

        resource_provider in~ (

            "MICROSOFT.KEYVAULT",

            "MICROSOFT.MANAGEDIDENTITY",

            "MICROSOFT.AUTHORIZATION"

        ), 3,

        2

      )

    | project

        mcp_time,

        correlated_event_time = downstream_event_time,

        severity_rank,

        detection_name = "LiteLLM MCP test endpoint activity followed by Azure secret or identity evidence",

        normalized_gateway,

        litellm_gateway_id,

        host_name,

        backend_host,

        pod_name,

        container_id,

        namespace,

        workload_identity,

        mcp_source_ip,

        x_forwarded_for,

        mcp_user_agent,

        http_method,

        url_path,

        url_full,

        http_status,

        downstream_source_ip,

        downstream_user_agent,

        resource_provider,

        operation_name,

        caller,

        identity,

        service_principal_id,

        app_id,

        managed_identity_id,

        result_type,

        result_description,

        provider_key_id = "",

        provider_account = "",

        model_provider = "",

        model_name = "",

        destination_domain = "",

        destination_ip = "",

        anomaly_type,

        request_id;

let provider_egress_correlated =

    mcp_correlation_keys

    | join kind=inner provider_or_egress_keys on correlation_key

    | where anomaly_event_time between (mcp_time .. mcp_time + lookback_correlation)

    | extend severity_rank = 2

    | project

        mcp_time,

        correlated_event_time = anomaly_event_time,

        severity_rank,

        detection_name = "LiteLLM MCP test endpoint activity followed by provider-key or rare-egress evidence",

        normalized_gateway,

        litellm_gateway_id,

        host_name,

        backend_host,

        pod_name,

        container_id,

        namespace,

        workload_identity,

        mcp_source_ip,

        x_forwarded_for,

        mcp_user_agent,

        http_method,

        url_path,

        url_full,

        http_status,

        downstream_source_ip,

        downstream_user_agent,

        resource_provider = "",

        operation_name = "",

        caller = "",

        identity = "",

        service_principal_id = "",

        app_id = "",

        managed_identity_id = "",

        result_type = "",

        result_description = "",

        provider_key_id,

        provider_account,

        model_provider,

        model_name,

        destination_domain,

        destination_ip,

        anomaly_type,

        request_id;

union azure_correlated, provider_egress_correlated

| summarize

    correlated_event_time = min(correlated_event_time),

    severity_rank = max(severity_rank),

    detection_name = any(detection_name),

    host_name = any(host_name),

    backend_host = any(backend_host),

    pod_name = any(pod_name),

    container_id = any(container_id),

    namespace = any(namespace),

    workload_identity = any(workload_identity),

    mcp_source_ip = any(mcp_source_ip),

    x_forwarded_for = any(x_forwarded_for),

    mcp_user_agent = any(mcp_user_agent),

    http_method = any(http_method),

    url_path = any(url_path),

    url_full = any(url_full),

    http_status = any(http_status),

    downstream_source_ip = any(downstream_source_ip),

    downstream_user_agent = any(downstream_user_agent),

    resource_provider = any(resource_provider),

    operation_name = any(operation_name),

    caller = any(caller),

    identity = any(identity),

    service_principal_id = any(service_principal_id),

    app_id = any(app_id),

    managed_identity_id = any(managed_identity_id),

    result_type = any(result_type),

    result_description = any(result_description),

    provider_key_id = any(provider_key_id),

    provider_account = any(provider_account),

    model_provider = any(model_provider),

    model_name = any(model_name),

    destination_domain = any(destination_domain),

    destination_ip = any(destination_ip),

    anomaly_type = make_set(anomaly_type),

    request_id = any(request_id)

  by mcp_time, normalized_gateway, litellm_gateway_id

| extend severity = case(

    severity_rank == 3, "critical",

    severity_rank == 2, "high",

    "medium"

  )

| project

    mcp_time,

    correlated_event_time,

    severity,

    detection_name,

    normalized_gateway,

    litellm_gateway_id,

    host_name,

    backend_host,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    mcp_source_ip,

    x_forwarded_for,

    mcp_user_agent,

    http_method,

    url_path,

    url_full,

    http_status,

    downstream_source_ip,

    downstream_user_agent,

    resource_provider,

    operation_name,

    caller,

    identity,

    service_principal_id,

    app_id,

    managed_identity_id,

    result_type,

    result_description,

    provider_key_id,

    provider_account,

    model_provider,

    model_name,

    destination_domain,

    destination_ip,

    anomaly_type,

    request_id

GCP

Detection Viability Assessment

Production-deployable only when LiteLLM is hosted on GCP and GCP telemetry can be joined with application-layer logs, endpoint or container telemetry, workload identity, provider audit records, and downstream cloud activity. GCP Audit Logs alone are not sufficient.

Rule

GCP Hosted LiteLLM MCP Test With Linked Service Account Secret Artifact or Egress Activity

Rule Format

BigQuery / Cloud Logging correlation pattern requiring LiteLLM asset and identity mapping.

Detection Purpose

Detect GCP-hosted LiteLLM MCP test endpoint activity that aligns with rare egress or LiteLLM-linked service-account activity involving Secret Manager, Artifact Registry, Cloud Storage, GKE, infrastructure modification, provider-key exposure, or production resources.

Detection Logic

Trigger when a GCP-hosted LiteLLM asset shows suspicious MCP endpoint activity and the same VM, GKE workload, private IP, service account, or LiteLLM-linked identity shows rare egress or high-risk GCP activity within 120 minutes.

Assign medium severity for rare egress from GCP-hosted LiteLLM assets.

Assign high severity when rare egress or high-risk GCP activity aligns with suspicious MCP endpoint activity on a mapped LiteLLM asset.

Promote to critical when LiteLLM-linked service accounts access secrets, alter storage backups, push artifacts, modify GKE workloads, modify production infrastructure, assign IAM policy, access provider-key stores, or alter production resources outside approved maintenance.

Required Telemetry

Cloud Load Balancing logs.

Cloud Armor logs where applicable.

API gateway or ingress logs where applicable.

LiteLLM logs or reverse-proxy logs.

Compute Engine or GKE asset mapping.

Endpoint or container telemetry from LiteLLM workloads.

VPC Flow Logs.

Cloud DNS logs.

Google Cloud Audit Logs.

Secret Manager audit logs.

GKE audit logs where applicable.

Cloud Storage audit logs where applicable.

Artifact Registry logs where applicable.

Vertex AI or model-provider audit logs where applicable.

Approved LiteLLM service-account lookup.

Approved maintenance-window lookup.

Approved egress-destination lookup.

Approved model-provider destination lookup.

Approved LiteLLM asset lookup.

Engineering Implementation Instructions

Map LiteLLM assets to Compute Engine instances, GKE pods, namespaces, service accounts, private IPs, load-balancer backends, Cloud Armor policies, Artifact Registry repositories, Cloud Storage buckets, Secret Manager secrets, deployment targets, production projects, provider keys, model-provider destinations, and gateway identifiers.

Validate URI visibility through Cloud Load Balancing, Cloud Armor, ingress, reverse proxy, API gateway, or LiteLLM logs before enabling high-severity logic.

Create local lookups for approved LiteLLM service accounts, approved workload identities, approved maintenance windows, approved egress destinations, approved model-provider destinations, approved projects, approved clusters, approved namespaces, approved secrets, approved registries, approved storage buckets, approved LiteLLM assets, approved deployment sources, and approved provider-key sources.

Join GCP telemetry to LiteLLM identity through backend service name, private IP, instance ID, GKE pod, namespace, node, service account, workload identity, project ID, load-balancer backend, and provider-key mapping where available.

Do not deploy a GCP Audit Log-only version of this rule. GCP Audit Logs, VPC Flow Logs, Cloud DNS logs, or service-account activity cannot prove LiteLLM exploitation without application-layer, host, container, provider, workload identity, or time-window correlation.

Use the cloud query as a cloud-hosted correlation template, not as an independently sufficient compromise rule. High or critical severity requires linkage to LiteLLM web activity, endpoint or container execution, provider-key activity, secret access, workload identity activity, or confirmed gateway state change.

Validate service-account mapping for Compute Engine default service accounts, custom service accounts, GKE Workload Identity, federated identities, deployment identities, and CI/CD identities before alert-mode deployment.

Validate VPC Flow Log directionality, private-IP reuse, load-balancer backend mapping, GKE pod churn, service-account reuse, log latency, cross-project visibility, and Cloud Load Balancing field consistency before production use.

Run in hunt mode first to baseline deployments, patching, provider validation, AI platform testing, autoscaling, image pulls, Secret Manager reads, Cloud Storage access, Artifact Registry writes, GKE changes, and approved incident-response activity.

Treat GCP asset mapping, workload identity mapping, URI-log validation, endpoint telemetry joins, provider-key mapping, maintenance-window tuning, identity-to-host mapping, BigQuery syntax validation, false-positive testing, query-performance validation, and SOC routing as required local deployment work.

DRI Assessment

Moderate with GCP-hosted LiteLLM asset mapping and application-layer telemetry. Low for GCP-native control-plane-only telemetry.

DRI

7.5 / 10

TCR Assessment

Moderate operational confidence when GCP logs show suspicious downstream activity. High confidence requires LiteLLM web, endpoint, provider, and identity correlation linked to the affected host or LiteLLM service account.

Operational TCR

7.2 / 10

Full-Telemetry TCR

8.7 / 10

Limitations

GCP Audit Logs do not show LiteLLM request handling, MCP endpoint behavior, or local subprocess execution. Application-layer and endpoint telemetry are required for production confidence; GCP audit, network, DNS, provider, or service-account activity should not be treated as proof of LiteLLM compromise without web, host, workload, identity, file, provider, or time-window correlation. Critical severity requires LiteLLM-linked service account or host-to-identity correlation, not time proximity alone.

Detection Query Pattern

WITH approved_sources AS (

  SELECT approved_source_ip

  FROM ENV_APPROVED_LITELLM_ADMIN_SOURCES

  UNION DISTINCT

  SELECT approved_source_ip

  FROM ENV_APPROVED_AI_PLATFORM_TESTING_SOURCES

  UNION DISTINCT

  SELECT approved_source_ip

  FROM ENV_APPROVED_LITELLM_SCANNERS

),

approved_maintenance AS (

  SELECT normalized_gateway

  FROM ENV_APPROVED_LITELLM_MAINTENANCE_WINDOWS

),

approved_provider_sources AS (

  SELECT

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(approved_source_ip AS STRING)) AS provider_source_key,

    TRUE AS source_approved

  FROM ENV_APPROVED_PROVIDER_KEY_SOURCES

),

approved_provider_models AS (

  SELECT

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(approved_model_name AS STRING)) AS provider_model_key,

    TRUE AS model_approved

  FROM ENV_APPROVED_PROVIDER_KEY_MODELS

),

approved_provider_accounts AS (

  SELECT

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(approved_provider_account AS STRING)) AS provider_account_key,

    TRUE AS account_approved

  FROM ENV_APPROVED_PROVIDER_KEY_ACCOUNTS

),

mcp_endpoint_candidates AS (

  SELECT

    event_timestamp AS mcp_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      host_name,

      backend_host,

      pod_name,

      container_id,

      workload_identity

    ) AS normalized_gateway,

    litellm_gateway_id,

    host_name,

    backend_host,

    pod_name,

    container_id,

    namespace,

    workload_identity,

    source_ip AS mcp_source_ip,

    x_forwarded_for,

    user_agent AS mcp_user_agent,

    UPPER(http_method) AS http_method,

    url_path,

    url_full,

    http_status,

    request_id

  FROM ENV_GCP_WEB_OR_GATEWAY_ACCESS_LOGS

  WHERE

    event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 60 MINUTE)

    AND UPPER(http_method) = 'POST'

    AND (

      LOWER(url_path) LIKE '%/mcp-rest/test/connection%'

      OR LOWER(url_path) LIKE '%/mcp-rest/test/tools/list%'

      OR LOWER(url_full) LIKE '%/mcp-rest/test/connection%'

      OR LOWER(url_full) LIKE '%/mcp-rest/test/tools/list%'

      OR LOWER(url_path) LIKE '%mcp%test%'

      OR LOWER(url_full) LIKE '%mcp%test%'

    )

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      host_name,

      backend_host,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

mcp_endpoint_filtered AS (

  SELECT

    m.*

  FROM mcp_endpoint_candidates m

  LEFT JOIN approved_sources s

    ON m.mcp_source_ip = s.approved_source_ip

  LEFT JOIN approved_maintenance mw

    ON m.normalized_gateway = mw.normalized_gateway

  WHERE

    s.approved_source_ip IS NULL

    AND mw.normalized_gateway IS NULL

),

mcp_correlation_keys AS (

  SELECT

    m.*,

    correlation_key

  FROM mcp_endpoint_filtered m,

  UNNEST([

    m.normalized_gateway,

    m.litellm_gateway_id,

    m.workload_identity

  ]) AS correlation_key

  WHERE

    correlation_key IS NOT NULL

    AND correlation_key != ''

),

gcp_secret_identity_activity AS (

  SELECT

    event_timestamp AS downstream_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      workload_identity

    ) AS gcp_normalized_gateway,

    litellm_gateway_id AS gcp_litellm_gateway_id,

    workload_identity AS gcp_workload_identity,

    source_ip AS downstream_source_ip,

    user_agent AS downstream_user_agent,

    service_name,

    method_name,

    principal_email,

    service_account_name,

    resource_name,

    request_payload,

    response_payload,

    request_id,

    'gcp_secret_or_identity_activity' AS anomaly_type

  FROM ENV_GCP_AUDIT_NORMALIZED_LOGS

  WHERE

    event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 120 MINUTE)

    AND (

      (

        service_name IN (

          'secretmanager.googleapis.com',

          'cloudkms.googleapis.com'

        )

        AND method_name IN (

          'google.cloud.secretmanager.v1.SecretManagerService.AccessSecretVersion',

          'google.cloud.secretmanager.v1.SecretManagerService.GetSecret',

          'google.cloud.kms.v1.KeyManagementService.Decrypt'

        )

      )

      OR (

        service_name IN (

          'iam.googleapis.com',

          'sts.googleapis.com'

        )

        AND method_name IN (

          'google.iam.admin.v1.CreateServiceAccountKey',

          'google.iam.admin.v1.SetIamPolicy',

          'google.iam.credentials.v1.GenerateAccessToken',

          'google.iam.credentials.v1.GenerateIdToken',

          'google.identity.sts.v1.ExchangeToken'

        )

      )

      OR (

        service_name IN (

          'container.googleapis.com',

          'run.googleapis.com',

          'cloudfunctions.googleapis.com',

          'artifactregistry.googleapis.com',

          'storage.googleapis.com'

        )

        AND method_name IN (

          'google.container.v1.ClusterManager.GetCluster',

          'google.container.v1.ClusterManager.GetServerConfig',

          'google.cloud.run.v2.Services.UpdateService',

          'google.cloud.functions.v2.FunctionService.UpdateFunction',

          'google.devtools.artifactregistry.v1.ArtifactRegistry.GetDockerImage',

          'storage.objects.get',

          'storage.objects.create',

          'storage.buckets.getIamPolicy',

          'storage.buckets.setIamPolicy'

        )

      )

    )

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      workload_identity

    ) IS NOT NULL

),

gcp_secret_identity_keys AS (

  SELECT

    g.*,

    correlation_key

  FROM gcp_secret_identity_activity g,

  UNNEST([

    g.gcp_normalized_gateway,

    g.gcp_litellm_gateway_id,

    g.gcp_workload_identity

  ]) AS correlation_key

  WHERE

    correlation_key IS NOT NULL

    AND correlation_key != ''

),

provider_key_activity AS (

  SELECT

    event_timestamp AS provider_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      provider_key_id,

      pod_name,

      container_id,

      workload_identity

    ) AS provider_normalized_gateway,

    litellm_gateway_id AS provider_litellm_gateway_id,

    provider_key_id,

    provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    workload_identity AS provider_workload_identity,

    source_ip AS provider_source_ip,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id,

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(source_ip AS STRING)) AS provider_source_key,

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(model_name AS STRING)) AS provider_model_key,

    CONCAT(CAST(provider_key_id AS STRING), '|', CAST(provider_account AS STRING)) AS provider_account_key

  FROM ENV_GCP_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_LOGS

  WHERE

    event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 120 MINUTE)

    AND provider_key_id IS NOT NULL

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      provider_key_id,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

provider_key_anomalies AS (

  SELECT

    p.provider_event_time AS anomaly_event_time,

    p.provider_normalized_gateway AS anomaly_normalized_gateway,

    p.provider_litellm_gateway_id AS anomaly_litellm_gateway_id,

    p.provider_workload_identity AS anomaly_workload_identity,

    p.provider_key_id,

    p.provider_account,

    p.model_provider,

    p.model_name,

    p.pod_name,

    p.container_id,

    p.namespace,

    p.provider_source_ip AS downstream_source_ip,

    CAST(NULL AS STRING) AS downstream_user_agent,

    p.destination_ip,

    p.destination_domain,

    p.event_source,

    p.event_name,

    p.request_id,

    'provider_key_anomaly' AS anomaly_type

  FROM provider_key_activity p

  LEFT JOIN approved_provider_sources ps

    ON p.provider_source_key = ps.provider_source_key

  LEFT JOIN approved_provider_models pm

    ON p.provider_model_key = pm.provider_model_key

  LEFT JOIN approved_provider_accounts pa

    ON p.provider_account_key = pa.provider_account_key

  WHERE

    IFNULL(ps.source_approved, FALSE) = FALSE

    OR IFNULL(pm.model_approved, FALSE) = FALSE

    OR IFNULL(pa.account_approved, FALSE) = FALSE

),

egress_activity AS (

  SELECT

    event_timestamp AS egress_event_time,

    COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) AS egress_normalized_gateway,

    litellm_gateway_id AS egress_litellm_gateway_id,

    workload_identity AS egress_workload_identity,

    CAST(NULL AS STRING) AS provider_key_id,

    CAST(NULL AS STRING) AS provider_account,

    model_provider,

    model_name,

    pod_name,

    container_id,

    namespace,

    source_ip AS downstream_source_ip,

    CAST(NULL AS STRING) AS downstream_user_agent,

    destination_ip,

    destination_domain,

    event_source,

    event_name,

    request_id

  FROM ENV_GCP_PROVIDER_PROXY_DNS_OR_EGRESS_NORMALIZED_LOGS

  WHERE

    event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 120 MINUTE)

    AND destination_domain IS NOT NULL

    AND COALESCE(

      litellm_gateway_id,

      normalized_gateway,

      pod_name,

      container_id,

      workload_identity

    ) IS NOT NULL

),

egress_anomalies AS (

  SELECT

    e.egress_event_time AS anomaly_event_time,

    e.egress_normalized_gateway AS anomaly_normalized_gateway,

    e.egress_litellm_gateway_id AS anomaly_litellm_gateway_id,

    e.egress_workload_identity AS anomaly_workload_identity,

    e.provider_key_id,

    e.provider_account,

    e.model_provider,

    e.model_name,

    e.pod_name,

    e.container_id,

    e.namespace,

    e.downstream_source_ip,

    e.downstream_user_agent,

    e.destination_ip,

    e.destination_domain,

    e.event_source,

    e.event_name,

    e.request_id,

    'rare_egress' AS anomaly_type

  FROM egress_activity e

  LEFT JOIN ENV_APPROVED_LITELLM_EGRESS_DESTINATIONS ad

    ON e.destination_domain = ad.approved_destination_domain

  LEFT JOIN ENV_APPROVED_MODEL_PROVIDER_DOMAINS pd

    ON e.destination_domain = pd.approved_provider_domain

  WHERE

    ad.approved_destination_domain IS NULL

    AND pd.approved_provider_domain IS NULL

),

provider_or_egress_anomalies AS (

  SELECT * FROM provider_key_anomalies

  UNION ALL

  SELECT * FROM egress_anomalies

),

provider_or_egress_keys AS (

  SELECT

    p.*,

    correlation_key

  FROM provider_or_egress_anomalies p,

  UNNEST([

    p.anomaly_normalized_gateway,

    p.anomaly_litellm_gateway_id,

    p.anomaly_workload_identity

  ]) AS correlation_key

  WHERE

    correlation_key IS NOT NULL

    AND correlation_key != ''

),

gcp_correlated AS (

  SELECT

    m.mcp_time,

    g.downstream_event_time AS correlated_event_time,

    CASE

      WHEN g.service_name IN (

        'secretmanager.googleapis.com',

        'cloudkms.googleapis.com',

        'iam.googleapis.com',

        'sts.googleapis.com'

      ) THEN 3

      ELSE 2

    END AS severity_rank,

    'LiteLLM MCP test endpoint activity followed by GCP secret or identity evidence' AS detection_name,

    m.normalized_gateway,

    m.litellm_gateway_id,

    m.host_name,

    m.backend_host,

    m.pod_name,

    m.container_id,

    m.namespace,

    m.workload_identity,

    m.mcp_source_ip,

    m.x_forwarded_for,

    m.mcp_user_agent,

    m.http_method,

    m.url_path,

    m.url_full,

    m.http_status,

    g.downstream_source_ip,

    g.downstream_user_agent,

    g.service_name AS downstream_event_source,

    g.method_name AS downstream_event_name,

    g.principal_email,

    g.service_account_name,

    g.resource_name,

    CAST(NULL AS STRING) AS provider_key_id,

    CAST(NULL AS STRING) AS provider_account,

    CAST(NULL AS STRING) AS model_provider,

    CAST(NULL AS STRING) AS model_name,

    CAST(NULL AS STRING) AS destination_domain,

    CAST(NULL AS STRING) AS destination_ip,

    g.anomaly_type,

    g.request_id

  FROM mcp_correlation_keys m

  INNER JOIN gcp_secret_identity_keys g

    ON m.correlation_key = g.correlation_key

  WHERE

    g.downstream_event_time BETWEEN m.mcp_time AND TIMESTAMP_ADD(m.mcp_time, INTERVAL 120 MINUTE)

),

provider_egress_correlated AS (

  SELECT

    m.mcp_time,

    p.anomaly_event_time AS correlated_event_time,

    2 AS severity_rank,

    'LiteLLM MCP test endpoint activity followed by provider-key or rare-egress evidence' AS detection_name,

    m.normalized_gateway,

    m.litellm_gateway_id,

    m.host_name,

    m.backend_host,

    m.pod_name,

    m.container_id,

    m.namespace,

    m.workload_identity,

    m.mcp_source_ip,

    m.x_forwarded_for,

    m.mcp_user_agent,

    m.http_method,

    m.url_path,

    m.url_full,

    m.http_status,

    p.downstream_source_ip,

    p.downstream_user_agent,

    p.event_source AS downstream_event_source,

    p.event_name AS downstream_event_name,

    CAST(NULL AS STRING) AS principal_email,

    CAST(NULL AS STRING) AS service_account_name,

    CAST(NULL AS STRING) AS resource_name,

    p.provider_key_id,

    p.provider_account,

    p.model_provider,

    p.model_name,

    p.destination_domain,

    p.destination_ip,

    p.anomaly_type,

    p.request_id

  FROM mcp_correlation_keys m

  INNER JOIN provider_or_egress_keys p

    ON m.correlation_key = p.correlation_key

  WHERE

    p.anomaly_event_time BETWEEN m.mcp_time AND TIMESTAMP_ADD(m.mcp_time, INTERVAL 120 MINUTE)

),

combined_correlations AS (

  SELECT * FROM gcp_correlated

  UNION ALL

  SELECT * FROM provider_egress_correlated

),

summarized_correlations AS (

  SELECT

    mcp_time,

    normalized_gateway,

    litellm_gateway_id,

    MIN(correlated_event_time) AS correlated_event_time,

    MAX(severity_rank) AS severity_rank,

    ANY_VALUE(detection_name) AS detection_name,

    ANY_VALUE(host_name) AS host_name,

    ANY_VALUE(backend_host) AS backend_host,

    ANY_VALUE(pod_name) AS pod_name,

    ANY_VALUE(container_id) AS container_id,

    ANY_VALUE(namespace) AS namespace,

    ANY_VALUE(workload_identity) AS workload_identity,

    ANY_VALUE(mcp_source_ip) AS mcp_source_ip,

    ANY_VALUE(x_forwarded_for) AS x_forwarded_for,

    ANY_VALUE(mcp_user_agent) AS mcp_user_agent,

    ANY_VALUE(http_method) AS http_method,

    ANY_VALUE(url_path) AS url_path,

    ANY_VALUE(url_full) AS url_full,

    ANY_VALUE(http_status) AS http_status,

    ANY_VALUE(downstream_source_ip) AS downstream_source_ip,

    ANY_VALUE(downstream_user_agent) AS downstream_user_agent,

    ANY_VALUE(downstream_event_source) AS downstream_event_source,

    ANY_VALUE(downstream_event_name) AS downstream_event_name,

    ANY_VALUE(principal_email) AS principal_email,

    ANY_VALUE(service_account_name) AS service_account_name,

    ANY_VALUE(resource_name) AS resource_name,

    ANY_VALUE(provider_key_id) AS provider_key_id,

    ANY_VALUE(provider_account) AS provider_account,

    ANY_VALUE(model_provider) AS model_provider,

    ANY_VALUE(model_name) AS model_name,

    ANY_VALUE(destination_domain) AS destination_domain,

    ANY_VALUE(destination_ip) AS destination_ip,

    ARRAY_AGG(DISTINCT anomaly_type IGNORE NULLS) AS anomaly_type,

    ANY_VALUE(request_id) AS request_id

  FROM combined_correlations

  GROUP BY

    mcp_time,

    normalized_gateway,

    litellm_gateway_id

)

SELECT

  mcp_time,

  correlated_event_time,

  CASE

    WHEN severity_rank = 3 THEN 'critical'

    WHEN severity_rank = 2 THEN 'high'

    ELSE 'medium'

  END AS severity,

  detection_name,

  normalized_gateway,

  litellm_gateway_id,

  host_name,

  backend_host,

  pod_name,

  container_id,

  namespace,

  workload_identity,

  mcp_source_ip,

  x_forwarded_for,

  mcp_user_agent,

  http_method,

  url_path,

  url_full,

  http_status,

  downstream_source_ip,

  downstream_user_agent,

  downstream_event_source,

  downstream_event_name,

  principal_email,

  service_account_name,

  resource_name,

  provider_key_id,

  provider_account,

  model_provider,

  model_name,

  destination_domain,

  destination_ip,

  anomaly_type,

  request_id

FROM summarized_correlations;

S26 Threat-to-Rule Traceability

MCP Test Endpoint Abuse

Covered by NDR / Network Behavioral Analytics, Splunk, Elastic, QRadar, AWS, Azure, and GCP where URI paths, LiteLLM gateway identity, and source context are available.

Authenticated Low-Privilege Key Abuse

Partially covered. Direct coverage requires LiteLLM logs that preserve proxy-key, user, team, or service-key attribution. Without LiteLLM identity telemetry, rules infer risk from source, endpoint, gateway, process, provider, and time-window correlation.

Authentication-Bypass Chain Conditions

Partially covered. Direct coverage requires reverse-proxy, Host header, routing, application-authentication, and LiteLLM access telemetry. Without authentication telemetry, rules infer risk from unauthenticated or unexpected source activity against MCP endpoint paths and downstream execution evidence.

Runtime Command Execution

Covered by SentinelOne, Splunk, Elastic, QRadar, and SIGMA where process creation, parent process, command-line, service-user, container, pod, or runtime telemetry is available.

Provider-Key Exposure

Covered by SentinelOne, Splunk, Elastic, QRadar, NDR, and cloud-hosted correlation rules where provider-key files, environment variables, mounted secrets, provider audit records, or billing telemetry can be tied to LiteLLM gateway activity.

Environment and Mounted-Secret Access

Covered by SentinelOne, Splunk, Elastic, QRadar, AWS, Azure, and GCP where file telemetry, process telemetry, cloud secret-manager logs, Kubernetes audit logs, or workload identity telemetry are available.

Provider API Abuse

Covered where model-provider audit logs, provider billing records, provider key identifiers, source IPs, model names, application identifiers, and provider usage baselines can be tied to LiteLLM-owned keys.

Rare Outbound Communication

Covered by NDR, DNS/proxy, Splunk, Elastic, QRadar, AWS, Azure, and GCP correlations where LiteLLM host identity and egress telemetry are available.

Gateway Configuration Tampering

Covered where LiteLLM administrative logs, database change records, configuration file telemetry, deployment change records, or Kubernetes/cloud configuration logs can be joined to affected gateway activity.

Cloud, Kubernetes, and CI/CD Expansion

Covered with adaptation where LiteLLM workload identity, host, pod, namespace, service-account, cloud role, CI/CD token, or provider key can be linked to suspicious downstream activity.

Post-Patch Compromise Validation

Covered through hunting logic, LiteLLM log review, endpoint and container review, secret access review, provider-key rotation validation, provider audit review, cloud and Kubernetes identity review, known-good configuration comparison, and model-provider billing review.

Evidence and Visibility Gaps

Covered by telemetry requirements, detection gaps, non-coverage conditions, cloud telemetry limitations, provider-key attribution limitations, and local implementation guidance.

S29 Detection Coverage Summary

Coverage is strongest where LiteLLM logs, WAF logs, reverse-proxy logs, endpoint process telemetry, endpoint file telemetry, container telemetry, Kubernetes audit logs, cloud audit logs, DNS/proxy logs, model-provider audit logs, billing records, LiteLLM administrative state, and asset inventory can be joined by LiteLLM gateway, virtual host, backend host, source, proxy key, user, team, pod, container, file path, process, provider key, destination, and time window.

Minimum viable coverage requires visibility into LiteLLM MCP test endpoint requests and runtime process telemetry or provider-key usage telemetry.

Stronger coverage requires file telemetry, endpoint telemetry, container telemetry, LiteLLM administrative state, provider audit logs, cloud audit logs, Kubernetes audit logs, and egress telemetry.

Highest confidence requires correlation across MCP test endpoint request activity, vulnerable LiteLLM version state, unexpected subprocess execution, secret or environment access, provider-key usage anomalies, LiteLLM administrative changes, rare egress, cloud or Kubernetes activity, and provider billing anomalies.

Cloud-native logs alone are insufficient unless combined with LiteLLM application-layer logs, host or container telemetry, workload identity mapping, provider-key mapping, and URI-preserving gateway telemetry.

Customer-specific telemetry validation is expected and does not reduce production-readiness when Required Telemetry, Engineering Implementation Instructions, Limitations, and Notes / Next Suggested Steps provide the engineer or administrator with a clear implementation path.

S33 Defensive Control & Hardening Improvements

Upgrade LiteLLM to 1.83.7 or later.

Validate dependency exposure where Starlette-based authentication-bypass conditions may apply.

Restrict or block access to /mcp-rest/test/connection and /mcp-rest/test/tools/list where immediate upgrade is not possible.

Limit LiteLLM administrative and MCP testing functionality to approved administrator networks, approved service accounts, and approved maintenance windows.

Preserve LiteLLM, reverse-proxy, WAF, load-balancer, endpoint, container, Kubernetes, cloud, provider, and DNS/proxy logs before rotation.

Review LiteLLM proxy keys, service keys, internal-user keys, teams, users, administrators, provider routes, model routes, and MCP server definitions for unfamiliar, recently created, or abnormal entries.

Rotate provider API keys where vulnerable exposure existed or command execution cannot be ruled out.

Rotate LiteLLM proxy keys and service keys after suspected compromise.

Review environment variables, mounted secrets, Kubernetes service-account tokens, cloud credentials, CI/CD secrets, database credentials, and provider-key storage for exposure risk.

Review model-provider audit and billing records for abnormal source locations, request volume, token usage, model selection, application identifiers, and quota consumption.

Review prompt and response logs or stores through approved legal, privacy, and data governance processes where sensitive AI traffic may have been exposed.

Restrict LiteLLM egress to approved model-provider endpoints, required internal services, and approved management destinations.

Disable broad cloud metadata access from LiteLLM workloads where not required.

Use workload identity least privilege for LiteLLM pods, containers, and cloud services.

Segment LiteLLM from CI/CD, source repositories, secret managers, production databases, and broad internal networks unless required.

Validate Kubernetes service-account permissions for LiteLLM namespaces.

Validate file-integrity or configuration monitoring for LiteLLM configuration, secrets, deployment manifests, and startup commands.

Maintain a current asset inventory of LiteLLM deployments, versions, routes, provider keys, owners, teams, and model-provider mappings.

Preserve logs from LiteLLM, WAF, reverse proxy, load balancer, endpoint, container, Kubernetes, cloud, DNS, proxy, provider, billing, CI/CD, and secret-manager sources during investigation.

S39 Economic Impact & Organizational Exposure

LiteLLM command injection creates organizational exposure by increasing uncertainty around AI gateway integrity, model-provider credential trust, provider billing control, prompt and response confidentiality, gateway configuration reliability, cloud and Kubernetes identity boundaries, AI service availability, and AI governance confidence. Exposure rises when affected LiteLLM deployments support customer-facing AI features, internal copilots, RAG workflows, autonomous agents, developer automation, security operations, regulated-data workflows, or multi-provider AI routing.

Estimated Economic Exposure

Estimated exposure should be scenario-based and tied to whether activity remains limited to vulnerable exposure or suspicious MCP test endpoint attempts, becomes runtime command execution, or expands into provider-key access, prompt or response exposure, model-provider abuse, cloud secret access, Kubernetes activity, CI/CD token exposure, legal review, customer communications, or multi-application compromise.

Low Impact Scenario

Estimated $20K - $100K.

This scenario applies when vulnerable-version exposure or suspicious LiteLLM MCP endpoint activity is identified quickly, exploit attempts are blocked or unsuccessful, and log review, endpoint review, provider audit review, gateway configuration review, and update validation confirm no subprocess execution, provider-key exposure, secret access, gateway tampering, provider abuse, prompt or response exposure, rare egress, or customer impact.

Moderate Impact Scenario

Estimated $150K - $850K.

This scenario applies when suspicious MCP endpoint activity, unexpected subprocess execution, uncertain secret access, suspicious provider-key usage, or incomplete telemetry prevents the organization from quickly ruling out command execution, provider-key exposure, prompt or response exposure, gateway tampering, cloud access, Kubernetes access, or provider abuse. Response may require provider-key rotation, LiteLLM key invalidation, host or container forensics, cloud and Kubernetes review, AI application regression testing, provider billing analysis, and customer-facing assurance.

High Impact Scenario

Estimated $900K - $6.5M+.

This scenario applies when attackers executed commands from the LiteLLM runtime, accessed provider keys, exposed prompt or response data, modified gateway policy, abused model-provider accounts, accessed cloud secrets, used Kubernetes service-account tokens, touched CI/CD systems, affected customer-facing AI services, triggered legal review, required customer notification analysis, generated billing abuse, compromised multiple AI applications, or forced emergency rebuild and restoration of AI gateway trust.

Annualized Risk Exposure

Estimated $150K - $900K for organizations with several LiteLLM deployments, incomplete AI gateway inventory, shared provider keys, limited gateway logging, weak container telemetry, missing provider audit retention, or incomplete workload identity mapping.

Exposure may exceed $900K - $6.5M+ where LiteLLM compromise results in provider-key theft, prompt or response data exposure, cloud secret access, Kubernetes workload impact, CI/CD token exposure, customer-facing AI service interruption, unauthorized provider billing, legal review, customer communications, or public restoration of AI governance trust.

Operational Dependency

Operational dependency is high where LiteLLM supports customer-facing AI features, internal copilots, autonomous agents, RAG systems, developer productivity, security workflows, analytics automation, support operations, regulated-data workflows, or centralized provider-cost controls.

Control Trust

Control trust is reduced when the organization cannot prove that LiteLLM proxy keys, provider keys, users, teams, routes, MCP definitions, model-provider configuration, environment variables, mounted secrets, cloud identities, Kubernetes service accounts, and prompt or response stores remained legitimate during the exposure window.

Visibility Confidence

Visibility confidence is highest when LiteLLM logs, WAF logs, reverse-proxy logs, endpoint telemetry, container telemetry, Kubernetes audit logs, cloud audit logs, provider audit logs, provider billing records, DNS/proxy logs, secret-manager logs, approved inventories, and maintenance records can be joined reliably.

Change-Control Confidence

Change-control confidence is high when LiteLLM upgrades, endpoint restrictions, key rotations, provider route changes, MCP configuration changes, deployment changes, Kubernetes changes, cloud identity changes, provider account changes, and emergency remediation are documented and attributable.

Downstream Dependency

Downstream dependency is high when LiteLLM connects to model-provider APIs, cloud services, Kubernetes workloads, RAG data stores, vector databases, prompt stores, response stores, customer applications, CI/CD systems, source repositories, container registries, secret managers, internal APIs, or production business workflows.

Customer and Regulatory Exposure

Customer and regulatory exposure increases when suspicious LiteLLM activity may affect customer prompts, customer responses, uploaded documents, regulated data, source code, security data, employee data, business-sensitive records, provider credentials, AI outputs, customer-facing model behavior, or AI-service availability.

Residual Economic Risk

Residual economic risk remains if the organization cannot prove that vulnerable LiteLLM versions were upgraded, MCP endpoint exposure was restricted, provider keys were rotated where required, proxy keys were invalidated where required, subprocess execution was ruled out, secret access was scoped, provider usage was reviewed, prompt and response exposure was assessed, cloud and Kubernetes identities were validated, and AI gateway integrity was restored.

Behavioral Coverage Assessment

This TTD covers LiteLLM AI gateway command injection aligned with MCP test endpoint abuse, low-privilege or unauthenticated gateway access conditions, runtime subprocess execution, provider-key exposure, environment and mounted-secret access, provider API abuse, rare egress, gateway configuration tampering, cloud and Kubernetes expansion, and post-patch compromise validation.

The TTD is not limited to one CVE string, exploit payload, request path, user agent, IP address, command line, provider, scanner result, vendor advisory, or proof-of-concept implementation.

Detection Engineering Coverage Interpretation

The S25 detection content provides direct behavioral coverage where observable activity falls inside the TTD’s detection model: suspicious LiteLLM MCP endpoint access, runtime subprocess execution, secret access, provider-key exposure, model-provider usage anomaly, gateway administrative change, rare egress, cloud identity activity, Kubernetes activity, and post-remediation validation.

The S25 detection content provides coverage with adaptation for related LiteLLM, MCP, AI gateway, model proxy, provider-key, agent gateway, cloud-hosted AI service, containerized gateway, Kubernetes workload, or AI infrastructure abuse activity where observable behavior aligns to gateway endpoint abuse, command execution, credential exposure, provider misuse, configuration tampering, or downstream workload impact.

Non-Coverage Conditions

Non-coverage applies where related activity does not produce observable LiteLLM MCP endpoint behavior, runtime subprocess execution, secret or environment access, provider-key exposure, provider-key usage anomaly, gateway configuration change, rare egress, cloud or Kubernetes activity, or AI gateway integrity impact.

Activity limited to unrelated LiteLLM vulnerabilities, unrelated AI tools, generic cloud anomalies, identity-only anomalies, network-only anomalies, isolated scanner findings, benign AI platform testing, normal provider validation, or unrelated model-provider billing changes should not be represented as covered by this TTD unless behavior aligns to the detection model.

Coverage Qualification

This is a behavioral detection-readiness statement, not a universal LiteLLM, AI gateway, MCP, cloud, Kubernetes, model-provider, CVE, KEV, or proof-of-concept coverage ledger. A related issue should only be considered aligned when it shares enough observable behavior with the TTD’s detection model to support credible detection or detection-readiness coverage.

KEV status, vendor severity, exploit availability, or public proof-of-concept status should be treated as urgency and remediation-prioritization signals, not as the basis for coverage by themselves. Coverage remains based on observable LiteLLM-to-command-execution and provider-key-exposure behavior aligned to the TTD’s S21 through S25 detection strategy.

Executive Exposure Statement

The organization’s economic exposure is highest when LiteLLM compromise creates uncertainty around whether AI gateway configuration, provider keys, prompt and response data, model routes, proxy keys, runtime secrets, cloud identities, Kubernetes service accounts, and customer-facing AI workflows remain trustworthy. The strategic risk is not one CVE or one endpoint path; it is the possibility that attackers can convert a trusted AI gateway into a command-execution and provider-key exposure point that undermines centralized AI governance after patching.

S40 References

Primary Vulnerability and Exploitation References

·        LiteLLM GitHub Repository - hxxps://github[.]com/BerriAI/litellm

·        GitHub Security Advisory - GHSA-v4p8-mg3p-g94g / CVE-2026-42271 - hxxps://github[.]com/BerriAI/litellm/security/advisories/GHSA-v4p8-mg3p-g94g

·        CISA - Known Exploited Vulnerabilities Catalog - hxxps://www[.]cisa[.]gov/known-exploited-vulnerabilities-catalog

·        NVD - CVE-2026-42271 - hxxps://nvd[.]nist[.]gov/vuln/detail/CVE-2026-42271

·        Horizon3.ai - CVE-2026-42271 Chained With CVE-2026-48710: LiteLLM Unauthenticated RCE - hxxps://horizon3[.]ai/attack-research/vulnerabilities/cve-2026-42271-chained-with-cve-2026-48710/

·        Cloud Security Alliance - LiteLLM AI Gateway: Active Exploitation via MCP Injection - hxxps://labs[.]cloudsecurityalliance[.]org/research/csa-research-note-litellm-cve-2026-42271-ai-gateway-exploita/

Supporting Framework Reference

·        MITRE ATT&CK Enterprise Matrix / Techniques Catalog - hxxps://attack[.]mitre[.]org/