In Your Subscription, Not Ours: Why Regulated Firms Can't Use SaaS AI Observability
In Your Subscription, Not Ours
Portkey, Helicone, LangSmith. Good products. They solve a real problem for teams that need to understand what their models and agents are doing.
But they are SaaS products.
If you are measuring latency for an internal chatbot, that may be perfectly reasonable. If the telemetry contains customer records, health information or payment details, the decision becomes more serious.
It is no longer just an observability choice. It is also a data processing and residency decision.
The data plane problem
When an AI agent calls a tool, the arguments it sends are not abstract metadata. They are often the real business data.
A journal-posting agent might send:
- An account code
- An amount
- A transaction description
A customer lookup agent might send:
- A name
- A policy number
- A date of birth
A payment agent might send:
- Card details
- Bank references
- Customer identifiers
Observability tools capture those arguments because that is how you understand what the agent actually did.
The important question is where that capture happens.
With a SaaS observability platform, the data leaves your environment, passes through the vendor’s infrastructure and is stored in their systems.
That means your observability provider is now processing the same regulated data your agents handle.
Observability is also a data protection decision
Article 32 of the UK GDPR requires organisations to use appropriate technical and organisational measures to protect personal data.
That includes understanding:
- Where the data is processed
- Who can access it
- Which sub-processors are involved
- Whether the data crosses borders
- How much data is being collected and retained
For special-category data, including health and biometric information, those questions become even more important.
Sending agent telemetry to a third-party SaaS platform is a processing decision. Your DPO or privacy team may need to review the data flows, processor terms, sub-processor chain and transfer mechanisms.
Many teams do not think of observability as a data residency issue.
But when telemetry contains regulated payloads, that is exactly what it becomes.
Where this leaves regulated firms
The firms I talk to do not have a problem with observability as a concept. They want to know what their agents did. They need to know.
The problem is architectural. SaaS observability assumes the data can leave the building. For a health insurer, a payments firm, or a bank, it often cannot.
What I built instead
The MCP Audit and Compliance Gateway runs inside the customer's own environment. Azure subscription, AWS VPC, GCP project, Oracle Cloud, on-premise Docker host. Wherever the agents and MCP servers already live.
I have no access to the customer's data. The gateway is a reverse proxy that sits between the agent and the MCP server. It captures every tool call, masks sensitive fields, evaluates policy, and writes a structured audit row to whatever log sink the firm already uses.
The telemetry never leaves the perimeter.
I did not design it that way to be clever. It is just the only architecture that works when the data is regulated. If your agents touch health records, payment card data, or customer PII, the observability layer has to live where the data lives.
One question
Next time you evaluate an AI observability product, ask where your agent's tool-call arguments end up.
If the answer is "our cloud", run it past your DPO. For some firms and some data, that will be fine. For health insurers, payments firms, and banks, it often will not be.