The privacy trade-off no AI agent paper wants to discuss.

The marketing copy for AI agents built on regulated data — health records, brokerage data, attorney-client material, defense data — almost always contains some version of the same sentence:

No customer data is ever sent to the LLM.

It's reassuring. It's often technically true in some narrow sense. And when readers who know what they're looking at see it, they immediately start asking what the rest of the sentence is.

I want to write the rest of the sentence down, because the industry will be more trusted for saying it out loud, and because I think the honest version is actually more defensible than the marketing one.

What “no data to the LLM” usually means

In a text-to-SQL system — or any agent that translates a natural- language question into a structured query — there are two distinct moments where data could be sent to the LLM.

Moment one: query construction.The agent receives the user's question and needs to turn it into a database query. To do that, the LLM needs to see the schema of the data — table names, column names, the natural-language description of each table, often a handful of example queries for in-context learning. It does notneed to see the rows themselves. A well-designed system sends only metadata at this step, and that's what most “no data to the LLM” claims correctly describe.

Moment two: narrative generation. If the system is conversational — if it returns prose answers instead of raw grids — then after the query runs, the result rows have to be summarized into natural language. To do that, the LLM needs to see the actual results. The customer data. The dollar amounts, the ticker symbols, the share counts, the timestamps.

Marketing claims about “no data ever to the LLM” almost always elide the second moment. They're describing a system that produces grids, not narratives. Or a system whose narrative is templated server-side after the LLM has finished. Or, honestly, a system that hasn't shipped narrative responses yet.

The real trade-off

The user experience of a conversational answer is substantially better than the user experience of a grid. By an embarrassing margin, in fact. Once you've seen the same data summarized in a single confident sentence (“Your top ten holdings represent forty-three percent of the portfolio's market value, with the largest single position being NVDA at six point eight percent”), going back to a raw grid feels like a regression.

So when you're designing this kind of system, you face a real choice. You can:

• Ship the grid-only version and truthfully say no data ever reaches the LLM.
• Ship the conversational version and accept that result rows are sent to the LLM at narrative-generation time.
• Ship the conversational version and lie about it. (Hopefully not.)

We took the second option for INDATA Nexus. It was a conscious decision, not an oversight, and it was predicated on a fact about the LLM-provider landscape that has quietly matured in the last eighteen months.

What the industry got right

Commercial LLM providers — Anthropic, OpenAI, Microsoft Azure — now offer paid-tier API agreements that look a lot like the data protections enterprises have had from their cloud providers for a decade. No training on customer data by default. Configurable or minimal retention. Standard data-processing agreements that plug into existing compliance frameworks. Enterprise SOC 2 and HIPAA postures where they apply.

That is not the same thing as “the data never leaves your perimeter,” and we should not pretend otherwise. But it is a meaningful, contractually enforceable, audit-trail-able protection — the same kind that financial services and healthcare have used to put customer data into AWS, Azure, GCP, Snowflake, and Salesforce for the last decade and a half. The objection to commercial LLM APIs in 2024 was almost entirely about the absence of those guarantees. Most of those guarantees now exist.

The honest framing

For our system, the precise truth is a two-part statement:

1. At query construction, only schema metadata is sent to the LLM — column names, table names, descriptive documentation, example queries. No customer row data reaches the model at this step.
2. At narrative generation, the result rows of the already-authorized query are sent to the LLM, under a paid-tier commercial agreement with explicit privacy terms — no training on data, retention limits, standard processor obligations.

All data remains inside the customer's authorization boundary at the execution layer; the query runs against the customer's data store with the customer's permissions; row-level access controls still apply. The LLM only ever sees rows that the requesting user is already entitled to see in every other surface we offer.

That is the version I would defend in front of an auditor, because it is the version that's true. It is also the version that the head of risk at any sophisticated client will recognize as honest, and engage with on its merits, instead of treating with suspicion.

Why the simple version is worse for trust

Here is the practical thing I'd say to other AI-agent builders: the simplified claim is more dangerous than the two-part claim, even though it sounds safer.

When a technically sophisticated reader — a CISO, a head of risk, a deal lawyer — encounters “no data ever goes to the LLM,” one of two things happens. Either they believe it, buy the product, and discover the truth later (very bad), or they don't believe it, lose trust in you, and either kill the deal or apply punitive conditions to it (also bad). The honest two-part version pre-empts both failure modes. The first time you say it, you sound less impressive than the marketing copy. The second and third time you say it, you sound like the only vendor in the room who has thought about it seriously.

The buy-side, the legal world, the healthcare world, the public sector — they are full of people who have been doing data governance professionally for thirty years. They are not going to believe a one-sentence claim. They are going to want the two-paragraph version. The vendors that lead with the two-paragraph version are the ones that get the second meeting.

A closing thought

I think we're going to look back at the 2024–2025 era of AI marketing the way we look back at the 1999 era of cloud marketing: full of claims that sounded technically reassuring and turned out, on inspection, to mean something narrower than what the reader assumed. The vendors who built durable trust then were the ones who explained the trade-offs out loud. I think it's the same play this time.