ai

Do not plug your data directly into LLMs

Koray

• 4 min read

This sounds like an obvious one however there are lots of small to medium businesses that are plugging their data directly into third party LLM provider(s) for many reasons.

Let's look into the worse case scenarios below:

  • Cross-permission data leakage. An employee asks an innocent question and the system retrieves information they were never authorized to see: executive email, salaries, HR records, legal documents, customer contracts, acquisition discussions, source code, etc. The LLM becomes an accidental bypass around existing application-level authorization.
  • Prompt injection through company data. An attacker puts malicious instructions inside an email, PDF, webpage, support ticket, résumé, invoice, or other document. The AI later retrieves the document and interprets those instructions as something it should follow. This is particularly dangerous when the model also has tools or agent capabilities.
  • Agent + excessive permissions = real-world actions. This is potentially much worse than information leakage. If an agent can access email, CRM, files, ERP, payments, cloud systems, etc., malicious or erroneous reasoning could cause it to send information, alter records, create transactions, contact customers, or trigger workflows. The security boundary must exist outside the model.
  • Mass data exfiltration. A poorly designed system might allow someone to progressively query the LLM to reconstruct large portions of confidential company information. Individual requests can look harmless while collectively extracting customer lists, pricing, contracts, financial data, employee information, or intellectual property.
  • Sensitive-data propagation. Data retrieved from one system can accidentally appear somewhere else: generated reports, summaries, logs, traces, observability platforms, caches, vector databases, conversation histories, downstream APIs, or another model. You can end up with many more copies of sensitive information than existed originally.
  • Loss of provenance. The model combines an email, spreadsheet, CRM entry and invoice and says: “This customer owes $680,000.” Nobody can determine exactly how it arrived at $680,000. Management may then make a decision based on an AI-generated synthesis whose evidence chain is unclear.
  • Hallucination becomes corporate “fact.” This is especially dangerous if LLM output gets written back into CRM, ERP or knowledge systems. An inference becomes a record; another AI later reads that record as evidence; the incorrect information propagates. You can create a kind of AI data contamination loop.
  • Inference of information that wasn't explicitly stored. This one is frequently overlooked. An LLM can combine individually innocuous records and infer something sensitive. For example, emails + calendars + financial documents + organizational data might reveal a confidential restructuring, likely acquisition, employee issue, pricing strategy, or deteriorating customer relationship. Access control at the individual-document level doesn't necessarily prevent this.
  • Data residency, retention and compliance problems. Depending on architecture, configuration, vendor and jurisdiction, information may cross organizational or geographic boundaries or be retained in ways the company didn't intend. Enterprise/API offerings can provide substantially different controls from consumer products, so this is an architecture/configuration issue rather than simply “cloud LLMs keep your data.”
  • Catastrophic credential exposure. If source repositories, configuration files, support conversations or internal documentation are indiscriminately indexed, retrieved context can contain API keys, tokens, credentials, connection strings or internal infrastructure information. Even if the LLM provider handles that information securely, the AI application may expose it to an unauthorized user or tool.

So how to do this?

Shortest answer is by building a controlled intelligence layer between your data and LLM.

Controlled intelligence layer to handle :

  • Ingestion & normalization — connectors for email, Microsoft 365/Google Workspace, CRM, ERP, documents, messaging, databases and APIs. Parse and normalize content rather than blindly forwarding raw data.
  • Classification & sensitive-data detection — identify PII, financial information, credentials, confidential material, legal documents, customer information and other sensitive categories. Attach classifications and metadata.
  • Identity & authorization — integrate with enterprise IAM and preserve source-system permissions. The fundamental rule should be: the AI cannot retrieve something the requesting user isn't authorized to see.
  • Business knowledge layer — turn unstructured information into durable entities and relationships such as Customer → Contact → Project → Contract → Commitment → Invoice → Payment. This can involve relational stores, document stores and/or a knowledge graph. Crucially, the company knowledge doesn't disappear when an LLM context window closes.
  • Evidence & provenance — every extracted fact or relationship should retain its source: document, email, spreadsheet cell, CRM record, timestamp, extraction method and potentially confidence. If AI says “Customer X owes $420k,” you should be able to click through to the evidence.
  • Retrieval & context assembly — determine the minimum information needed for a particular request. Use semantic retrieval, structured queries, graph traversal and business rules rather than dumping everything vaguely relevant into a huge prompt.
  • Privacy & redaction — remove or mask information the model doesn't need. The user might be entitled to an answer without the external LLM needing the underlying person's name, account number or other sensitive fields.
  • Policy engine / AI gateway — the final control point before information leaves your environment. It can enforce approved models, permitted use cases, token/data limits, geographic restrictions, model routing, blocked data classes and other organizational policies.
  • Agent/tool controls — particularly important as AI becomes agentic. Separate read from write permissions and require stronger controls for consequential actions. An agent being allowed to read an invoice does not imply it should be allowed to issue a refund.
  • Audit, monitoring & evaluation — record who asked, what was retrieved, what data was sent to which model, what tools were invoked, what answer came back and what actions followed. Add quality, security, cost and anomaly monitoring.