How to Keep Secrets Out of AI Prompts and Logs

Figures on this page were verified 31 August 2026 against the providers' own documentation. Pricing, context windows and rate limits change without notice, so confirm any number against the provider before you rely on it. Tell us if something here is out of date.

Assume every token you send to a model is written to a log you do not control. Not because providers are careless, but because prompts pass through your own logging, error trackers, traces and retry queues long before they reach an API. The leak is almost always local. The fix is to make secrets structurally unable to reach the prompt, rather than reminding people not to paste them.

Where prompts actually leak

Note that only one of those is the provider. If you treat this as a vendor trust question you will secure the wrong boundary.

Redact at construction, not at logging

The common approach is a log filter that strips secrets on the way out. It fails because it only protects the surfaces you remembered to wrap. Build the prompt from data that never contained the secret in the first place.

import re

PATTERNS = [
    (re.compile(r"sk-[A-Za-z0-9]{20,}"),               "[API_KEY]"),
    (re.compile(r"gh[pousr]_[A-Za-z0-9]{20,}"),        "[GITHUB_TOKEN]"),
    (re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----[sS]+?-----END [A-Z ]*PRIVATE KEY-----"), "[PRIVATE_KEY]"),
    (re.compile(r"b[w.+-]+@[w-]+.[w.]+b"),       "[EMAIL]"),
    (re.compile(r"b(?:d[ -]*?){13,16}b"),           "[CARD]"),
]

def scrub(text: str) -> str:
    for pattern, token in PATTERNS:
        text = pattern.sub(token, text)
    return text

prompt = scrub(build_prompt(user_input, retrieved_docs))

Regexes catch the shapes you anticipated and nothing else, so treat this as a safety net rather than the control. The control is not putting secrets in scope.

Give the model a handle, not the value

The strongest pattern is indirection. The model reasons about a reference; your code resolves it. A model that never receives a credential cannot leak one, and no amount of prompt injection can extract what was never sent.

# WRONG: the credential is in the prompt and now in every log downstream
prompt = f"Call the billing API with key {settings.BILLING_KEY} for {customer}"

# RIGHT: the model asks for an action; your code holds the secret
tools = [{
    "name": "fetch_invoice",
    "description": "Fetch an invoice by customer reference.",
    "input_schema": {
        "type": "object",
        "properties": {"customer_ref": {"type": "string"}},
        "required": ["customer_ref"],
        "additionalProperties": False,
    },
}]
# The handler injects the credential server-side, out of the model's reach.

The same logic applies to retrieved documents. If a document store contains credentials, retrieval will eventually surface one into a prompt. Filter at indexing time, because filtering at query time only works for the queries you predicted.

Turn off what records prompts by default

• Tracing. Most LLM instrumentation records prompt and completion as span attributes. Disable content capture, or hash it, unless you have decided otherwise deliberately.
• Error tracking. Turn off local-variable capture on the code paths that assemble prompts.
• Queues. Set a short retention on payloads and scrub before enqueuing, not after dequeuing.
• Caching. Keep secrets out of any cached prefix. See how prompt caching works for why the prefix is stored at all.

Test that it holds

Add a canary. Put a distinctive fake secret into a test fixture, run a representative request, and assert the string appears in no log, span or queue entry. Run it in CI. This is the only way to know your redaction still works after someone adds a new logging call six months from now.

Frequently asked questions

Where do prompts actually leak?

Almost always locally, before the API is reached: application logs left over from debugging, error trackers capturing local variables, tracing spans that record prompt content by default, retry queues with long retention, and developers pasting failing prompts into tickets.

Is redacting at the logging layer enough?

No, because it only protects the surfaces you remembered to wrap. Build the prompt from data that never contained the secret, and treat pattern-based scrubbing as a safety net rather than the control.

How do I let a model use a credential safely?

You do not give it one. Expose a tool that takes a reference such as a customer ID, and have your handler inject the credential server-side. A model that never receives a secret cannot leak it, and no prompt injection can extract it.