API key security best practices for LLMs

An LLM API key is a credit card with a memory. If it leaks, the attacker can mint tokens until your spend cap kicks in (or your bank account empties, if you didn't set one). The mechanics of keeping a key safe are not exotic — they're the same secrets-handling discipline you'd apply to any production credential — but enough teams skip them that "leaked LLM key" is now a regular occurrence on Hacker News.

Where to keep keys

Pick one secrets manager and use it for everything. Pre-baked options: AWS Secrets Manager, Google Secret Manager, HashiCorp Vault, 1Password Secrets Automation, Doppler, Infisical. Roll-your-own options: none, please. The win from a real secrets manager is not the storage — it's the rotation, the audit trail, and the ability to revoke a key without touching code.

For local development, use direnv with an .envrc that's gitignored. Never put a real key in .env.example; that file gets committed. (Search GitHub for "OPENAI_API_KEY=sk-" — the live keys you'll find are a constant stream of accidents.)

Scope the key

Every modern provider supports per-project or per-workspace keys. Use them. The blast radius of a leaked key is bounded by how widely it's been authorized: a single-project, $50/month-capped key is a survivable mistake; a root-org key with no spend limit is a Slack post to the legal team.

OpenAI: project keys + restricted permissions + per-project spend caps. Anthropic: workspaces + spend limits. Gemini: API key restrictions to the Generative Language API only. Bedrock: IAM policies scoped to specific model ARNs. The pattern is the same; the verbs differ.

Rotate on a schedule

90 days is the boring default. Pin a calendar event or schedule a task. The mechanism: provision the new key, deploy code that reads both old and new for a window, switch the prod env var to the new key, watch the dashboard to confirm zero traffic on the old key, then revoke. This pattern is called blue/green secrets; we cover it in Rotating API keys safely.

Detect leaks

GitHub's secret scanning catches OpenAI, Anthropic, and HF tokens automatically and notifies the provider, which auto-revokes. That's a great backstop, not a primary defense — by the time GitHub catches it, the key is already in the public timeline of someone's commit history. Treat any key that touches a public repo as compromised, even if scanners auto-revoke it.

Add a pre-commit hook (gitleaks, trufflehog) so leaked keys never reach git push in the first place.

Watch your bill

The first sign of a key compromise is usually the bill, not a security alert. Set up provider-side spend alerts (every provider has them). Set them low — 25%, 50%, 75% of monthly cap. If you get the 25% alert on day 2 of the month, something is off.

Don't log the key

It sounds obvious. It happens constantly. Express's default error handler will dump the request headers including Authorization if you let an exception bubble. Sentry will too. Most APM tools do, unless you explicitly redact authorization and x-api-key. KeyForge itself was designed around this — the proxy never logs headers, never persists keys, never echoes them in error messages. You should hold your own code to the same bar.

What to do next

Set up rotation, watch provider status so you can tell apart "my key is broken" from "OpenAI is having a day", and consider an in-pipeline rate-limit calculator to detect rogue clients early.