Prompt injection as a structural vulnerability: lessons from buffer overflow and SQL injection

Hypothesis

Buffer overflow is built into C's memory model — no bounds checking, pointer arithmetic with direct memory access. SQL injection is built into string-based query construction — no architectural separation between query structure and user-supplied content. Prompt injection may be structurally analogous: transformer attention treats instruction tokens and data tokens identically. There is no architectural boundary between "command space" and "content space."

The pattern

In each case, the vulnerability was not a bug — it was a consequence of a core design decision:

• C: power through raw memory access → buffer overflow
• SQL: flexibility through string concatenation → injection
• LLMs: generality through uniform token attention → prompt injection

And in each case, surface-level defenses proved insufficient:

• strcpy() safety guidelines did not stop buffer overflows — memory-safe languages (Rust, Ada) did
• Input sanitization did not stop SQL injection — parameterized queries (structural separation) did

The prediction

If the analogy holds, RLHF and input filtering will not solve prompt injection. The durable fix requires architectural separation analogous to parameterized queries.

Candidate approaches

1. Cryptographically tagged instruction tokens — system prompt tokens carry unforgeable provenance markers that attention cannot ignore
2. Dual-channel architecture — system prompt processed in a separate forward pass; outputs merged at a privileged layer inaccessible to content tokens
3. Capability-based sandboxing at the agent level — the model never sees raw external content; a sandboxed retrieval layer mediates access

Falsification

This hypothesis is false if: (a) sufficiently capable RLHF-trained models demonstrate robust zero-shot resistance to novel prompt injection without architectural changes, or (b) a training-only approach achieves the same error rates as parameterized queries achieved for SQL injection (near-zero at scale).

Open question

What is the LLM-equivalent of parameterized queries — a protocol-level intervention that separates instruction structure from content at the point of construction, not at the point of filtering?