As AI Agents become mainstream development tools, every conversation turn sends the full message history to the LLM provider. This poses a significant challenge for DLP (Data Loss Prevention) systems: the same content gets scanned over and over, creating massive performance waste.
The Problem: O(N²) Scanning Overhead
The LLM API conversation format requires every request to carry the complete conversation history in its messages[] array. A 50-turn Agent conversation means:
- Turn 1: scan 1 message
- Turn 2: scan 2 messages
- Turn 3: scan 3 messages
- …
- Turn 50: scan 50 messages
Total scans: 1 + 2 + 3 + … + 50 = 1,275 message scans, of which only 50 contain new content. The remaining 1,225 are redundant scans of historical messages.
This doesn’t just waste compute. More critically, it generates duplicate alerts and audit records, drowning security teams in noise.
Current Solution: Message-Level Hash Caching
Bastion’s current approach is message-level hash caching. The idea is straightforward: compute a SHA-256 hash for each message’s content, use it as a cache key, and store the DLP scan results.
The workflow:
- Parse the
messages[]array from the incoming API request - Compute a content hash for each message
- Cache hit → return cached scan results, marked as “known findings”
- Cache miss → run the actual scan, store results in cache, marked as “new findings”
- Only trigger alerts and audit records for “new findings”
With this approach, a 50-turn conversation only scans 50 messages total (each scanned exactly once), reducing complexity from O(N²) to O(N). In practice, the 50th turn achieves a 98% cache hit rate — only the latest message needs scanning.
Edge Cases
Message-level caching solves the most prominent performance issue, but some edge cases remain.
Cache granularity. The cache operates at whole-message level. If an Agent appends a small amount of content to a reply (e.g., tool call results), the entire message hash changes and triggers a rescan. This reduces hit rates in tool-use-heavy scenarios.
Cross-session isolation. The cache is an in-process LRU cache that’s lost on restart. Different user sessions share the same cache instance — reasonable for a single-user gateway, but multi-tenant deployments would need isolation.
Streaming response timing. For streaming responses, DLP scanning happens after the complete response is assembled. If sensitive content is detected during streaming, the data has already been partially delivered to the client.
Future Directions
Chunked hashing. Split message content into fixed-size chunks (e.g., 512 bytes) and hash each independently. This way, even if content is appended to a message, earlier chunks still hit the cache. The tradeoff is increased implementation complexity and the need to handle cross-chunk pattern matching.
Persistent cache. Write cache entries to SQLite so they survive restarts. For long-running Agent tasks that may span multiple gateway restarts, this avoids full-scan warmup on the first request.
Session-aware scanning. Maintain session state at the proxy layer, tracking the last scanned message index per session. When a new request arrives, only scan messages after that index, skipping the history entirely. This is more efficient than hash caching but requires accurate session boundary tracking.
Incremental redaction. For users in redact mode, cache the redacted version of content upon first detection. Subsequent requests use the pre-redacted version directly, avoiding repeated regex matching and replacement.
Takeaway
Repeated DLP scanning in Agent mode is fundamentally an “incremental processing with history” problem. Message-level hash caching is a simple and effective first step, reducing O(N²) to O(N). But as Agent conversations grow longer and tool calls become more frequent, finer-grained caching strategies and session-aware scanning will become necessary next steps.
Security shouldn’t be a bottleneck for the AI development experience. A good DLP system should be like Git — you know it’s protecting you, but you barely notice it’s there.