High-signal thoughts on infrastructure, artificial intelligence, and the intricacies of building resilient systems at scale.
Prompt injection is one of the most important security risks in LLM-powered applications, especially if you're building AI agents, RAG systems, or tools that can execute actions on behalf of users.
Read moreLearn how semantic caching and AI gateways reduce latency, cut API costs, and add resilience to your LLM-powered applications.
Time-to-first-token and total generation time are different products. Streaming, deadline propagation, and why your timeout budget is probably wrong.
Reducing latency in large vector indexes: quantization, graph pruning, and why the page cache is usually a better tiering strategy than the one you were about to write.