Cache the Meaning, Not the Message: Building Faster Chatbots with Valkey
20 de noviembre, 14:35Escenario verde
“How do I return a product?” and “What’s the return process?” are different messages that often need the same answer. In a WhatsApp-style chatbot, repeatedly sending these questions to a large language model adds latency…
“How do I return a product?” and “What’s the return process?” are different messages that often need the same answer. In a WhatsApp-style chatbot, repeatedly sending these questions to a large language model adds latency and cost. Exact-match caching misses the connection. Can we cache by meaning instead? In this session, we’ll walk through a conversational application that uses Valkey with vector-search support as a semantic cache. We’ll turn incoming questions into embeddings, search for similar previously answered requests, and decide whether to reuse a response or invoke the model. A live demo will follow the request path from incoming message to reply, comparing behavior with and without caching. The harder question is deciding when a similar question deserves a different answer. We’ll explore how conversation context, user-specific information, and changing business policies affect cache eligibility. You’ll see how similarity thresholds, expiration, and explicit invalidation help prevent incorrect reuse, and why personalized requests sometimes need to bypass the cache entirely. You’ll leave with a reference architecture and an evaluation approach for measuring latency, model-call savings, and incorrect cache hits, so you can assess whether semantic caching improves your chatbot without compromising answer quality. Sobre Roberto Luna Rojas: Roberto Luna Rojas is a Sr Developer Advocate for Valkey based in the USA, originally from Mexico. He tries to get the most out of in-memory databases one bit at the time, been around developing software since the mid 90s focused on performance and scalability. When not in front of the computer, he loves spending time with his family, listening to music, and watching movies, TV shows, and sports.
Este taller práctico explora el uso de caché con Valkey a través de tres casos de uso clave: optimización de consultas a bases de datos, gestión de sesiones y tablas de clasificación en tiempo real. Los participantes apr…
Este taller práctico explora el uso de caché con Valkey a través de tres casos de uso clave: optimización de consultas a bases de datos, gestión de sesiones y tablas de clasificación en tiempo real. Los participantes aprenderán a cachear operaciones de base de datos costosas, llamadas a APIs externas y configuraciones JSON, al mismo tiempo que construyen almacenamiento de sesiones escalable y tablas dinámicas de clasificación.
Más allá de la implementación, el taller enfatiza desafíos críticos del caché, incluyendo invalidación de caché, prevención de stampede, gestión de memoria y consistencia de datos. Los asistentes experimentarán de primera mano errores comunes y aprenderán estrategias comprobadas para cache warming, optimización de TTL y monitoreo en producción.
Los participantes se llevarán ejemplos de código funcionales, patrones de arquitectura y una comprensión clara de los trade-offs del caché que podrán aplicar de inmediato en sus aplicaciones. Sobre Roberto Luna Rojas: Roberto Luna Rojas is a Sr Developer Advocate for Valkey based in the USA, originally from Mexico. He tries to get the most out of in-memory databases one bit at the time, been around developing software since the mid 90s focused on performance and scalability. When not in front of the computer, he loves spending time with his family, listening to music, and watching movies, TV shows, and sports.