KubeCon + CloudNativeCon Europe 2025

How can you enhance your load balancer with AI?

As LLMs become ubiquitous, managing AI traffic requires innovative approaches. We'll show how to use HAProxy's Stream Processing Offload Engine (SPOE) to create an intelligent AI gateway.

This proof-of-concept demonstrates how to evaluate prompts in transit, adding functionality beyond basic routing. You'll learn to:
Calculate tokens in real time.
Detect PII before it reaches your LLMs.
Evaluate for potential threats or abuse.
Set up the whole system with real examples you can use.

This demo explores extending AI Gateway functionality beyond conventional implementations. By using AI to secure AI traffic, we address the new challenges organizations face when scaling LLM deployments.

Join us to see how a load balancer can be enhanced with AI capabilities for more intelligent, secure gateways.

In order to facilitate networking and business relationships at the event, you may choose to visit a third party’s booth or access sponsored content. You are never required to visit third party booths or to access sponsored content. When visiting a booth or participating in sponsored activities, the third party will receive some of your registration data. This data includes your first name, last name, title, company, address, email, standard demographics questions (i.e. job function, industry), and details about the sponsored content or resources you interacted with. If you choose to interact with a booth or access sponsored content, you are explicitly consenting to receipt and use of such data by the third-party recipients, which will be subject to their own privacy policies.