Name: Access AI Models Anywhere: Scaling AI Traffic With Envoy AI Gateway - Dan Sun, Bloomberg & Takeshi Yoneda, Tetrate.io
Start: 2025-06-16T14:10:00+0900
End: 2025-06-16T14:40:00+0900

16-17 June
Learn More and Register to Attend

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon Japan 2025 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Japan Standard Time (UTC+9:00). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change and session seating is available on a first-come, first-served basis.

Monday June 16, 2025 14:10 - 14:40 JST

Level 1 | Pegasus Ballroom A+B1

As Generative AI adoption increases, organizations face accelerating challenges in deploying, scaling, and managing access to diverse AI models across cloud and on-prem environments. Envoy AI Gateway utilizes Envoy Proxy’s powerful filter architecture and extensibility through ext-proc to deliver key features such as centralized credential management, intelligent model routing, and LLM token usage control.
As the first CNCF-backed open source AI gateway, Envoy AI Gateway is built on top of a robust, high performance Envoy Gateway to help democratize AI infrastructure for organizations of all sizes.

In this talk, we will dive into the architecture of Envoy AI Gateway to learn how it extends Envoy’s capabilities to efficiently manage AI-driven workloads for enterprise needs, while providing robustness, scalability, and adaptability in the rapidly-changing generative AI landscape. We will also showcase a demo of an AI agent seamlessly accessing models anywhere through a unified API.

Speakers

Dan Sun

Team Lead, Bloomberg

Dan Sun is a software engineer team lead at Bloomberg. He is the co-founder and maintainer of KServe, an open source Serverless AI inference platform project. He is a co-founder of the Envoy AI Gateway project.

Takeshi Yoneda

Open Source Software Engineer, Tetrate.io

Takeshi Yoneda is a software engineer at Tetrate.io, with contributions to numerous open source projects, including compilers and network proxies. He is a co-founder of the Envoy AI Gateway project.

Monday June 16, 2025 14:10 - 14:40 JST
Level 1 | Pegasus Ballroom A+B1

AI + ML

Content Experience Level Any
Presentation Language English

KubeCon + CloudNativeCon Japan 2025

Dan Sun

Takeshi Yoneda

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!