KubeCon + CloudNativeCon Japan 2025

In this presentation, we will unveil the secrets and journey to enable an IBM's AI Accelerator in Kubernetes. We utilize wide range of tools and frameworks such as device plugin, custom scheduler, metrics exporter, webhooks and custom resources together to satisfy real requirements from various stakeholders - general users, cluster administrators and driver/runtime developers. Our device plugin and custom scheduler can accept special preference such as topology-aware allocation to enable RDMA, and webhook-based validator guides them to follow specification changes. To sync up allocation status among the components, we carefully defined a custom resource after performance estimation. From developer perspective, we provide various debug-purpose capabilities: for example, device allocation by PCI address for inspection, and pseudo device mode to achieve non-real-device test. In addition, multi-architecture support gives freedom of platform choice to all of the participants.

Pods are PENDING?!? Ugh, none of the latest GPUs are available. But there are tons of older ones! If only you could tell Kubernetes “use the best GPU available, as long as it has 20GB+”...(enter: DRA).

Kubernetes’ Dynamic Resource Allocation (DRA) system, beta since 1.32, allows variations on which GPUs get allocated to Pods. You can write a flexible spec so that when the Deployment scales, Pods can land on whatever nodes have available GPUs. DRA works even if you need different numbers of devices for different hardware! This enables a new level of utilization and efficiency, saving your organization real money.

Combined with an advanced Node Autoscaler like Google’s Custom Compute Classes or Karpenter, you can spin up more VMs with whatever GPUs are available - or the most economical - all for a single Deployment. Scaling is simpler and more reliable, and your workload can scale even when your preferred type of GPU is stocked out.

Come learn how, and see it in action with a demo!

As Generative AI adoption increases, organizations face accelerating challenges in deploying, scaling, and managing access to diverse AI models across cloud and on-prem environments. Envoy AI Gateway utilizes Envoy Proxy’s powerful filter architecture and extensibility through ext-proc to deliver key features such as centralized credential management, intelligent model routing, and LLM token usage control.
As the first CNCF-backed open source AI gateway, Envoy AI Gateway is built on top of a robust, high performance Envoy Gateway to help democratize AI infrastructure for organizations of all sizes.

In this talk, we will dive into the architecture of Envoy AI Gateway to learn how it extends Envoy’s capabilities to efficiently manage AI-driven workloads for enterprise needs, while providing robustness, scalability, and adaptability in the rapidly-changing generative AI landscape. We will also showcase a demo of an AI agent seamlessly accessing models anywhere through a unified API.

Cold-start delays for GPU-heavy GenAI apps like ComfyUI aren’t just about speed—they’re architectural failures. While others optimize incremental steps, we eliminate entire phases: no image downloads, no layer extraction, no redundant model copies.

We introduce a radical Kubernetes-native pattern: Direct-to-GPU streaming via FUSE-mounted object storage (S3/GCS), bypassing legacy container workflows. By rearchitecting the snapshotter to support seekable, on-demand FUSE streaming, we enable:

- Instant container boot: Models/CUDA dependencies mount directly from object storage, avoiding registry bottlenecks (40MB/s → 900MB/s throughput)
- Zero-extraction overhead: Layers load incrementally via range-optimized fetches, eliminating Zstd unpack/copy latency
- True cold start elimination: ComfyUI pods activate in 90s (vs. 8+ mins) by co-locating model mounting and inference prep

We’ll dissect a live ComfyUI deployment using 100% OSS primitives to hack container internals in the session.

Panel Discussion - As AI continues to reshape industries, organizations face mounting pressure to scale AI systems responsibly while addressing challenges in efficiency, sustainability, and trust. This panel convenes leading experts to discuss how cloud-native technologies and CNCF projects are paving the way for scalable, ethical, and resource-efficient AI. Attendees will gain actionable insights into optimizing AI workflows, reducing environmental impact, and ensuring transparency in AI decision-making. From leveraging open-source tools to implementing cost-effective and ethical AI practices, this session will equip you with the knowledge to build AI systems that are both innovative and responsible. Discover how to harness the power of cloud-native ecosystems to drive AI transformation without compromising on sustainability or trust.
AI/ML engineers and data scientists looking to scale AI systems in cloud-native environments.

As organizations scale their model training workloads in cloud-native environments, they face significant data processing and storage challenges: managing massive training datasets across distributed storage systems while ensuring optimal I/O performance. While Kubernetes excels at compute orchestration, the increasing distribution of data across multiple storage backends creates bottlenecks that impact training performance and infrastructure costs.

This presentation introduces a Kubernetes-native distributed caching system that utilizes NVMe storage to overcome data locality challenges. Haoyuan Li will also share real-world, large-scale production use cases to show how this architecture lowers data infrastructure costs, increases GPU utilization, and enables workload portability to navigate GPU scarcity challenges.

Projects both within and outside of the CNCF ecosystem provide additional capabilities for Kubernetes clusters. These "addons" become integral to the functioning of our clusters, but we don't often talk about their impact as a whole or managing them holistically as first-class citizens.

We know there are barriers to keeping things like addons up-to-date and that it can be difficult to get buy-in for allocating the time and resources for updating something that is working just fine (for now), especially if you’re multiple major versions behind. In this session we will help you understand and articulate the benefits of catching up and keeping addons updated and how to be proactive moving forward. You will walk away with some tools and strategies for navigating the complexity of the addon ecosystem and make the process as painless as possible. You will be able to create an action plan for improving the stability and security of your clusters and share that with stakeholders.

You probably have more than one cluster and there is a decent chance you are using Argo CD. Additionally, it is quite likely that you have a few other variations of Kubernetes cluster lists. We posit that writing glue code to stitch together these clusters lists is not an awesome use of your time. Thankfully the good folks in SIG-Multicluster built this super cool api for cluster lists, cluster profile/cluster inventory! We are going to show you how to use said fancy new list with Argo CD along with other multi-cluster tools across Kubernetes clusters hosted by different providers. There will be demos. Possibly Mustaches. And a decent amount of awful puns. So come on down to bear witness to some sweet multi-cluster abstractions that will surely get your heart rate up.

Platform Engineering enables developers to focus on business value-aligned tasks by providing internal developer platforms (IDPs) that automate non-essential tasks. Kubernetes is widely used as a foundation for IDPs thanks to its scalability and flexibility.

However, Kubernetes was designed as a general workload orchestrator, not a platform component. As a result, IDP builders must integrate additional Cloud Native technologies and customizations, which can create scalability bottlenecks. At LY Corporation, his team has developed a Kubernetes-based, multi-tenant IDP running over 140K pods, and they faced such scalability challenges.

In this session, he will discuss scalability bottlenecks faced in the IDP, including observability pipelines, access control, etc. He will also explore scaling strategies for IDPs and how they address real-world scalability issues. By the end of this session, you will gain deeper insights into scalability challenges from a platform builder’s perspective.

BGP (Border Gateway Protocol) is increasingly being used to connect Kubernetes networking with the rest of the IT estate, especially in large-scale and on-premises environments. However, the complexity of many network architectures requires users to have more flexibility and control over how they deploy BGP. Based on the experience at Preferred Networks, this session introduces key BGP peering patterns that enhance Kubernetes networking while maintaining operational simplicity, including:

1. The Sidecar BGP Peering Pattern: A method of running a dedicated BGP speaker alongside Kubernetes networking components, balancing automation with fine-grained control.
2. Native Routing over IP Clos Networks – A tunneling-free approach that integrates Kubernetes with large-scale BGP-based datacenter fabrics for better performance.

Based on real-world experience, we will share best practices and lessons learned, helping attendees design scalable and reliable Kubernetes networking with BGP.

It’s a not-so-dirty little secret that the technology that we so heavily rely on comes at an environmental cost. As technology becomes more complex, we need Observability to better understand it, and yet this too contributes to an increasing global tech carbon footprint.

Luckily, we have tools at our disposal that can help us understand our carbon footprint, and take mitigating actions. Tools like Kepler, Kube-Green, and green reviews.

In this talk, attendees will learn about Kepler, Kube-Green and green reviews. They will learn how to use these tools to make tweaks to their OpenTelemetry Collectors and other Kubernetes infrastructure. This will therefore keep systems observable while keeping the environment in mind.

OpenMetrics (OM) had a wild journey: it started as a project to standardize the Prometheus exposition format, and it became an entirely separate CNCF Incubating project. Even though the project had high maturity, it struggled for years to find tools to comply with the first version of the spec. Finally, in 2025, it was incorporated back into the Prometheus Github organization so Prometheus developers could lead the efforts for OM 2.0.

In this talk, Arthur, a Prometheus maintainer and OpenMetrics contributor, will walk you through the main challenges that tools like Prometheus and OpenTelemetry face when trying to comply with OpenMetrics 1.0 and how the community plans to address these challenges in OM 2.0.

The audience will also learn how changing an exposition format can make Prometheus and OpenTelemetry-Collector more memory-efficient while making their specifications easier to translate into each other!

Join us on an anime-powered journey to explore the power of eBPF in cloud native. Inspired by Sailor Moon and Dragon Ball GT, this talk breaks down how eBPF enhances observability, networking, and security, using the Sailor Guardians and Gohan to make eBPF approachable. We’ll break down topics like real-time observability by comparing it to psychic flames.
And just as unchecked power in anime can lead to chaos, eBPF’s immense capabilities need to be used wisely. Drawing from Dragon Ball GT's arc where the Dragon Balls become a threat, we’ll examine best practices for safe deployment of eBPF (e.g., LSM, verifier constraints, Linux Capability restrictions). Spoiler alert: our Sailor Guardians and Gohan will come out victorious, worry not!

By the end, you’ll gain a clear understanding of how to wield eBPF safely and effectively in cloud native environments, no Super Saiyan transformation required!