Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO
AI infrastructureGPU inferenceAgent experienceDeveloper experienceElastic scalingSandboxingSpeculative decodingAuto endpointsDistributed trainingOverlay networkingRDMACompute strategyAuto researchLLM inferenceManaged agentsMeta harnessesPython SDKTypeScript SDKAI agentsCloud capacity
In this episode, Akshat Bubna, CTO of Modal, discusses the evolution of AI infrastructure tailored for agent experience, highlighting Modal's journey from a runtime platform to a specialized cloud for AI workloads. They explore challenges in scaling inference, sandboxing, speculative decoding, and the unique demands of agent-native primitives in modern AI applications.