Amber Huffman, Google & Cynthia Hsu, Sandisk | theCUBE + NYSE Wired: GSA Series

In this episode of the GSA Series – Women in Leadership, John Furrier hosts a discussion with Amber Huffman from Google and Cynthia Hsu from SanDisk, focusing on “Memory and Data Movement for AI.” The conversation delves into how evolving standards, such as NVMe, JEDEC, and ONFI, along with advancements in infrastructure, particularly power and cooling, are critical for optimizing time-to-market in hyperscale settings. Hsu shares insights on how flash memory designs are adapting to meet the demands of AI workloads, specifically during training and inference. Key highlights include the introduction of SanDisk’s High Bandwidth Flash (HBF), designed to deliver increased capacity and bandwidth necessary for complex AI models. Huffman underscores the importance of software/firmware interfaces and the role of the Open Compute Project in fostering collaboration and standardization across the industry. The discussion also touches on memory-centric AI patterns and the imperative of overcoming PCIe bottlenecks to optimize data flows in AI systems.

• Understanding the impact of memory design on AI workloads and data movement.
• The significance of established standards such as NVMe and JEDEC in infrastructure development.
• Cynthia Hsu presents SanDisk's High Bandwidth Flash technology and its benefits for AI.
• Amber Huffman emphasizes the critical link between software interfaces and hardware standards.
• Challenges and solutions related to PCIe bottlenecks in AI data flows.
• The concept of creating agile, fungible data centers for better resource allocation.
• Exploring the future of memory capabilities and the role of AI in chip design.
• Insights into the development of renewable hardware and its implications for data centers.