AI-DRIVEN DESIGN AND OPTIMIZATION OF HETEROGENEOUS CHIPLET SYSTEMS FOR SERVER-SCALE AI WORKLOADS

Harsh Sharma

doi:10.7273/000008360

We are at the cusp of a revolution driven by deep generative models, also referred to as foundation models (FMs), such as ChatGPT for text and Stable Diffusion for images. These models, containing hundreds of billions of parameters trained on massive unlabeled datasets, have redefined the computational scale of modern artificial intelligence. In the near future, fine-tuning foundation models on domain-specific and privacy-sensitive data will emerge as a central challenge. The dominant architecture enabling this new era is the Transformer, whose scalability and generalization capabilities have made it indispensable across natural language processing, computer vision, and multi-modal learning. However, the exponential growth in model size and computational demand has outpaced the capabilities of monolithic System-on-Chip (SoC) designs. The yield, compute, and power-delivery limits of a monolithic-die have led to chiplet-based architectures, where multiple smaller dies called chiplets are interconnected through an interposer. This architectural paradigm provides modularity, higher yield, and cost efficiency but also introduces new design challenges and opportunities related to communication bottlenecks, defect tolerance, thermal balance, and heterogeneous integration. This dissertation aims to address many of these conflicting challenges through a series of analytical, architectural, and optimization frameworks proposed that in conjunction enable scalable, reliable, and energy-efficient heterogeneous multi-chiplet systems for deep learning acceleration. First, SWAP introduces a server-scale communication-aware framework that co-optimizes chiplet and link placement for server-scale deep learning workloads. Building on this foundation, Florets for chiplets proposes a dataflow-aware Network-on-Interposer (NoI) using space-filling curves (SFCs) to align computational and communication locality, achieving high performance and robustness even under defective chiplets. While space filling curve-based dataflow optimization enhances locality for CNN inference, homogeneous ReRAM based systems face fundamental scalability limits due to limited rewrites when inferencing Transformer models with large number of parameters. Transformer models consist of heterogeneous computational kernels such as dynamic matrix multiplications, attention, and feed forward, each exhibiting distinct data access characteristics and memory footprints. Executing all these kernels on a single type of device leads to inefficiencies due to repeated memory rewrites, and the inability to balance compute and memory intensive operations efficiently. To address these challenges, a heterogeneous chiplet architecture is proposed that integrates ReRAM based processing in memory units with streaming multiprocessors and memory controllers connected through a co-optimized communication network. This SM–MC network enables efficient interaction between compute centric and memory centric kernels, reducing interconnect overhead while sustaining high throughput. The PIM-based ReRAM chiplets are employed to accelerate static computational kernels. By tailoring the hardware specialization to each computational phase, the architecture achieves significant improvements in latency, energy efficiency, and scalability for LLM inference, thereby extending the dataflow aware design principles beyond spatial locality toward heterogeneous execution. To exploit device- and architecture-level heterogeneity, HeMu introduces a multi-objective optimization framework that integrates diverse PIM technologies (SRAM, ReRAM, and their varying architectures) in a unified 2.5D platform. It determines optimal chiplet configurations, achieving an order of magnitude higher energy-efficiency improvement over homogeneous baselines. Building on this, HetOU extends the optimization to the operation-unit level, dynamically managing wordline and bitline activations in PIM crossbars for fine-grained energy control. Finally, paving a way towards next generation of multi-chiplet architectures, an architecture-package co-design framework for glass-interposer-based systems is proposed. The architecture bridges architecture and packaging, ensuring mechanical integrity and sustained high performance within the allowable thermal limits. Collectively, these contributions establish a unified design methodology that spans communication, architecture, device, and package co-optimization, forming a complete stack for next-generation heterogeneous computing systems. The proposed frameworks not only improve scalability, performance, and thermal balance but also lay the foundation for adaptive, ML-driven design exploration which can be exploited in future computing paradigms.

AI-DRIVEN DESIGN AND OPTIMIZATION OF HETEROGENEOUS CHIPLET SYSTEMS FOR SERVER-SCALE AI WORKLOADS

Files and links (1)

Abstract

Metrics

Details