MLCommons Chakra: Standardized Execution Traces for AI Benchmarking and Co-design

This episode examines Chakra, an MLCommons effort to define one open, graph-based execution-trace format. The format is meant to carry a distributed AI workload from production observation through benchmark replay to hardware simulation, without exposing a proprietary model. It explains how a trace is a per-device directed acyclic graph of compute, memory, and communication nodes linked by control, data, and synchronization dependencies. Process groups encode tensor, pipeline, data, and expert parallelism, and the traces carry no weights or data. The discussion separates post-execution traces, which are recorded from real runs and carry that machine's timings, from pre-execution traces, which are portable but need timing from elsewhere. It also covers how PyTorch's execution graph observer and Kineto profiler are merged to capture traces, and how replay, simulation (ASTRA-sim), and emulation consume them. The hosts treat the ecosystem story as strong but press on whether the evidence supports the paper's boldest claims, which makes this useful for anyone interested in benchmarking and co-designing AI infrastructure.

Sources:

1. MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces — Srinivas Sridharan, Theodor-Adrian Badea, Andy Balogh, Bradford M. Beckmann, Brian Coutinho, Louis Feng, Sheng Fu, Sanshan Gao, Mehryar Garakani, Taekyung Heo, David Kanter, Josh Ladd, Ziwei Li, Winston Liu, Changhai Man, Dan Mihailescu, Spandan More, Joongun Park, Ashwin Ramachandran, Vinay Ramakrishnaiah, Saeed Rashidi, Vijay Janapa Reddi, Puneet Sharma, Phio Tian, William Won, Hanjiang Wu, Huan Xu, Jinsun Yoo, Tushar Krishna, 2026

http://arxiv.org/abs/2605.11333

2. ASTRA-sim: Enabling SW/HW Co-Design Exploration for Distributed DL Training Platforms — Saeed Rashidi, Srinivas Sridharan, Sudarshan Srinivasan, Tushar Krishna, 2020

https://scholar.google.com/scholar?q=ASTRA-sim%3A+Enabling+SW%2FHW+Co-Design+Exploration+for+Distributed+DL+Training+Platforms

3. ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale — William Won, Taekyung Heo, Saeed Rashidi, Srinivas Sridharan, Sudarshan Srinivasan, Tushar Krishna, 2023

https://scholar.google.com/scholar?q=ASTRA-sim2.0%3A+Modeling+Hierarchical+Networks+and+Disaggregated+Systems+for+Large-model+Training+at+Scale

4. Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces — Srinivas Sridharan