Heterogenous Cluster

LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
Adaptive Quantization and Phase-Aware Partition for LLM serving atop heterogenous cluster.
Juntao Zhao, Borui Wan, Yuanghua Peng, Haibin Lin, Chuan Wu
PDF Cite Code Poster DOI Full paper
Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Precision Recovery for Heteorgenous Cluster Training
Juntao Zhao, Borui Wan, Yuanghua Peng, Haibin Lin, Yibo Zhu, Chuan Wu
Cite Code DOI