Tony Zhao
  • Bio
  • Papers
  • Experience
  • Game Projects
  • Recent & Upcoming Talks
    • Example Talk
  • Publications
    • Efficient LLM Serving on Hybrid Real-time and Best-effort Requests
    • OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training
    • Cdmpp: A device-model agnostic framework for latency prediction of tensor programs
    • Llm-pq: Serving llm on heterogeneous clusters with phase-aware partition and adaptive quantization
    • POSTER: LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
    • QSpec: Speculative Decoding with Complementary Quantization Schemes
    • QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
    • Adaptive message quantization and parallelization for distributed full-graph gnn training
    • CryptoArcade: A Cloud Gaming System With Blockchain-Based Token Economy
    • Synchronization in games sound: an audiovisual study on player experience and performance
    • CloudArcade: A blockchain empowered cloud gaming system
    • An example preprint / working paper
    • An example journal article
    • An example conference paper
  • Projects
  • Blog
    • ๐ŸŽ‰ Easily create your own simple yet highly customizable blog
    • ๐Ÿง  Sharpen your thinking with a second brain
    • ๐Ÿ“ˆ Communicate your results effectively with the best data visualizations
    • ๐Ÿ‘ฉ๐Ÿผโ€๐Ÿซ Teach academic courses
    • โœ… Manage your projects
  • Experience
  • Teaching
    • Learn JavaScript
    • Learn Python
  • Projects
    • CookingMasterSimulator
    • TransHome
    • Shadow Ticker

Efficient LLM Serving on Hybrid Real-time and Best-effort Requests

Jan 1, 2025ยท
Wan Borui
,
Zhao Juntao
,
Jiang Chenyu
,
Guo Chuanxiong
,
Wu Chuan
ยท 0 min read
Cite
Type
Journal article
Publication
arXiv preprint arXiv:2504.09590
Last updated on Jan 1, 2025
OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training Jan 1, 2025 โ†’

ยฉ 2025 Me. This work is licensed under CC BY NC ND 4.0

Published with Hugo Blox Builder โ€” the free, open source website builder that empowers creators.