Hongpei Li
  • Bio
  • Research
  • Software
  • Blog
  • Experience
  • Gallery
  • Recent & Upcoming Talks
    • Example Talk
  • Gallery
    • Drawings
    • Models
    • Holy Hill
    • Chicago
    • Wuhan
    • Xixinan
    • Chengkan
  • Research
    • GPU-Accelerated Conic Quadratic Programming with Local Linear Convergence under Strict Complementarity
    • A Curvature-Aware Rank-Adaptive Distributed Augmented-Lagrangian Solver for Large-Scale SDPs
    • OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention
    • PDHCG-II: An Enhanced Version of PDHCG for Large-Scale Convex QP
    • D-PDLP: Scaling PDLP to Distributed Multi-GPU Systems
    • BenLOC: A Benchmark for Learning to Configure MIP Optimizers
    • PDHCG: A Scalable First-Order Method for Large-Scale Competitive Market Equilibrium Computation
    • A Restarted Primal-Dual Hybrid Conjugate Gradient Method for Large-Scale Quadratic Programming
    • FMIP: Joint Continuous-Integer Flow For Mixed-Integer Linear Programming
    • OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training
    • Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning
  • Blog
    • Accelerating Nonlinear Programming on GPUs
    • Use Customized CUDA kernel in your PyTorch Code
    • Configure Your Server (Part II) -- Python Environment
    • Unlocking Your GPU's Potential--A Guide to CUDA Samples and Performance Testing
    • Configure Your Server (Part I) -- SSH and Git
  • Gallery_pics
    • Angels&demons
    • Archer_elf
    • bilibili
    • character_design
    • christmas_trees
    • emblem_practice
    • Gundam_model_free
    • Gundam_model_snow
    • High_school_class_emblem
    • icon-design
    • Miku
    • New_Year_Greetings
    • New_Year_Greetings_2021
    • practice
    • practice_armour
    • practice_cha
    • practice_face
    • practice_ugly
    • self_logo
    • Silhouette
    • Zodiac_mecha-Chicken
    • Zodiac_mecha-Rabbit
  • Projects
  • Projects
    • DRL4IPPS
    • PDHCG
    • ML4MOC
    • PDHCG-Net
    • LSTM-based Quasi-Newton
  • Experience
  • Teaching
    • Learn JavaScript
    • Learn Python
  • Software

OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training

Jan 1, 2025·Hongpei Li, Han Zhang, Huikang Liu, Dongdong Ge, Yinyu Ye · 0 min read
PDF Cite
Type
Manuscript
Publication
arXiv
Last updated on Jan 1, 2025
← FMIP: Joint Continuous-Integer Flow For Mixed-Integer Linear Programming Jan 1, 2025
Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning Sep 1, 2024 →

Email · GitHub · Google Scholar