Liang Zhang

About

I received my Ph.D. in computer science from ETH Zurich, jointly advised by Niao He from ETH Zurich and Michael Muehlebach from the Max Planck Institute for Intelligent Systems. I am affiliated with the Max Planck ETH Center for Learning Systems. I also closely collaborate with Sewoong Oh from University of Washington and Kiran Koshy Thekumparampil from Amazon.

Prior to this, I received my master's degree in computer science from ETH Zurich in 2021 and bachelor's degree in physics from Peking University in 2019.

My research focuses on the intersection between optimization and machine learning, and I have a keen interest in developing efficient optimization algorithms for trustworthy machine learning. My current research targets the efficient training of large language models (LLMs), particularly on:

• Zeroth-order optimization for memory-efficient LLMs fine-tuning; see DPZero [ICML'24] on private and memory-efficient LLMs fine-tuning, [NeurIPS'25] on implicit regularization of zeroth-order optimization towards flat minima, and [ICML'26] on the edge of stability in zeroth-order optimization.
• Parameter-efficient fine-tuning (PEFT) such as LoRA; see [NeurIPS'24] on balancedness of LoRA matrix, NoRA [ICLR'25] on LoRA initialization, and PoLAR [NeurIPS'25] on how rank inefficiency in LoRA can be resolved.

Publications

Journal Article

• A Hessian-Aware Stochastic Differential Equation for Modelling SGD.
  Xiang Li, Zebang Shen, Liang Zhang, Niao He.
  Math. Program., 2026 [Springer] [arXiv] [HiLD@ICML'24]
• Primal Methods for Variational Inequality Problems with Functional Constraints.
  Liang Zhang, Niao He, Michael Muehlebach.
  Math. Program., 2025 [Springer] [arXiv]

Conference Paper

• Zeroth-Order Optimization at the Edge of Stability.
  Minhak Song, Liang Zhang, Bingcong Li, Niao He, Michael Muehlebach, Sewoong Oh.
  ICML, 2026 [arXiv] [Sci4DL@ICLR'26 (Oral)]
• SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference.
  Hao Ma, Melis Ilayda Bal, Liang Zhang, Bingcong Li, Niao He, Melanie Zeilinger, Michael Muehlebach.
  ICML, 2026 [arXiv] [CPAL'26]
• On the Benefits of Weight Normalization for Overparameterized Matrix Sensing.
  Yudong Wei, Liang Zhang, Bingcong Li, Niao He.
  ICLR, 2026 [OpenReview] [arXiv] [OPT-ML@NeurIPS'25]
• ConMeZO: Adaptive Descent-Direction Sampling for Gradient-Free Finetuning of Large Language Models.
  Lejs Deen Behric, Liang Zhang, Bingcong Li, Kiran Koshy Thekumparampil.
  AISTATS, 2026 [OpenReview] [arXiv] [ES-FoMo@ICML'25]
• Zeroth-Order Optimization Finds Flat Minima.
  Liang Zhang, Bingcong Li, Kiran Koshy Thekumparampil, Sewoong Oh, Michael Muehlebach, Niao He.
  NeurIPS, 2025 [arXiv] [GitHub]
• PoLAR: Polar-Decomposed Low-Rank Adapter Representation.
  Kai Lion, Liang Zhang, Bingcong Li, Niao He.
  NeurIPS, 2025 [arXiv] [GitHub] [ES-FoMo@ICML'25] [CPAL'26]
• On the Crucial Role of Initialization for Matrix Factorization.
  Bingcong Li, Liang Zhang, Aryan Mokhtari, Niao He.
  ICLR, 2025 [OpenReview] [arXiv] [OPT-ML@NeurIPS'24] [CPAL'25]
• Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems.
  Bingcong Li, Liang Zhang, Niao He.
  NeurIPS, 2024 [NeurIPS] [OpenReview] [arXiv] [TF2M@ICML'24]
• DPZero: Private Fine-Tuning of Language Models without Backpropagation.
  Liang Zhang, Bingcong Li, Kiran Koshy Thekumparampil, Sewoong Oh, Niao He.
  ICML, 2024 [ICML] [OpenReview] [arXiv] [GitHub] [Video] [FL-FM@NeurIPS'23] [FORC'24] [TPDP'24]
• Generalization Bounds of Nonconvex-(Strongly)-Concave Stochastic Minimax Optimization.
  Siqi Zhang*, Yifan Hu*, Liang Zhang, Niao He.
  AISTATS, 2024 [AISTATS] [arXiv] [OPT-ML@NeurIPS'22]
• Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization.
  Liang Zhang*, Junchi Yang*, Amin Karbasi, Niao He.
  NeurIPS, 2023 (Spotlight) [NeurIPS] [OpenReview] [arXiv]
• Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization.
  Liang Zhang, Kiran Koshy Thekumparampil, Sewoong Oh, Niao He.
  NeurIPS, 2022 [NeurIPS] [OpenReview] [arXiv]

Thesis

• Selecting Minima in Optimization: Implicit Regularization and Active Steering.
  Doctoral Thesis [ETH]. Examined by Niao He, Michael Muehlebach, Sewoong Oh, and Fanny Yang.
• Variance Reduction for Non-Convex Stochastic Optimization: General Analysis and New Applications.
  Master Thesis [ETH]. Supervised by Yifan Hu and Niao He.

Presentations

• INFORMS Optimization Society Conference 2026, Atlanta. Talk on how zeroth-order optimization finds flat minima.
• AI+X Summit 2025, Zurich. Talk on efficient LLMs fine-tuning (ELF).
• Conference on Parsimony and Learning (CPAL) 2025, Stanford. Poster presentation on LoRA initialization (NoRA).
• Theory and Practice of Differential Privacy (TPDP) 2024, Boston. Poster presentation on DPZero.
• SIAM Conference on Optimization (OP23), Seattle. Talk on differentially private stochastic minimax optimization.

Teaching Assistant

• Optimization for Data Science (Spring 2024, Autumn 2025), ETH Zurich.
• Foundations of Reinforcement Learning (Autumn 2021, Spring 2023), ETH Zurich.
• Large-Scale Convex Optimization (Spring 2025), ETH Zurich.
• Big Data (Autumn 2024), ETH Zurich.
• AI for Mathematics and Optimization (Spring 2026), ETH Zurich.
• Advanced Topics in Machine Learning (Autumn 2022, Autumn 2023, Autumn 2024), ETH Zurich.

Project Supervision

• Colomban Duclaux, semester project at ETH Zurich, 2025 (co-supervised with Bingcong Li).
  Topic: Aligning LLMs with Differentially Private Reinforcement Learning from Human Feedback.
• Yudong Wei, semester project at ETH Zurich, 2025 (co-supervised with Bingcong Li).
  Topic: Overparameterized Matrix Sensing.
• Lejs Behric, bachelor thesis at ETH Zurich, 2024 (co-supervised with Bingcong Li and Kiran Koshy Thekumparampil).
  Topic: Fine-tuning LLMs with Efficient Zeroth-Order Optimization.
• Nikolija Bojkovic, Student Summer Research Fellowship at ETH Zurich, 2024 (co-supervised with Bingcong Li).
  Topic: Differentially Private Stochastic Zeroth-Order Optimization.
• Lukas Altun, bachelor thesis at ETH Zurich, 2024 (co-supervised with Bingcong Li).
  Topic: Computationally Efficient Sharpness-Aware Minimization.
• Martin Mason, bachelor thesis at ETH Zurich, 2023.
  Topic: Exploring Gradient Clipping in Private Machine Learning.

Service

• Journal reviewer: Journal of Machine Learning Research (JMLR), Transactions on Machine Learning Research (TMLR), IEEE Transactions on Signal Processing (TSP).
• Conference reviewer: NeurIPS (2023, 2024, 2025), ICLR (2023, 2024, 2025, 2026), ICML (2024, 2025, 2026), AISTATS (2025, 2026).
• Top reviewers for NeurIPS (2024, 2025).