DA
Dan Alistarh
cs.LGcs.DCcs.CLcs.DSstat.MLcs.AImath.OCcs.CVcs.NEcs.DB
On Valency
published · living versionsW_u7fa5rax·v1 · currentpublished
MLSys: The New Frontier of Machine Learning Systems
with Alexander Ratner, Gustavo Alonso, David G. Andersen, Peter Bailis +64
1 version
Preprints & journals
151 papers in the corpus · 2013–2026MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models2602.03537v2 · Maximilian Kleinegger, Elvir Crncevi'c, Dan Alistarh2026 · 0 citationsarXiv
Model Compression with Exact Budget Constraints via Riemannian Manifolds2605.00649v3 · Michael Helcig, Dan Alistarh2026 · 0 citationsarXiv
One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training2609.11936v1 · Erik Schultheis, Maximilian Kleinegger, Dan Alistarh2026 · 0 citationsarXiv
Statistically-Lossless Quantization of Large Language Models2605.02404v2 · Michael Helcig, Eldar Kurtic, Dan Alistarh2026 · 0 citationsarXiv
The Sparsity Whisperer2608.06630v1 · Linghao Kong, Inimai Subramanian, Micah Adler et al.2026 · 0 citationsarXiv
The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm2507.18553v4 · Jiale Chen, Yalda Shabanzadeh, Elvir Crncevi'c et al.2025 · 0 citationsInternational Conference on Learning Representations, 2026, pp. 122653-122695
GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding2607.27042v1 · Jiale Chen, Torsten Hoefler, Dan Alistarh2026 · 0 citationsarXiv
DarwinLM: Evolutionary Structured Pruning of Large Language Models2502.07780v4 · Shengkun Tang, Oliver Sieberling, Eldar Kurtic et al.2025 · 2 citationsarXiv
DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers2602.02016v2 · Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis et al.2026 · 0 citationsarXiv
CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training2510.18784v3 · Soroush Tabesh, Mher Safaryan, Andrei Panferov et al.2025 · 0 citationsProceedings of Machine Learning and Systems 8 (MLSys 2026)
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication2602.04396v2 · Andrej Jovanovi'c, Alex Iacob, Mher Safaryan et al.2026 · 0 citationsarXiv
Apertus LLM Family Expansion via Distillation and Quantization2605.29128v2 · Andrei Panferov, Davit Melikidze, Martin Jaggi et al.2026 · 0 citationsarXiv
The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws2501.12486v2 · Tian Jin, Ahmed Imtiaz Humayun, Utku Evci et al.2025 · 0 citationsThe Thirteenth International Conference on Learning Representations (ICLR), 2025
Expand Neurons, Not Parameters2510.04500v3 · Linghao Kong, Inimai Subramanian, Yonadav Shavit et al.2025 · 0 citationsarXiv
WUSH: Near-Optimal Adaptive Transforms for LLM Quantization2512.00956v3 · Jiale Chen, Vage Egiazarian, Roberto L. Castro et al.2025 · 0 citationsarXiv
Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation2601.22813v2 · Andrei Panferov, Erik Schultheis, Soroush Tabesh et al.2026 · 0 citationsarXiv
Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning2410.06074v3 · Jiale Chen, Dingling Yao, Adeel Pervez et al.2024 · 0 citationsInternational Conference on Learning Representations, 2025, pp. 10018-10039
"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization2411.02355v4 · Eldar Kurtic, Alexandre Marques, Shubhra Pandit et al.2024 · 8 citationsarXiv
GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling2604.18556v2 · Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan et al.2026 · 0 citationsarXiv
Grid Games: The Power of Multiple Grids for Quantizing Large Language Models2605.12327v1 · Vage Egiazarian, Erik Schultheis, Andrei Panferov et al.2026 · 0 citationsarXiv
MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning2605.07850v1 · Ionut-Vlad Modoranu, Mher Safaryan, Dan Alistarh2026 · 0 citationsarXiv
Towards Robust Scaling Laws for Optimizers2602.07712v2 · Alexandra Volkova, Mher Safaryan, Christoph H. Lampert et al.2026 · 0 citationsarXiv
The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM2510.01650v2 · Kwanhee Lee, Hyeondo Jang, Dongyeop Lee et al.2025 · 0 citationsarXiv
Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data2601.23153v1 · Eugenia Iofinova, Dan Alistarh2026 · 0 citationsarXiv
ECO: Quantized Training without Full-Precision Master Weights2601.22101v1 · Mahdi Nikdan, Amir Zandieh, Dan Alistarh et al.2026 · 0 citationsarXiv
Quartet: Native FP4 Training Can Be Optimal for Large Language Models2505.14669v4 · Roberto L. Castro, Andrei Panferov, Soroush Tabesh et al.2025 · 0 citationsarXiv
Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks2512.11718v1 · Sergey Pankratov, Dan Alistarh2025 · 0 citationsarXiv
Hogwild! Inference: Parallel LLM Generation via Concurrent Attention2504.06261v4 · Gleb Rodionov, Roman Garipov, Alina Shutova et al.2025 · 0 citationsarXiv
HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs2501.02625v3 · Saleh Ashkboos, Mahdi Nikdan, Soroush Tabesh et al.2025 · 0 citationsarXiv
Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence2210.07703v4 · Matin Ansaripour, Shayan Talaei, Giorgi Nadiradze et al.2022 · 0 citationsarXiv
FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models2505.17967v4 · Ionut-Vlad Modoranu, Mher Safaryan, Erik Schultheis et al.2025 · 0 citationsarXiv
Beyond Outliers: A Study of Optimizers Under Quantization2509.23500v2 · Georgios Vlassis, Saleh Ashkboos, Alexandra Volkova et al.2025 · 0 citationsarXiv
Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations1908.04207v5 · Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo et al.2019 · 56 citationsarXiv
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging2005.00124v5 · Shigang Li, Tal Ben-Nun, Giorgi Nadiradze et al.2020 · 15 citationsIEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 7, pp. 1725-1739, 1 July 2021
Optimizers Qualitatively Alter Solutions And We Should Leverage This2507.12224v1 · Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu et al.2025 · 0 citationsarXiv
EvoPress: Accurate Dynamic Model Compression via Evolutionary Search2410.14649v2 · Oliver Sieberling, Denis Kuznedelev, Eldar Kurtic et al.2024 · 0 citationsarXiv
QuEST: Stable Training of LLMs with 1-Bit Weights and Activations2502.05003v2 · Andrei Panferov, Jiale Chen, Soroush Tabesh et al.2025 · 1 citationarXiv
Position: It's Time to Act on the Risk of Efficient Personalized Text Generation2502.06560v2 · Eugenia Iofinova, Andrej Jovanovic, Dan Alistarh2025 · 0 citationsarXiv
Unified Scaling Laws for Compressed Representations2506.01863v1 · Andrei Panferov, Alexandra Volkova, Ionut-Vlad Modoranu et al.2025 · 0 citationsarXiv
Efficient Data Selection at Scale via Influence Distillation2505.19051v1 · Mahdi Nikdan, Vincent Cohen-Addad, Dan Alistarh et al.2025 · 0 citationsarXiv
Communication-Efficient Federated Learning With Data and Client Heterogeneity2206.10032v4 · Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze et al.2022 · 0 citationsarXiv
Layer-wise Quantization for Quantized Optimistic Dual Averaging2505.14371v1 · Anh Duc Nguyen, Ilia Markov, Frank Zhengqing Wu et al.2025 · 0 citationsarXiv
Towards Combinatorial Interpretability of Neural Computation2504.08842v2 · Micah Adler, Dan Alistarh, Nir Shavit2025 · 0 citationsarXiv
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics2410.16103v4 · Thomas Robert, Mher Safaryan, Ionut-Vlad Modoranu et al.2024 · 0 citationsarXiv
Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models2501.19392v4 · Alina Shutova, Vladimir Malinovskii, Vage Egiazarian et al.2025 · 0 citationsarXiv
Wasserstein Distances, Neuronal Entanglement, and Sparsity2405.15756v4 · Shashata Sawmya, Linghao Kong, Ilia Markov et al.2024 · 0 citationsarXiv
Compression Scaling Laws:Unifying Sparsity and Quantization2502.16440v1 · Elias Frantar, Utku Evci, Wonpyo Park et al.2025 · 0 citationsarXiv
Panza: Design and Analysis of a Fully-Local Personalized Text Writing Assistant2407.10994v4 · Armand Nicolicioiu, Eugenia Iofinova, Andrej Jovanovic et al.2024 · 0 citationsarXiv
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem2411.17525v1 · Vladimir Malinovskii, Andrei Panferov, Ivan Ilin et al.2024 · 0 citationsarXiv
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence2405.15593v2 · Ionut-Vlad Modoranu, Mher Safaryan, Grigory Malinovsky et al.2024 · 2 citationsarXiv
Career total: 320 works. 151 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.