DS
Daniel Soudry
cs.LGstat.MLcs.AIcs.CVq-bio.NCcs.NEq-bio.QMcs.NAmath.NAModels, Neurological
On Valency
published · living versionsW_pe7z2qpq·v1 · currentpublished
Positional Encoding is All You Need for Scalable Equivariance Constraint Relaxation
with Hagay Michaeli, Haggai Maron
1 version
Preprints & journals
96 papers in the corpus · 2011–2026Global Convergence in Deep Networks via the Second Law of Thermodynamics2610.24775v1 · Matan Tsipory, Ofir Gaash, Itamar Harel et al.2026 · 0 citationsarXivon Valency
Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression2610.20804v1 · Gilad Karpel, Daniel Soudry2026 · 0 citationsarXiv
UNREAL: Unifying Retrieval and Long-Context with a Single Model2610.08463v1 · Edan Kinderman, Elad Hoffer, Yochai Blau et al.2026 · 0 citationsarXiv
More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding2610.04753v2 · Noam Elata, Itay Lamprecht, Mikey Shechter et al.2026 · 0 citationsarXiv
Convergence of Continual Learning in Homogeneous Deep Networks2606.30559v1 · Matan Schliserman, Gon Buzaglo, Itay Evron et al.2026 · 0 citationsarXiv
Workspace Optimization: How to Train Your Agent2605.09650v1 · Elad Sarafian, Gal Kaplun, Ron Banner et al.2026 · 0 citationsarXiv
Retrieval from Within: An Intrinsic Capability of Attention-Based Models2605.05806v3 · Elad Hoffer, Yochai Blau, Edan Kinderman et al.2026 · 0 citationsarXiv
Normalized Architectures are Natively 4-Bit2605.06067v1 · Maxim Fishman, Brian Chmiel, Ron Banner et al.2026 · 0 citationsarXiv
Optimal L2 Regularization in High-dimensional Continual Linear Regression2601.13844v3 · Gilad Karpel, Edward Moroshko, Ran Levinstein et al.2026 · 0 citationsarXiv
Alias-Free ViT: Fractional Shift Invariance via Linear Attention2510.22673v1 · Hagay Michaeli, Daniel Soudry2025 · 1 citationarXiv
Are Greedy Task Orderings Better Than Random in Continual Linear Regression?2510.19941v1 · Matan Tsipory, Ran Levinstein, Itay Evron et al.2025 · 0 citationsarXiv
Tensor-Parallelism with Partially Synchronized Activations2506.19645v2 · Itay Lamprecht, Asaf Karnieli, Yair Hanani et al.2025 · 1 citationarXiv
When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets2506.19031v1 · Chen Zeno, Hila Manor, Greg Ongie et al.2025 · 0 citationsarXiv
Optimal Rates in Continual Linear Regression via Increasing Regularization2506.06501v2 · Ran Levinstein, Amit Attia, Matan Schliserman et al.2025 · 0 citationsarXiv
Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes2505.19087v2 · Itamar Harel, Yonathan Wolanowsky, Gal Vardi et al.2025 · 0 citationsarXiv
FP4 All the Way: Fully Quantized Training of LLMs2505.19115v2 · Brian Chmiel, Maxim Fishman, Ron Banner et al.2025 · 1 citationarXiv
PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training2505.18313v1 · Matan Haroush, Daniel Soudry2025 · 0 citationsarXiv
From Continual Learning to SGD and Back: Better Rates for Continual Linear Models2504.04579v3 · Itay Evron, Ran Levinstein, Matan Schliserman et al.2025 · 0 citationsarXiv
The Implicit Bias of Gradient Descent on Separable Multiclass Data2411.01350v2 · Hrithik Ravi, Clayton Scott, Daniel Soudry et al.2024 · 1 citationarXiv
Provable Tempered Overfitting of Minimal Nets and Typical Nets2410.19092v1 · Itamar Harel, William M. Hoza, Gal Vardi et al.2024 · 1 citationarXiv
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks2410.01483v2 · Edan Kinderman, Itay Hubara, Haggai Maron et al.2024 · 0 citationsTransactions on Machine Learning Research, 2025
Scaling FP8 training to trillion-token LLMs2409.12517v2 · Maxim Fishman, Brian Chmiel, Ron Banner et al.2024 · 1 citationarXiv
Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes2406.06838v1 · Dan Qiao, Kaiqi Zhang, Esha Singh et al.2024 · 0 citationsarXiv
How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers2402.06323v3 · Gon Buzaglo, Itamar Harel, Mor Shpigel Nacson et al.2024 · 0 citationsarXiv
Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators2401.14110v1 · Yaniv Blumenfeld, Itay Hubara, Daniel Soudry2024 · 1 citationarXiv
The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model2401.12617v2 · Daniel Goldfarb, Itay Evron, Nir Weinberger et al.2024 · 1 citationarXiv
How do Minimum-Norm Shallow Denoisers Look in Function Space?2311.06748v2 · Chen Zeno, Greg Ongie, Yaniv Blumenfeld et al.2023 · 0 citationsarXiv
Exponential Quantum Communication Advantage in Distributed Inference and Learning2310.07136v3 · Dar Gilboa, Hagay Michaeli, Daniel Soudry et al.2023 · 4 citationsarXiv
The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks2306.17499v1 · Mor Shpigel Nacson, Rotem Mulayoff, Greg Ongie et al.2023 · 5 citationsarXiv
DropCompute: simple and more robust distributed synchronous training via compute variance reduction2306.10598v2 · Niv Giladi, Shahar Gottlieb, Moran Shkolnik et al.2023 · 0 citations37th Conference on Neural Information Processing Systems (NeurIPS 2023)
Continual Learning in Linear Classification on Separable Data2306.03534v1 · Itay Evron, Edward Moroshko, Gon Buzaglo et al.2023 · 3 citationsarXiv
Explore to Generalize in Zero-Shot RL2306.03072v3 · Ev Zisselman, Itai Lavie, Daniel Soudry et al.2023 · 3 citationsarXiv
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond2305.13064v1 · Itai Kreisler, Mor Shpigel Nacson, Daniel Soudry et al.2023 · 1 citationarXiv
Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations2303.08085v2 · Hagay Michaeli, Tomer Michaeli, Daniel Soudry2023 · 10 citationsarXiv
The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study2302.05334v1 · Itay Evron, Ophir Onn, Tamar Weiss Orzech et al.2023 · 3 citationsarXiv
How catastrophic can catastrophic forgetting be in linear regression?2205.09588v2 · Itay Evron, Edward Moroshko, Rachel Ward et al.2022 · 11 citations35th Annual Conference on Learning Theory (2022)
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients2203.10991v3 · Brian Chmiel, Itay Hubara, Ron Banner et al.2022 · 2 citationsarXiv
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats2112.10769v4 · Brian Chmiel, Ron Banner, Elad Hoffer et al.2021 · 6 citationsarXiv
Task-Agnostic Continual Learning Using Online Variational Bayes With Fixed-Point Updates.34474486 · Zeno, Chen, Golan, Itay, Hoffer, Elad et al.2021 · 16 citationsNeural computation. 2021;33(11):3139-3177
Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability2109.11792v1 · Aviv Tamar, Daniel Soudry, Ev Zisselman2021 · 5 citationsarXiv
Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling2106.07218v2 · Niv Giladi, Zvika Ben-Haim, Sella Nevo et al.2021 · 4 citations35th Conference on Neural Information Processing Systems (NeurIPS 2021)
A statistical framework for efficient out of distribution detection in deep neural networks2102.12967v3 · Matan Haroush, Tzviel Frostig, Ruth Heller et al.2021 · 10 citationsarXiv
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent2102.09769v1 · Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson et al.2021 · 8 citationsarXiv
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks2102.08124v2 · Itay Hubara, Brian Chmiel, Moshe Island et al.2021 · 36 citationsarXiv
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy2007.06738v1 · Edward Moroshko, Suriya Gunasekar, Blake Woodworth et al.2020 · 12 citationsarXiv
Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?2007.01038v1 · Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry2020 · 6 citationsarXiv
Neural gradients are near-lognormal: improved quantized and sparse training2006.08173v3 · Brian Chmiel, Liad Ben-Uri, Moran Shkolnik et al.2020 · 17 citationsarXiv
Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming2006.10518v2 · Itay Hubara, Yury Nahshan, Yair Hanani et al.2020 · 77 citationsarXiv
Training of Quantized Deep Neural Networks using a Magnetic Tunnel Junction-Based Synapse1912.12636v2 · Tzofnat Greenberg Toledo, Ben Perach, Itay Hubara et al.2019 · 2 citationsSemicond. Sci. Technol. 36 114003 (2021)
Is Feature Diversity Necessary in Neural Network Initialization?1912.05137v3 · Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry2019 · 0 citationsarXiv
Career total: 123 works. 96 are in this corpus.Showing the 50 most recent.
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in alpha. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.