Publications

Recent Preprints

  1. Learning the identity: a case study of how SGD selects among functional decompositions

    Andy Arditi, Weian Xie, David Bau, Liu Ziyin

    Preprint 2026 PhysicsAI arXiv

  2. Connectome-based biophysical modeling of a figure-ground discrimination circuit

    Liu Ziyin, Yifei Lu, Alexander Borst, Sven Dorkenwald, Tomaso Poggio

    Preprint 2026 Neuroscience bioRxiv

  3. Width-Independent Compressibility of Deep Neural Networks

    Hong-Yi Wang, Mingze Wang, Liu Ziyin

    Preprint 2026 PhysicsAI arXiv

  4. Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

    Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang

    Preprint 2026 PhysicsAI arXiv

  5. Thermodynamic Irreversibility of Training Algorithms

    Liu Ziyin, Yuanjie Ren, Adam Levine, Isaac Chuang

    Preprint 2026 PhysicsAI arXiv

  6. Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model

    Yizhou Xu, Pierfrancesco Beneventano, Isaac Chuang, Liu Ziyin

    Preprint 2026 PhysicsAI arXiv

  7. An Equivariance Toolbox for Learning Dynamics

    Yongyi Yang, Liu Ziyin

    Preprint 2025 PhysicsAI arXiv

  8. Does Feedback Alignment Work at Biological Timescales?

    Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio

    Preprint 2025 NeuroscienceAI arXiv

  9. Heterosynaptic Circuits Are Universal Gradient Machines

    Liu Ziyin, Isaac Chuang, Tomaso Poggio

    Preprint 2025 NeuroscienceAI arXiv
2026

  1. How Biological Synapses Self-Assemble Gradient Learning

    Qianli Liao*, Liu Ziyin*, Yulu Gan*, Brian Cheung, Mark T. Harnett, Tomaso Poggio

    Proceedings of the National Academy of Sciences 123 (41), e2606245123 (2026) NeuroscienceAI paperarXiv

  2. Topological Invariance and Breakdown in Learning Dynamics

    Yongyi Yang, Tomaso Poggio, Isaac Chuang, Liu Ziyin*

    NeurIPS 2026 PhysicsAI arXiv

  3. On the Invariance and Generality of Neural Scaling Laws

    Xing Han, Liu Ziyin, Suchi Saria, Paul Pu Liang

    NeurIPS 2026 AI arXiv

  4. When Reasoning Meets Its Laws

    Junyu Zhang, Yifan Sun, Tianang Leng, Jingyan Shen, Liu Ziyin, Paul Pu Liang, Huan Zhang

    NeurIPS 2026 AI arXiv

  5. Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes

    John Blue, Harshil Avlani, Zhiyang He, Liu Ziyin, Isaac L. Chuang

    Quantum 10, 2149 (2026) PhysicsAI arXiv

  6. Ubiquity of Emergent Hebbian Dynamics in Regularized Learning

    David Koplow, Tomaso Poggio, Liu Ziyin

    ICML 2026 NeuroscienceAI arXiv

  7. A universal compression theory for lottery ticket hypothesis and neural scaling laws

    Hong-Yi Wang, Di Luo, Tomaso Poggio, Isaac L. Chuang, Liu Ziyin*

    ICLR 2026 PhysicsAI arXiv
2025

  1. Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning

    Liu Ziyin*, Yizhou Xu*, Isaac Chuang

    NeurIPS 2025 PhysicsAI arXiv

  2. Noise Balance and Stationary Distribution of Stochastic Gradient Descent

    Liu Ziyin*, Hongchao Li*, Masahito Ueda

    Physical Review E 111, 065303 (2025) PhysicsAI paperarXiv

  3. Compositional Generalization via Forced Rendering of Disentangled Latents

    Qiyao Liang, Daoyuan Qian, Liu Ziyin, Ila Fiete

    ICML 2025 NeuroscienceAI arXiv

  4. Understanding the Emergence of Multimodal Representation Alignment

    Megan Tjandrasuwita, Chanakya Ekbote, Liu Ziyin, Paul Pu Liang

    ICML 2025 AI arXiv

  5. Formation of Representations in Neural Networks

    Liu Ziyin, Isaac Chuang, Tomer Galanti, Tomaso Poggio

    ICLR 2025 ★ spotlight: 5% of all submissionsPhysicsAI paper

  6. Remove Symmetries to Control Model Expressivity and Improve Optimization

    Liu Ziyin*, Yizhou Xu*, Isaac Chuang

    ICLR 2025 PhysicsAI paper

  7. Three Mechanisms of Feature Learning in a Linear Network

    Yizhou Xu*, Liu Ziyin*

    ICLR 2025 PhysicsAI paper
2024

  1. Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent

    Liu Ziyin, Mingze Wang, Hongchao Li, Lei Wu

    NeurIPS 2024 PhysicsAI arXiv

  2. Symmetry Induces Structure and Constraint of Learning

    Liu Ziyin

    ICML 2024 PhysicsAI arXiv
2023

  1. Zeroth, first, and second-order phase transitions in deep neural networks

    Liu Ziyin, Masahito Ueda

    Physical Review Research 2023 PhysicsAI arXiv

  2. Exact Solutions of a Deep Linear Network

    Liu Ziyin, Botao Li, Xiangming Meng

    Journal of Statistical Mechanics: Theory and Experiment, 2023 PhysicsAI arXiv

  3. On the stepwise nature of self-supervised learning

    James B. Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J. Fetterman, Joshua Albrecht

    ICML 2023 AI arXiv

  4. Sparsity by Redundancy: Solving L1 with SGD

    Liu Ziyin*, Zihao Wang*

    ICML 2023 AI arXiv

  5. What shapes the loss landscape of self-supervised learning?

    Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda, Hidenori Tanaka

    ICLR 2023 PhysicsAI arXiv
2022

  1. Exact Solutions of a Deep Linear Network

    Liu Ziyin, Botao Li, Xiangming Meng

    NeurIPS 2022 PhysicsAI arXiv

  2. Posterior Collapse of a Linear Latent Variable Model

    Zihao Wang*, Liu Ziyin*

    NeurIPS 2022 ★ oral: 1% of all submissionsAI arXiv

  3. Universal Thermodynamic Uncertainty Relation in Non-Equilibrium Dynamics

    Liu Ziyin, Masahito Ueda

    Physical Review Research (2022) Physics arXiv

  4. Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction

    Liu Ziyin, Kentaro Minami, Kentaro Imajo

    ICAIF 2022 (3rd ACM International Conference on AI in Finance) AI arXiv

  5. Power Laws and Symmetries in a Minimal Model of Financial Market Economy

    Liu Ziyin, Katsuya Ito, Kentaro Imajo, Kentaro Minami

    Physical Review Research (2022) Physics arXiv

  6. Logarithmic landscape and power-law escape rate of SGD

    Takashi Mori, Liu Ziyin, Kangqiao Liu, Masahito Ueda

    ICML 2022 PhysicsAI arXiv

  7. SGD with a Constant Large Learning Rate Can Converge to Local Maxima

    Liu Ziyin, Botao Li, James B. Simon, Masahito Ueda

    ICLR 2022 ★ spotlight: 5% of all submissionsPhysicsAI arXiv

  8. Strength of Minibatch Noise in SGD

    Liu Ziyin*, Kangqiao Liu*, Takashi Mori, Masahito Ueda

    ICLR 2022 ★ spotlight: 5% of all submissionsPhysicsAI arXiv
2021

  1. On the Distributional Properties of Adaptive Gradients

    Zhang Zhiyi*, Liu Ziyin*

    UAI 2021 AI arXiv

  2. Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent

    Kangqiao Liu*, Liu Ziyin*, Masahito Ueda

    ICML 2021 PhysicsAI arXiv

  3. Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

    Paul Pu Liang*, Peter Wu*, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov

    ACM Multimedia 2021 also NeurIPS 2020 Workshop on Meta LearningAI arXivcode
2020

  1. Neural Networks Fail to Learn Periodic Functions and How to Fix It

    Liu Ziyin, Tilman Hartwig, Masahito Ueda

    NeurIPS 2020 AI arXiv
2019

  1. Deep Gamblers: Learning to Abstain with Portfolio Theory

    Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

    NeurIPS 2019 AI paperarXivcode

  2. Think Locally, Act Globally: Federated Learning with Local and Global Representations

    Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Ruslan Salakhutdinov, Louis-Philippe Morency

    NeurIPS 2019 Workshop on Federated Learning ★ oral, distinguished student paper awardAI workshoparXivcode
2018

  1. Multimodal Language Analysis with Recurrent Multistage Fusion

    Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency

    EMNLP 2018 ★ oral presentationAI paperarXiv

Tutorial / Notes