本书涵盖了用于理解神经网络原理的必要统计力学知识，包括复本方法、空腔方法、平均场近似、变分法、随机能量模型、Nishimori条件、动力学平均场理论、对称性破缺、随机矩阵理论等，同时详细描述了监督学习、无监督学习、联想记忆网络、感知器网络、随机循环网络等神经网络及其功能的物理模型以及解析理论，通过简洁的模型展示了神经网络原理的数学美和物理深度，介绍了相关历史并展望了未来研究的重要课题，可供对神经网络原理感兴趣的学生、研究人员以及工程师参考使用。

黄海平，中山大学物理学院教授，博士生导师。本科毕业于中山大学理工学院，博士毕业于中国科学院理论物理研究所，随后在香港科技大学物理系、东京工业大学计算智能系以及日本理化学研究所脑科学中心从事统计物理与机器学习、神经计算交叉的基础理论研究，2017年因在无监督学习方面的研究获得RIKEN杰出研究奖。2018年入选中山大学百人计划，目前的研究兴趣包括感知学习的孤立解空间证明、无监督学习的对称性破缺本质、深度网络的维度下降和系综学习等方向。曾主持国家自然科学基金青年基金、优秀青年基金等国家级项目。

1 Introduction
References
2 Spin Glass Models and Cavity Method
2.1 Multi-spin Interaction Models
2.2 Cavity Method
2.3 From Cavity Method to Message Passing Algorithms
References
3 Variational Mean-Field Theory and Belief Propagation
3.1 Variational Method
3.2 Variational Free Energy
3.2.1 Mean-Field Approximation
3.2.2 Bethe Approximation
3.2.3 From the Bethe to Naive Mean-Field Approximation
3.3 Mean-Field Inverse Ising Problem
References
4 Monte Carlo Simulation Methods
4.1 Monte Carlo Method
4.2 Importance Sampling
4.3 Markov Chain Sampling
4.4 Monte Carlo Simulations in Statistical Physics
4.4.1 Metropolis Algorithm
4.4.2 Parallel Tempering Monte Carlo
References
5 High-Temperature Expansion
5.1 Statistical Physics Seting
5.2 High-Temperature Expansion
5.3 Properties of the TAP Equation
References
6 Nishimori Line
6.1 Model Setting
6.2 Exact Result for Internal Energy
6.3 Proof of No RSB Effects on the Nishimori Line
References
7 Random Energy Model
7.1 Model Setting
7.2 Phase Diagram
References
8 Statistical Mechanical Theory of Hopfield Model
8.1 Hopfield Model
8.2 Replica Method
8.2.1 Replica-Symmetric Ansatz
8.2.2 Zero-Temperature Limit
8.3 Phase Diagram
8.4 Hopfield Model with Arbitrary Hebbian Length
8.4.1 Computation of the Disorder-Averaged Free Energy
8.4.2 Derivation of Saddle-Point Equations
8.4.3 Computation Transformation to Solve the SDE
8.4.4 Zero-Te mperature Limit
References
9 Replica Symmetry and Replica Symmetry Breaking
9.1 Generalized Free Energy and Complexity of States
9.2 Applications to Constraint Satisfaction Problems
9.3 More Steps of Replica Symmetry Breaking
References
10 Statistical Mechanics of Restricted Boltzmann Machine
10.1 Boltzmann Machine
10.2 Restricted Boltzmann Machine
10.3 Free Energy Calculation
10.4 Thermodynamic Quantities Related to Learning
10.5 Stability Analysis
10.6 Variational Mean-Field Theory for Training Binary RB Ms
10.6.1 RBMs with Binary Weights
10.6.2 Variational Principle
10.6.3 Experiments
References
11 Simplest Model of Unsupervised Learning with Binary Synapses
11.1 Model Setting
11.2 Derivation of sMP and AMP Equations
11.3 Replica Computation
11.3.1 Explicit form of 11.3.2 Estimation of 11.3.3 Derivation of Free Energy and Saddle-Point Equations
11.4 Phase Transitions
11.5 Measuring the Temperature of Dataset
References
12 Inherent-Symmetry Breaking in Unsupervised Learning
12.1 Model Setting
12.1.1 Cavity Approximation
12.1.2 Replica Computation
12.1.3 Stability Analysis
12.2 Phase Diagram
12.3 Hyper-Parameters Inference
References
13 Mean-Field Theory of Ising Perceptron
13.1 Ising Perceptron model
13.2 Message-Passing-Based Learning
13.3 Replica Analysis
13.3.1 Replica Symmetry
13.3.2 Replica Symmetry Breaking
13.4 Further Theory Development
References
14 Mean-Field Model of Multi-layered Perceptron
14.1 Random Active Path Model
14.1.1 Results from Cavity Method
14.1.2 An Infinite Depth Analysis
14.2 Mean-Field Training Algorithms
14.3 Spike and Slab Model
14.3.1 Ensemble Perspective
14.3.2 Training Equations
References
15 Mean-Field Theory of Dimension Reduction
15.1 Mean-Field Model
15.2 Linear Dimensionality and Correlation Strength
15.2.1 Iteration Equations for Correlation Strength
15.2.2 Mechanism of Dimension Reduction
15.3 Dimension Reduction with Correlated Synapses
15.3.1 Model Setting
15.3.2 Mean-Field Calculation
15.3.3 Numerical Results Compared with Theory
References
16 Chaos Theory of Random Recurrent Neural Networks
16.1 Spiking and Rate Models