The theory (mainly Learning theory) of Artificial neural networks. See also Deep learning theory, Mathematical modelling of neural networks, Statistical mechanics of neural networks

Neural networks class - Université de Sherbrooke (Hugo Larochelle)

Note that the notation in this paper is opposite to that standard in Machine learning. In the paper, $y$ is the input, and $x$ is the output.

Expressivity of neural networks

Learning of neural networks

See Learning theory, Artificial neural networks, Deep learning theory, Non-convex optimization...

Learning theory and neural networks gingko tree

See talk at ICLR2017 by Riccardo Zecchina

Domains of Solutions and Replica Symmetry Breaking in Multilayer Neural Networks

Weight Space Structure and Internal Representations: A Direct Approach to Learning and Generalization in Multilayer Neural Networks

Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses

Computational efficiency and optimization: Unreasonable effectiveness of learning neural nets: Accessible states and robust ensembles

Generalization

Theory of Deep Learning III: Generalization Properties of SGD

See Generalization, Generalization in deep learning, Statistical physics and inference

Statistical mechanics of learning

Uniqueness of the weights for minimal feedforward nets with a given input-output map,. more here

For neural networks, function determines form, for 0-hidden layer, neural nets...

Functionally Equivalent Feedforward Neural Networks

See more at Singular learning theory

Neural network dynamics, see Dynamical systems

Dynamical Systems Theory for Transparent Symbolic Computation in Neuronal Networks We show that a correspondence can be found between these networks and Finite-state transducers, and use the derived abstraction to investigate how noise affects computation in this class of systems, unveiling a surprising facilitatory effect on information transmission.

http://ieeexplore.ieee.org.sci-hub.cc/document/58339/