Statistical/information-theoretic aspects of learning.

• Statistical supervised learning theory
• Statistical unsupervised learning theory

The other major branch of learning theory is Computational learning theory, which also studies Computational complexity of learning.

See Generalization

http://maxim.ece.illinois.edu/teaching/SLT/SLT.pdf – http://www.stats.ox.ac.uk/~rebeschi/teaching/AFoL/18/