Statistical Learning and Applications

Prerequisites (knowledge of topic)
The course will focus on the statistical and mathematical foundations of machine learning theory. The aim is to provide the students with a thorough understanding of the basic principles so as to prepare them to develop innovative methods and algorithms in their own field of applications. The course will be reasonably self-contained and does not require any specific prior knowledge in learning theory. It is nevertheless targeted at students with a sufficient quantitative background and it will rely on a basic knowledge of statistics and mathematics (probability, regression methods, linear algebra, elements of optimization theory, etc.) such as provided by standard undergraduate courses.  Although students will be encouraged to perform some numerical experiments on their own, this will by no means be compulsory and could be made by the software/hardware of their choice.

Hardware
none mandatory

Software
none mandatory

Course content
This is a first course on statistical/machine learning and high-dimensional data analysis, aiming at providing a mathematical toolkit to deal with large datasets.

A tentative list of topics which will be covered is as follows (NB. some might be optional and reserved to students with a more mathematical background):

INTRODUCTION

MATHEMATICAL FRAMEWORK

CLASSIFICATION PROBLEMS AND SUPPORT VECTOR MACHINES (SVM)

KERNEL METHODS

CONSISTENCY AND COMPLEXITY ISSUES

LINEAR REGRESSION PROBLEMS

APPLICATIONS

RISK BOUNDS

Structure
There will be 5 whole-day lessons (tentatively).

Literature
Mandatory: none

Supplementary / voluntary:
Besides the lecture notes/slides which will be distributed to the students, the main supplementary reference book for the course will be

M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning. MIT Press, 2012.

Also recommended is
T. Hastie, R. Tibshirani, and M. Wainwright. Statistical Learning with Sparsity:
The Lasso and Generalizations. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. CRC Press, 2015.

as well as the more encyclopedic references

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer New York, 2013.

K. Murphy. Machine learning. A Probabilistic Perspective. MIT Press 2012.

The following textbooks
H. Kobayashi, B.L. Mark, and W. Turin. Probability, Random Processes, and Statistical Analysis: Applications to Communications, Signal Processing, Queueing Theory and Mathematical Finance. Cambridge University Press, 2011.

B. Efron and T; Hastie. Computer Age Statistical Inference. Algorithms, Evidence, and Data Science. Cambridge UP 2016.

can be useful as refreshers or to learn more about basic probability and statistics.

Mandatory readings before course start: none

Examination
Examination paper written at home.