Informationen zum Seminar: Mathematical Foundations of Statistical Learning (SS 2020)

Dozent: Prof. Dr. Angelika Rohde

Assistent: Dario Kieffer

Zeit, Ort: Di 10-12, SR 127, Ernst-Zermelo-Straße 1, online

Vorbesprechung: Do 06.02.2020, 14:00 Uhr, Raum 232, Ernst-Zermelo-Str. 1

Aktuelles

• Donnerstag, 06. Februar 2020, 14:00 Uhr: Vorbesprechung und Themenvergabe.

Vorträge

• Dienstag, 12. Mai 2020, 10:00 Uhr: Einführung in Statistical Learning, basierend auf Vapnik, V. (1999).

Folien: pdf.     Handout: pdf.

• Dienstag, 26. Mai 2020, 10:00 Uhr: Support Vector Machines, basierend auf Blanchard, G. & Bousquet, O. & Massart, P. (2008).

Folien: pdf.     Programmcode: zip.

• Dienstag, 02. Juni 2020, 10:00 Uhr: Vergleich von Koltchinskii, V. & Panchenko, D. (2002) und Steinwart, I. & Scovel, C. (2007), Teil 1.

Folien: pdf.     Ausarbeitung: pdf.

• Dienstag, 09. Juni 2020, 10:00 Uhr: Vergleich von Koltchinskii, V. & Panchenko, D. (2002) und Steinwart, I. & Scovel, C. (2007), Teil 2.

Folien: pdf.     Ausarbeitung: pdf.

• Dienstag, 23. Juni 2020, 10:00 Uhr: Nichtparametrische Regression und neuronale Netze basierend auf Schmidt-Hieber, J. (2020).

Folien: pdf.     Beweis Theorem 2: pdf.

• Dienstag, 21. Juli 2020, 10:00 Uhr: Bachelorvortrag über Risk Bounds for Statistical Learning.

Folien: pdf.

• Dienstag, 28. Juli 2020, 10:00 Uhr: Bachelorvortrag über M-Schätzung und MLE-Theorie in logistischer Regression.

Folien: pdf.

Inhalt

Statistical Learning Theory has demonstrated its usefulness by providing the ground for developing successful and well-founded learning algorithms. The usual framework is as follows. We consider a space of possible inputs (instance space) and a space of possible outputs (label sets). The product space × is assumed to be measurable and is endowed with an unknown probability measure. Based on n independent input-output pairs (X₁,Y₁), …, (X_n,Y_n) sampled according to this probability measure, the goal of a learning algorithm is to pick a function g in a space of functions from to in such a way that this function should capture as much as possible the relationship (which may not be of a functional nature) between the random variables X and Y.

In this seminar, we study this problem in a mathematically rigorous way, particularly focusing on recent learning algorithms. The theory of empirical processes will be shown to play a fundamental role in their analysis.

Vorkenntnisse

• Notwendige Vorkenntnisse: Analysis und Grundlagen der Stochastik
• Nützliche Vorkenntnisse: Wahrscheinlichkeitstheorie

Literatur

Einführung in Statistical Learning

• Vapnik, V. (1999). An Overview of Statistical Learning Theory. IEEE Transactions on Neural Networks 10, 988-999. pdf
• Bousquet, O. & Boucheron, S. & Lugosi, G. (2004). Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, 169-207. pdf
• Hastie, T. & Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. pdf

Weiterführende Literatur

• Koltchinskii, V. & Panchenko, D. (2002). Empirical Margin Distributions and Bounding the Generalization Error of Combined Classifiers. The Annals of Statistics 30, 1-50. pdf
• Tsybakov, A. (2004). Optimal Aggregation of Classifiers in Statistical Learning. The Annals of Statistics 32, 135-166. pdf
• Massart, P. & Nédélec, É. (2006). Risk Bounds for Statistical Learning. The Annals of Statistics 34, 2326-2366. pdf
• Steinwart, I. & Scovel, C. (2007). Fast Rates for Support Vector Machines using Gaussian Kernels. The Annals of Statistics 35, 575-607. pdf 
• Blanchard, G. & Bousquet, O. & Massart, P. (2008). Statistical Performance of Support Vector Machines. The Annals of Statistics 36, 489-531. pdf
• Wang, Z. & Liu, H. & Zhang, T. (2014). Optimal Computational and Statistical Rates of Convergence for Sparse Nonconvex Learning Problems. The Annals of Statistics 42, 2164-2201. pdf
• van Erven, T. & Grünwald, P. & Mehta, N. & Reid, M. & Williamson, R. (2015). Fast Rates in Statistical and Online Learning. Journal of Machine Learning Research 16, 1793-1861. pdf
• Bauer, B. & Kohler, M. (2019). On Deep Learning as a Remedy for the Curse of Dimensionality in Nonparametric Regression. The Annals of Statistics 47, 2261-2285. pdf
• Cai, T. & Wei, H. (2020). Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier. The Annals of Statistics, to appear. pdf
• Schmidt-Hieber, J. (2020). Nonparametric Regression using Deep Neural Networks with ReLu Activation Function. The Annals of Statistics (discussion paper), to appear. pdf

Ergänzende Literatur

• Dümbgen, L. (2017). Empirische Prozesse. Vorlesungsskript, Universität Bern. pdf