Machine Learning Course SS 14 U Stuttgart

DATE OF THE WRITTEN EXAM: August 26, 11:00-14:00, room V 55.02 -- Please always check the Seiten des Prüfungsamts

See my general teaching page for previous versions of this lecture.

Exploiting large-scale data is a central challenge of our time. Machine Learning is the core discipline to address this challenge, aiming to extract useful models and structure from data. Studying Machine Learning is motivated in multiple ways: 1) as the basis of commercial data mining (Google, Amazon, Picasa, etc), 2) a core methodological tool for data analysis in all sciences (vision, linguistics, software engineering, but also biology, physics, neuroscience, etc) and finally, 3) as a core foundation of autonomous intelligent systems.

This lecture introduces to modern methods in Machine Learning, including discriminative as well as probabilistic generative models. A preliminary outline of topics is:

• motivation
• probabilistic modeling and inference
• regression and classification methods (kernel methods, Gaussian Processes, Bayesian kernel logistic regression, relations)
• discriminative learning (logistic regression, Conditional Random Fields)
• feature selection
• boosting and ensemble learning
• representation learning and embedding (kernel PCA and derivatives, deep learning)
• graphical models
• inference in graphical models (MCMC, message passing, variational)
• learning in graphical models

Students should bring basic knowledge of linear algebra, probability theory and optimization.

Organization

• This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
• See the 01-introduction slides for further information.

Schedule, slides & exercises

date	topics	slides	exercises (due on 'date'+3)
14.4.	Introduction	01-introduction	e01-intro
21.4.	[Easter holiday]	cancelled	cancelled
28.4.	Regression	02-regression	[holiday]
5.5.	Classification, Discriminative Learning	03-classification	e02-linearRegression ../data/dataLinReg1D.txt ../data/dataLinReg2D.txt ../data/dataQuadReg1D.txt ../data/dataQuadReg2D.txt ../data/dataQuadReg2D_noisy.txt
12.5.	Classification, Discriminative Learning		e03-classification ../data/data2Class.txt ../data/digit_pca.txt ../data/digit_label.txt
19.5.	Breadth of ML methods	04-MLbreadth	e04-kernelsAndCRFs
26.5.	Breadth of ML methods (cont.)		holiday
2.6.	Breadth of ML methods (cont.)		e05-PCA-NN
9.6.	[Pfingstferien]
16.6.	Probabilities & Bayesian ML	05-probabilities 06-BayesianRegressionClassification	14-days exercise e06-SVM-scikit
23.6.	Bayesian ML (cont.) & Graphical Models	07-graphicalModels	14-days exercise e06-SVM-scikit
30.6.	Graphical Models (cont.)		e07-Bayes-GPs
7.7.	Learning with Graphical Models	08-graphicalModels-Learning	e08-graphicalModels
14.7.	Summary	14-MachineLearning-script paper mentioned in the lecture: A few useful things to know about machine learning	e09-inference

Literature

[1] The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman. Springer, Second Edition, 2009.
full online version available
(recommended: read introductory chapter)

[2] Pattern Recognition and Machine Learning by Bishop, C. M.. Springer 2006.
online
(especially chapter 8, which is fully online)

[email by Stefan Otte:] This is a nice little (26 pages) linear algebra and matrix calculus reference. It's used for the ML class in Stanford. Maybe it's interesting for your ML class. link

[email by Stefan Otte:] Feature selection, l1 vs. l2 regularization, and rotational invariance Paper: link Comments: link