Machine Learning Course SS 14 U Stuttgart


DATE OF THE WRITTEN EXAM: August 26, 11:00-14:00, room V 55.02 -- Please always check the Seiten des Prüfungsamts


See my general teaching page for previous versions of this lecture.


picture
Exploiting large-scale data is a central challenge of our time. Machine Learning is the core discipline to address this challenge, aiming to extract useful models and structure from data. Studying Machine Learning is motivated in multiple ways: 1) as the basis of commercial data mining (Google, Amazon, Picasa, etc), 2) a core methodological tool for data analysis in all sciences (vision, linguistics, software engineering, but also biology, physics, neuroscience, etc) and finally, 3) as a core foundation of autonomous intelligent systems.


This lecture introduces to modern methods in Machine Learning, including discriminative as well as probabilistic generative models. A preliminary outline of topics is:

  • motivation
  • probabilistic modeling and inference
  • regression and classification methods (kernel methods, Gaussian Processes, Bayesian kernel logistic regression, relations)
  • discriminative learning (logistic regression, Conditional Random Fields)
  • feature selection
  • boosting and ensemble learning
  • representation learning and embedding (kernel PCA and derivatives, deep learning)
  • graphical models
  • inference in graphical models (MCMC, message passing, variational)
  • learning in graphical models
Students should bring basic knowledge of linear algebra, probability theory and optimization.
Organization
  • This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
  • See the 01-introduction slides for further information.
Schedule, slides & exercises
date topics slides exercises
(due on 'date'+3)
14.4. Introduction 01-introduction e01-intro
21.4. [Easter holiday] cancelled cancelled
28.4. Regression 02-regression [holiday]
5.5. Classification, Discriminative Learning 03-classification e02-linearRegression
../data/dataLinReg1D.txt
../data/dataLinReg2D.txt
../data/dataQuadReg1D.txt
../data/dataQuadReg2D.txt
../data/dataQuadReg2D_noisy.txt
12.5. Classification, Discriminative Learning e03-classification
../data/data2Class.txt
../data/digit_pca.txt
../data/digit_label.txt
19.5. Breadth of ML methods 04-MLbreadth e04-kernelsAndCRFs
26.5. Breadth of ML methods (cont.) holiday
2.6. Breadth of ML methods (cont.) e05-PCA-NN
9.6. [Pfingstferien]
16.6. Probabilities & Bayesian ML 05-probabilities
06-BayesianRegressionClassification 		14-days exercise
e06-SVM-scikit
23.6. Bayesian ML (cont.) & Graphical Models 07-graphicalModels 14-days exercise
e06-SVM-scikit
30.6. Graphical Models (cont.) e07-Bayes-GPs
7.7. Learning with Graphical Models 08-graphicalModels-Learning e08-graphicalModels
14.7. Summary 14-MachineLearning-script
paper mentioned in the lecture: A few useful things to know about machine learning		 e09-inference
Literature
[1] The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman. Springer, Second Edition, 2009.
full online version available
(recommended: read introductory chapter)

[2] Pattern Recognition and Machine Learning by Bishop, C. M.. Springer 2006.
online
(especially chapter 8, which is fully online)

[email by Stefan Otte:] This is a nice little (26 pages) linear algebra and matrix calculus reference. It's used for the ML class in Stanford. Maybe it's interesting for your ML class. link

[email by Stefan Otte:] Feature selection, l1 vs. l2 regularization, and rotational invariance Paper: link Comments: link

Recent Posts