Machine Learning Course SS 13 U StuttgartSee my general teaching page for previous versions of this lecture.
This lecture introduces to modern methods in Machine Learning, including discriminative as well as probabilistic generative models. A preliminary outline of topics is:
- probabilistic modeling and inference
- regression and classification methods (kernel methods, Gaussian Processes, Bayesian kernel logistic regression, relations)
- discriminative learning (logistic regression, Conditional Random Fields)
- feature selection
- boosting and ensemble learning
- representation learning and embedding (kernel PCA and derivatives, deep learning)
- graphical models
- inference in graphical models (MCMC, message passing, variational)
- learning in graphical models
- This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
- See the 01-introduction slides for further information.
- Schedule, slides & exercises
 The Elements of Statistical Learning: Data Mining, Inference, and Prediction
by Trevor Hastie, Robert Tibshirani and Jerome Friedman. Springer, Second Edition, 2009.
full online version available
(recommended: read introductory chapter)
 Pattern Recognition and Machine Learning by Bishop, C. M.. Springer 2006.
(especially chapter 8, which is fully online)
[email by Stefan Otte:] This is a nice little (26 pages) linear algebra and matrix calculus reference. It's used for the ML class in Stanford. Maybe it's interesting for your ML class. link
[email by Stefan Otte:] Feature selection, l1 vs. l2 regularization, and rotational invariance Paper: link Comments: link
[email by Stefan Otte:]
ich habe vor kurzem einen sehr guten Google Tech Talk zum Thema Ensembles gesehen. In dem Talk "The Counter-Intuitive Properties of Ensembles for Machine Learning, or, Democracy Defeats Meritocracy" argument W. Philip Kegelmeyer (vereinfacht gesagt), dass man fuer Supervised Learning Ensembles benutzen soll. Vll. ist das fuer den ein oder anderen Studenten von Interesse. Hier ein paar meiner Notizen: - Boosting: overfitting, sensitive to outliers. - "Ensembles of experts": diversity of experts --> diversity in error --> robustness/no overfitting - "Out of Bag validation" (OOB) to determine ensemble size (vs. learning the weights for the voting (which does not scale)) - unstable classifiers (e.g. decision trees) are a good fit for ensembles - decision trees without pruning work well with ensembles. (pruning is normally expensive!) - "Ensembles of bozos": LOTS of bozos which train on tiny subsets (1%) of the data - traditional < experts < bozos - training bozos is faster than training one traditional sage! http://csmr.ca.sandia.gov/~wpk/ http://csmr.ca.sandia.gov/~wpk/slides/avatar-ensembles.pdf http://csmr.ca.sandia.gov/~wpk/avi/avatar-tools-background-video.avi Beste Grüße, Stefan