Unsupervised Learning

203.4700 (3700)

Semester A

General Course Information

Meeting Times: Monday 10-12

Instructor: Dr. Rita Osadchy

e-mail: rita [at]cs [dot]haifa.ac.il
Office: Jacobs 410
____________________________________________________________________________________________________________

Course Description

This course will survey algorithms for unsupervised learning and high dimensional data analysis. The course will cover probabilistic/generative models of high dimensional data, such as Gaussian mixture models, factor analysis, probabilistic latent semantic analysis, independent component analysis as well as spectral methods for dimensionality reduction, including multidimensional scaling, Isomap, locally linear embedding, graph Laplacian methods and kernel PCA. The course will provide an introduction to probabilistic graphical models.

Recommended Prerequisites

The course assumes some basic knowledge of probability theory and linear algebra;
for example, you should be familiar with

• Joint and marginal probability distributions
• Normal (Gaussian) distribution
• Expectation and variance
• Statistical correlation and statistical independence

Tutorials of the above topics.

_____________________________________________________________________

Announcements

• All announcement, home assignments, and guidelines will be distributed by email.
• Those who do not send their contact address on time will not be added to the contact list!!!
• You must send me an email (rita[at]cs [dot]haifa.ac.il) by November 14 from your active address with the subject “UML course contact”.

____________________________________________________________________

Lecture Notes

Date	Topic	Lecture notes	Reading material
31.10	Introduction	PDF	Probability Tutorial PDF MLE PDF Bayesian Learning & Naive Bayes PDF Parzen Windows PDF Hierarchical clustering PDF
7.11	No class
14.11	Linear Dimensionality Reduction: PCA, MDS	PDF	Min Error Derivation of PCA PDF MDS Tutorial: http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm
21.11	Manifold Mapping: Isomap, LLE	PDF	Isomap: http://isomap.stanford.edu/ LLE: http://cs.nyu.edu/~roweis/lle/papers/lleintroa4.pdf Laplacian Eigenmaps: http://www.cse.ohio-state.edu/~mbelkin/papers/LEM_NC_03.pdf
28.11	Manifold Mapping -- Laplacian Eigenmaps. KPCA	PDF PDF	Laplacian Eigenmaps: http://www.cse.ohio-state.edu/~mbelkin/papers/LEM_NC_03.pdf KPCA: http://cseweb.ucsd.edu/classes/fa01/cse291/kernelPCA_article.pdf Comparison: here
5.12	Kmeans, Mixture of Gaussians, EM	PDF PDF	Lecture notes: kmeans, MoG
12.12	EM derivation, EM for MoG	PDF	Lecture notes: EM
19.12	Cont. Laten Variables: Factor Analysis	PDF	Lecture notes: FA, Bishop PRML Ch. 12.2
26.12	PPCA, LSI,	PDF PDF
2.12	ICA Intro to Graphical Models Part1	PDF PDF	Lecture notes: ICA http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf
9.12	Intro to Graphical Models Part I	PDF	http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf
16.12	Intro to Graphical Models part 2	PDF	http://research.microsoft.com/en-us/um/people/cmbishop/prml/Bishop-PRML-sample.pdf
23.12	HMM	PDF	Lecture notes: HMM

_________________________________________________________________

Textbook:

• Pattern Recognition and Machine Learning, by Christopher Bishop. Springer, August 2006.

______________________________________________________________________

Probability tutorials:

http://www.autonlab.org/tutorials/prob18.pdf - first half

http://www-stat.stanford.edu/~susan/courses/s116/

Linear Algebra tutorial:

Eigen value decomposition

______________________________________________________________________