Introduction to Machine Learning

Instructor: Dr. Rita Osadchy

e-mail: rita [at]cs [dot]haifa.ac.il
Office: Jacobs 410
__________________________________________________________________

Course Description

Machine learning is concerned with the development of computer algorithms that are able to learn solving tasks given a set of examples of those tasks and some prior knowledge about them. Machine learning has a wide spectrum of applications including handwritten or speech recognition, image classification, medical diagnosis, stock market analysis, bioinformatics etc. The goal of this course is to present the main concepts of modern machine learning methods including some theoretical background.

Recommended Prerequisites

The course assumes some basic knowledge of probability theory and linear algebra,
for example, you should be familiar with

• Joint and marginal probability distributions
• Normal (Gaussian) distribution
• Expectation and variance
• Statistical correlation and statistical independence

Tutorials of the above topics.

Problems, Concepts, Methods, and Tools within in the course

The list is partial and be can changed.

Problems

• Regression and classification
• Density estimation
• Clustering
• Model selection
• Inference

Concepts

• Estimation, bias, variance, loss, Empirical risk, maximum likelihood
• Generalization, overfitting
• Regularization
• Capacity, VC-dimension
• Generative/discriminative models

Models and Methods

• Linear regression
• Classification Linear Models
• Non-parametric Models
• Support Vector Machine (SVM)
• Decision Trees
• Boosting

Tools

• Cross-validation
• Gradient descent

The course will furthermore use several real-life applications to illustrate the interest of statistical machine learning.

Requirements

1) Home assignments 0-20% of the final grade (could be done in pairs but the pairs should be the same for all assignments).

2) Final exam 80-100%

Home Assignments:

General Instructions

· We will have 2-3 assignments this semester.

· You should submit a pdf file of the report and your implementation (running code) in a digital form. Zip it together and submit in moodle.

· Identical (or very similar solutions) are not allowed!

Textbooks:

• R. O. Duda, P. E. D. Hart, and G.Stork, Pattern Classification New York, NY: Wiley, 2000.
• T. Hastie, R. Tibshirani, and J. Friedman: Elements of Statistical Learning, Springer-Verlag, 2001.
• Christopher Bishop, Pattern Recognition and Machine Learning. Springer, August 2006.

Probability tutorials:

http://www-stat.stanford.edu/~susan/courses/s116/

Linear Algebra tutorial:

Eigen value decomposition

MATLAB resources:

Matlab is installed in the computer labs in Jacobs building.
For a student license see:
http://www.haifa.ac.il/index.php/he/2015-11-19-07-16-50

Introductory Tutorial

MATLAB tutorial from Carnegie Mellon University

  Slightly more advanced Tutorial

• MATLAB tutorial from Michigan Technological University

  More complete references/tutorials/FAQs

• The MathWorks home page for MATLAB