Dimension Reduction and Metric Learning
STAT 946
News:
|
|
Wikicoursenote | ||
Project | ||
Course Outline | ||
Readings | ||
Resources | ||
Important Dates | ||
|
Fall 2009
Department of Statistics and
Actuarial Science
University of Waterloo
Instructor: Ali
Ghodsi
Room: MC 4063
Time: TTh 01:00-02:20
Office Hours: 2:300-3:30 T or by appointment
(MC 6081G)
Tutorials:
A. GhodsiGeometric methods for feature extraction and dimensional reduction. C. J. C. Burges.
Kernel PCA:
Kernel PCA pattern reconstruction
Nonlinear Component Analysis as a Kernel Eigenvalue Problem,
Locally Linear Embedding (LLE):
Nonlinear
dimensionality reduction by locally linear embedding.
Sam Roweis & Lawrence Saul. Science,
v.290 no.5500
, Dec.22, 2000. pp.2323--2326.
[Abstract]
[Full article
(html)
(pdf)]
Think Globally, Fit Locally: Unsupervised Learning of Nonlinear Manifolds.
Isomap:
A Global Geometric Framework for Nonlinear Dimensionality Reduction. Joshua B. Tenenbaum, Vin de Silva, and John C. Langford Science 22 December 2000 290: 2319-2323
MDS, Landmark MDS and Nystrom Approximation:
FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms
Maximum Variance Unfolding (MVU) / Semidefinite Embedding (SDE):
Learning a kernel matrix for nonlinear
dimensionality reduction. ,K. Q. Weinberger, F. Sha, and L. K. Saul
(2004).
In Proceedings of the Twenty First International Conference on Machine
Learning (ICML-04). (pdf)
Landmark SDE:
Action Respecting Embedding (ARE):
Clustering (Impossibility Theorem):
An Impossibility Theorem for Clustering.
K-means Clustering:
Metric Learning:
Distance metric learning with application to clustering with side-information. E. Xing, A. Ng, M. Jordan and S. Russell. In Proceedings of Advances in Neural Information Processing Systems 15 (NIPS 2003) (pdf)
Improving Embeddings by Flexible Exploitation of Side Information. A. Ghodsi, D. Wilkinson and F. Southey. In proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI 2006) (pdf)
Spectral Clustering
:A Tutorial on Spectral Clustering. Ulrike von Luxburg1 (pdf)
Resources:
|
Old Lectures from Fall 2006:
Lecture 1 and 2 | Motivation | |
Lecture 3 and 4 | Principal Components Analysis (PCA) Slides | |
Lecture 5 | PCA, Kernel function | |
Lecture 6 | Dual PCA, Kernel PCA Slides | |
Lectures 7 and 8 | Centering, Locally Linear Embedding (LLE) Slides (Examples are taken from this paper.) | |
Lecture 9 | Locally Linear Embedding | |
Project Discussion | ||
Thanksgiving | ||
Lectures 10 and 11 | Multidimensional Scaling (MDS), Isomap Slides | |
Lecture 12 | Nystrom Approximation, Landmark MDS | |
Lecture 13 | Landmark MDS | |
Lectures 14, 15 and 16 | Unified Framework, Semidefinite Embedding (SDE) | |
Lecture 17 | Landmark SDE | |
Lecture 18 | Action Respecting Embedding (ARE) | |
Lecture 19 | Clustering | |
Lectures 20 and 21 | Combinatorial Algorithms, K-means clustering | |
Lectures 22 and 23 | Mixture Models | |
Lectures 24 and 25 | Learning a Metric (Class-Equivalence Side Information) | |
Lecture 26 | Learning a Metric (Partial Distance Side Information) | |
Slides for some of the New Lectures
Oct 30 | Proposal due |
Dec 22 | Final project reports due |
Register the date of your presentation here
Note: You need to have a Wikicoursenote account. To obtain a user account, you must request one.
Chose a paper from this list
Final project reports (up to 8 pages of PDF) are worth 50% of your final grade .You are encouraged to chose a topic related to your research area. However, you cannot borrow part of an existing thesis work, nor can you re-use a project from another course.
Academic Dishonesty:
If you use ideas, plots, text and other intellectual property developed by someone else you have to cite the original source.
If you copy a sentence or a paragraph from work done by someone else, in addition to citing the original source you have to use quotation marks to identify the scope of the copied material.
Example:
Plagiarism is an act of “using ideas, plots, text and other intellectual property developed by someone else while claiming it is your original work.”1
References:
1. Tec Encyclopedia. http://www.answers.com/topic/plagiarism
Evidence of copying or plagiarism will cause a failing mark in the course.
Please attach this cover page to your report.
I use this marking scheme to mark the projects.