Data Visualization
STAT 890 / 442, CM 462,
News:



Fall 2006
Department of Statistics and
Actuarial Science
University of Waterloo
Instructor: Ali
Ghodsi
Room: B2 350
Time: MWF 1:302:20
Office Hours: Monday 3:00pm 4:00pm or by appointment (MC 6081G)
Tutorials:
Geometric methods for feature extraction and dimensional reduction. C. J. C. Burges.
Kernel PCA:
Kernel PCA pattern reconstruction via approximate preimages. B. SchÄolkopf, S. Mika, A. Smola, G. RÄatsch, and K.R. MÄuller. (pdf)
Nonlinear Component Analysis as a Kernel Eigenvalue Problem, Ian Fasel (pdf)
Locally Linear Embedding (LLE):
Nonlinear
dimensionality reduction by locally linear embedding.
Sam Roweis & Lawrence Saul. Science,
v.290 no.5500
, Dec.22, 2000. pp.23232326.
[Abstract]
[Full article
(html)
(pdf)]
Think Globally, Fit Locally: Unsupervised Learning of Nonlinear Manifolds. Lawrence Saul and Sam Roweis (pdf)
Isomap:
A Global Geometric Framework for Nonlinear Dimensionality Reduction. Joshua B. Tenenbaum, Vin de Silva, and John C. Langford Science 22 December 2000 290: 23192323
MDS, Landmark MDS and Nystrom Approximation:
Multidimensional Scaling. T. Cox and M. Cox., Number 59 in Monographs on Statistics and Applied Probability. Chapman & Hall, 1994.
FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms
Semidefinite Embedding (SDE):
Learning a kernel matrix for nonlinear
dimensionality reduction. ,K. Q. Weinberger, F. Sha, and L. K. Saul
(2004).
In Proceedings of the Twenty First International Conference on Machine
Learning (ICML04). (pdf)
Landmark SDE:
Action Respecting Embedding (ARE):
Clustering (Impossibility Theorem):
An Impossibility Theorem for Clustering. J. Kleinberg. In Proceedings of Advances in Neural Information Processing Systems 15, (NIPS 2003) (pdf)
Kmeans Clustering:
Metric Learning:
Distance metric learning with application to clustering with sideinformation. E. Xing, A. Ng, M. Jordan and S. Russell. In Proceedings of Advances in Neural Information Processing Systems 15 (NIPS 2003) (pdf)
Improving Embeddings by Flexible Exploitation of Side Information. A. Ghodsi, D. Wilkinson and F. Southey. In proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI 2006) (pdf)
Spectral Clustering:
A Tutorial on Spectral Clustering. Ulrike von Luxburg1 (pdf)
Resources:

Sept 11 and Sept 13  Lecture 1 and 2  Motivation 
Sep 18 and Sept 20  Lecture 3 and 4  Principal Components Analysis (PCA) 
Sep 22  Lecture 5  PCA, Kernel function 
Sep 25  Lecture 6  Dual PCA, Kernel PCA 
Sep 27 and Sep 29  Lectures 7 and 8  Centering, Locally Linear Embedding (LLE) Slides (Examples are taken from this paper.) 
Oct 4  Lecture 9  Locally Linear Embedding 
Oct 6  Project Discussion  
Oct 9  Thanksgiving  
Oct 11 and 13  Lectures 10 and 11  Multidimensional Scaling (MDS), Isomap Slides 
Oct 16  Lecture 12  Nystrom Approximation, Landmark MDS 
Oct 18  Lecture 13  Landmark MDS 
Oct 20, 23 and 25  Lectures 14, 15 and 16  Unified Framework, Semidefinite Embedding (SDE) 
OCT 27  Lecture 17  Landmark SDE 
Oct 30  Lecture 18  Action Respecting Embedding (ARE) 
Nov 1  Lecture 19  Clustering 
Nov 3 and 6  Lectures 20 and 21  Combinatorial Algorithms, Kmeans clustering 
Nov 8 and 10  Lectures 22 and 23  Mixture Models 
Nov 13 and Nov 15  Lectures 24 and 25  Learning a Metric (ClassEquivalence Side Information) 
Nov 17  Lecture 26  Learning a Metric (Partial Distance Side Information) 
Assignment 1 Data for Assignment 1
Assignment 2 Data for Assignment 2 code
November 20  Presentations will start 
October 23  Proposal due 
November 3  Takehome exam 
December 20  Final project reports due 
Final project reports (up to 8 pages of PDF) are worth 25% of your final grade .You are encouraged to chose a topic related to your research area. However, you cannot borrow part of an existing thesis work, nor can you reuse a project from another course.
Due Date: Final project reports are due December 20 .Hand in your report to Joan Hatton at MC 6028 by 4:00 pm.
Academic Dishonesty:
If you use ideas, plots, text and other intellectual property developed by someone else you have to cite the original source.
If you copy a sentence or a paragraph from work done by someone else, in addition to citing the original source you have to use quotation marks to identify the scope of the copied material.
Example:
Plagiarism is an act of “using ideas, plots, text and other intellectual property developed by someone else while claiming it is your original work.”^{1}
References:
1. Tec Encyclopedia. http://www.answers.com/topic/plagiarism
Evidence of copying or plagiarism will cause a failing mark in the course.
Please attach this cover page to your report.
I use this marking scheme to mark the projects.