CSE6740大纲

CSE6740大纲

k-means

• k means algorithm
• similarity and dissimilarity function
• distance functions for vectors(Euclidian distance, Minkowski distance, Hamming distance, edit distance)
• hierarchical clustering(bottom up hierarchical clustering algorithm)

clustering nodes in graph

• spectral clustering algorithm
• graph laplacian matrix and its property
• high level idea of spectral clustering
• neighbor graph and spectral clustering for vectorial data
• the solutions to large graph

dimensionality reduction and principal component analysis:

• dimensionality reduction and its application
• principal component analysis
• the criterions for reduction
• how to formulate the problem
• eigen value problem and SVD

Nonliear dimensionality reduction

• dimensionality reduction data recover to original data point
• limitation of PCA and SVD
• ISOmap to solve above limitations
• use isomap to obtain low dimensional representation
• after ISOmap, recover data point to original data point

density estimation

• why do we need density estimation?
• parametric models and non parametric models
• estimation of parametric models(MLE)
• MLE for Guassian distribution
• estimation of non parametric models(1D histogram and higher dimensional histogram and their disadvantages)
• estimation of non parametric models(kernel density estimator)

mixture of gaussian:

• the comparison between parametric and non parametric
• Gaussian mixture model(EM algorithm(expectation step and maximization step))
• bayes rules
• the comparison between EM and modified K-means

EM

• EM algorithm
• convex sets-common convex sets-operations that preserve convex sets-convex function-concave functions-first order conditions-second order conditions-operations that preserve convexity
• theory underlying EM
• jensen’s inequality
• lower bound of log-likelihood

feature selection

• A feature selection algorithm
• Informativenessof a feature(特征的信息性) – entropy（度量uncertainty）
• formula to computing entropy
• conditional entropy
• mutual information(reduction in uncertainty)

bayes decision rule/naive bayes/Nearest Neighbor classifier/ logistic regression

• how to come up with decision boundary? use bayes rules and bayes decision rule.
• example: gaussian class conditional distribution
• what do people do in practice?
• naive bayes classifier
• nearest neighbor classifier(KNN classifier)
• KNN algorithm
• discriminative classifier
• logistic regression model
• learning parameters in logistic regression
• gradient ascent/descent algorithm

SVM(use geometric intuitions to design classifier)

• three ways to design classifier(bayes rule+assumption for P(x|y=1)/use geometric intuitions/directly go for the decision boundary)
• batch gradient compared with stochastic gradient
• multiclass logistic regression model
• learning parameters in multiclass logistic regression
• classifier margin’maximum margin classifier
• SVM(support vector machines)
• lagrangian duality/the KTT conditions/dual problem of support vector machines/deriving the dual problem/the dual problem of SVM/support vectors/interpretation of SVM/soft margin constraints/ hard margin constraints/soft margin SVM/deriving the dual problem.

由于后面的PPT实在是太乱，所以先把课看完过一遍再来整理一下