随笔分类 - cs229
摘要：Independent Components Analysis Ambiguity ICA is ambiguous to scaling and permutation. but usually it doesn't matter. As long as the data is not Gauss
阅读全文
摘要：Principal Components Analysis Intuition PCA tries to identify the subspace in which the data approximately lies. Intuitively, we choose a direction fo
阅读全文
摘要：Decision Tree Greedy, Top-down, Recurrent Classification Tree misclassification loss is not suitable for decision tree loss, because $$ L(R_p) - (\lam
阅读全文
摘要：Factor Analysis Background when m (number of samples) < n (dimension of samples), the covariance matrix might be singular. So it is necessary to find
阅读全文
摘要：EM Algorithm Jensen's inequality convex function: $f''(x) \ge 0$ or $H \ge 0$ (Hessian matrix when x is a vector) $$ E[f(x)] \ge f(EX) $$ EM Algorithm
阅读全文
摘要：Reinforcement Learning Background Credit Assignment Problem: Explore how actions in an action sequence contribute to the outcome finally. MDP(Markov D
阅读全文
摘要：Learning Mathematical Theorem Feature Selection If we have n features, the VC dimension of hypothesis class would be O(n), if n>>m, over-fitting would
阅读全文
摘要：Support Vector Machine Two Margins Functional Margin $\gamma$ and Geometrical Margin $\hat{\gamma}$ $$ \hat{\gamma^{(i)}}=\frac{\gamma^{(i)}}{||w||} $
阅读全文
摘要：Error Theorem in ML Basic Concepts $$ \varepsilon(h) = \text{Generalization Error} = E_{(x,y)\sim D}[1(h(x) \neq y)] $$ $$ \hat{\varepsilon}(h)=\text{
阅读全文
摘要：Exponential Family The pdf is as follows, $$ p(y;\eta)=b(y) e^{\eta^TT(y)-a(\eta)} $$ Mathematical properties: MLE w.r.t. $\eta$ is concave, but negat
阅读全文
摘要：Locally Weighted Regression None parametric learning algorithm. Need to keep training data in the memory. Formally fit $\theta$ to minimize $$ \sum_{i
阅读全文
摘要：cs229 Generative Model & Discriminative Model Defination learns P(x|y) : x is feature and y is the class Bayes rule $$ P(y|x)=\frac{P(x|y)P(y)}{P(x)}
阅读全文
浙公网安备 33010602011771号