Applied Statistics Notes for [1 Probability]

Stirling's approximation

• \(\lim_{n\rightarrow\infty}n!=\frac{\sqrt{2\pi}n^{n+\frac{1}{2}}}{e^n}\)

Proof

只需证：\(\lim_{n\rightarrow\infty}\frac{n!e^n}{n^{n+\frac{1}{2}}}=\sqrt{2\pi}\)

Part 1

Let \(a_n:=\frac{n!e^n}{n^{n+\frac{1}{2}}}\)

\(\frac{a_n}{a_{n+1}}=\frac{(n+1)^{n+1+\frac{1}{2}}}{n^{n+\frac{1}{2}}e}=\left(1+\frac{1}{n}\right)^{n+\frac{1}{2}}\frac{1}{e}\)

\(\frac{a_n}{a_{n+1}}\) is monotone decreasing.

\(\because\frac{a_1}{a_{2}}>1\)

And \(\because\lim_{n\rightarrow \infty}\frac{a_n}{a_{n+1}}=1\)

\(\therefore \forall n\in\mathbb N, \frac{a_n}{a_{n+1}}>1\), which shows \(a_n>a_{n+1}\)

\(\because \ln n!>(n+\frac{1}{2})\ln n-n\)

• Proof:

• \(\because \ln x-\frac{1}{x}>\ln (x-1)\)

• \(\therefore \int_{t-1}^{t}\ln x\text{d}x<\frac{\ln t-\frac{1}{t}+\ln t}{2}\)

• \(\therefore \int_{1}^n \ln x\text{d}x=n\ln n-n+1<\sum_{t=2}^n(\ln t-\frac{1}{2t})=\ln n!-\frac{1}{2}\sum_{t=2}^n\frac{1}{t}\)

• \(\because \sum_{i=1}^n\frac{1}{i}<\ln n\)

• \(\therefore n\ln n-n+1<\ln n!-\frac{1}{2}(\ln n-1)\)

• \(\therefore \ln n!>(n+\frac{1}{2})\ln n-n+\frac{1}{2}>(n+\frac{1}{2})\ln n-n\)

(待upd)

Axiomatic Approach to Probability

• Axiom I: \(0\leq P[A]\)

• Axiom II: \(P[S]=1\)

• Axiom III: if $ A\cap B=\emptyset$, then \(P[A\cup B]=P[A]+P[B]\)

Corollary

• For any events \(A\) and \(B\), \(P[A\cup B]=P[A]+P[B]-P[A\cap B]\)

Conditional Probability

\[P[A|B]=\frac{P[A\cap B]}{P[B]} \]

Properties of Conditional Probability

• If \(B\subseteq A\), then \(P[A|B]=1\)

• If \(A\subseteq B\), then \(P[A|B]=\frac{P[A]}{P[B]}\geq P[A]\)

• If \(A_1\) and \(A_2\) are mutually exclusive then

\[P[A_1\cup A_2|B]=P[A_1|B]+P[A_2|B] \]

Total Probability Theorem

If \(B_1, B_2,\cdots B_n\) partition the sample space, then for any event \(A\),

\[P[A]=\sum_{i=1}^nP[A|B_i]P[B_i] \]

Bayes' Theorem

\(P[B_j|A]=\frac{P[AB_j]}{P[A]}=\frac{P[A|B_j]P[B_j]}{\sum_{k=1}^nP[A|B_k]P[B_k]}\)

Confusion Matrix

Precision

• \(\text{Precision}=\frac{TP}{TP+FP}\)

Recall

• \(\text{Recall}=\frac{TP}{TP+FN}\)

Accuracy

• \(\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\)

F1-score

\(\text{F1-score}=\frac{2\times \text{Precison}\times \text{Recall}}{\text{Precision}+\text{Recall}}\)

[CSDN] FP、FN、TP、TN、精确率(Precision)、召回率(Recall)、准确率(Accuracy)等评价指标介绍 - 不吃香菜的小趴菜

Sensitivity

\(\text{Sensitivity}=\frac{TP}{TP+FN}\)

Specificity

\(\text{Specificity}=\frac{TN}{TN+FP}\)

ROC curve

[cnblogs] 小白也能看懂的 ROC 曲线详解 - PrimiHub

[ResearchGate] Introduction to ROC analysis - Tom Fawcett

Independence

• Two events \(A\) and \(B\) are independent if \(P[AB]=P[A]\times P[B]\)

Independent vs Disjoint

\(A\) and \(B\) are Independent \(\Leftrightarrow\) \(P[A\cap B]=P[A]P[B]\)

\(A\) and \(B\) are disjoint \(\Leftrightarrow\) \(A\cap B=\emptyset\)