Decision Tree & Random Forest

1 Decision Tree

1.1 Information content

This measures the surprise or information gained when an event x occurs:

$$I(x)=-log_{2}P(x)$$

• if P(x) is high, then I(x) is small
• if P(x) is low, then  I(x) is large

 

1.2 Information Entropy