Consider a simple statistical model of a coin flip, with a single parameter that expresses the "fairness" of the coin. This parameter is the probability that a given coin lands heads up ("H") when tossed. can take on any numeric value within the range 0.0 to 1.0. For a perfectly fair coin, = 0.5.
Imagine flipping a coin twice, and observing the following data : two heads in two tosses ("HH"). Assuming that each successive coin flip is IID, then the probability of observing HH is
Hence: given the observed data HH, the likelihood that the model parameter equals 0.5, is 0.25. Mathematically, this is written as
This is not the same as saying that the probability that , given the observation HH, is 0.25. (For that, we could apply Bayes' theorem, which implies that the posterior probability is proportional to the likelihood times the prior probability.)
Suppose that the coin is not a fair coin, but instead it has . Then the probability of getting two heads is
More generally, for each value of , we can calculate the corresponding likelihood. The result of such calculations is displayed in Figure 1.
In Figure 1, the integral of the likelihood over the interval [0, 1] is 1/3. That illustrates an important aspect of likelihoods: likelihoods do not have to integrate (or sum) to 1, unlike probabilities.