Likelihood vs probability
Probability and likelihood use the same formula but answer opposite questions. Probability fixes the parameter (say, a fair coin) and asks how likely different data are. Likelihood fixes the data you actually got and asks how well each possible parameter value explains them.
The short version
- Probability: P(data | parameter). The parameter is fixed; the data vary. Probabilities of all possible outcomes add up to 1.
- Likelihood: L(parameter | data) = P(data | parameter), read as a function of the parameter with the data held fixed. Likelihoods do not add up to 1 and are not probabilities that a parameter is correct.
- The maximum likelihood estimate (MLE) is the parameter value that makes the observed data most probable.
- Likelihoods are meaningful only in comparison: the likelihood ratio says how much better one value explains the data than another.
Same formula, two questions
Flip a coin 10 times. If the chance of heads on each flip is p, the probability of getting exactly k heads is given by the binomial formula:
P(k heads) = C(10, k) × pᵏ × (1 − p)¹⁰⁻ᵏ
where C(10, k) is the number of ways to choose which k flips are heads (for k = 7, C(10, 7) = 120).
Probability: fix p, vary the data
Assume a fair coin, p = 0.5. Each specific sequence of 10 flips has probability 0.5¹⁰ = 1/1,024. So:
| Heads (k) | Probability |
|---|---|
| 5 | 252/1,024 ≈ 0.246 |
| 6 | 210/1,024 ≈ 0.205 |
| 7 | 120/1,024 ≈ 0.117 |
| 8 | 45/1,024 ≈ 0.044 |
| 9 | 10/1,024 ≈ 0.010 |
| 10 | 1/1,024 ≈ 0.001 |
Across all eleven possible outcomes (0 to 10 heads), these probabilities add up to exactly 1. That is what makes them a probability distribution.
Likelihood: fix the data, vary p
Now suppose you actually flipped and got 7 heads. The data are fixed; what you do not know is p. Plug each candidate p into the same formula with k = 7:
| Candidate p | L(p) = 120 × p⁷ × (1 − p)³ |
|---|---|
| 0.3 | 0.009 |
| 0.5 | 0.117 |
| 0.6 | 0.215 |
| 0.7 | 0.267 |
| 0.8 | 0.201 |
| 0.9 | 0.057 |
This column is the likelihood function. It is not a probability distribution over p: if you add up (integrate) the likelihood over every value of p from 0 to 1, you get 1/11 ≈ 0.09, not 1. A likelihood of 0.267 does not mean "a 26.7% chance that p is 0.7". It means "if p were 0.7, the observed data would have had a 26.7% chance of occurring".
Maximum likelihood and likelihood ratios
Maximum likelihood estimation
The likelihood peaks at p = 0.7. That value is the maximum likelihood estimate: the parameter that makes what you saw most probable. For a binomial, the MLE is always the observed proportion, heads ÷ flips = 7 ÷ 10 = 0.7, which matches intuition. In practice statisticians maximize the log-likelihood (ln L) instead: the logarithm turns products of many probabilities into sums, which are easier to work with, and it peaks at the same place. Maximum likelihood is the workhorse behind logistic regression, many models in genetics and evolution, and much of modern statistics.
The likelihood ratio
To compare two candidate values, divide their likelihoods:
L(0.7) ÷ L(0.5) = 0.267 ÷ 0.117 ≈ 2.3
The data are about 2.3 times more probable if p = 0.7 than if the coin is fair. That is a modest ratio; 7 heads in 10 is not strong evidence against a fair coin. A formal likelihood ratio test uses 2 × ln(ratio) = 2 × ln(2.28) ≈ 1.65, which for large samples follows approximately a chi-square distribution with 1 degree of freedom (one parameter estimated). That gives p ≈ 0.20: no good reason to reject a fair coin from only 10 flips. (With so few flips the approximation is rough, but the conclusion is the same.)
The ratio approach also explains the LOD score used in genetic linkage, which is the base-10 logarithm of a likelihood ratio; see finding genes in families.
Where Bayes comes in
If you really want "the probability that p is 0.7", you need more than the likelihood: you need a prior, a statement of what you believed about p before the data. Bayes' theorem combines them: the posterior probability of a parameter value is proportional to its likelihood times its prior probability. With a flat prior (every p from 0 to 1 equally plausible beforehand), the posterior after 7 heads in 10 peaks at 0.7, the same as the MLE, with a mean of 8/12 ≈ 0.67. With a strong prior that coins are nearly always fair, the posterior stays much closer to 0.5. Likelihood is the part of the calculation that comes from the data; Bayesian and classical (frequentist) statistics both use it but treat the parameter differently.
Memory aid
"Probability predicts; likelihood looks back." Probability looks forward from a known coin to the flips that might happen. Likelihood looks back from flips that did happen to the coins that might have made them.
A picture: probability is a weather forecast (given today's conditions, what will tomorrow bring?). Likelihood is a detective (given the footprints, which suspect fits best?). The detective compares suspects; no single suspect's "fit score" is a probability of guilt until you add what you knew beforehand (the prior).
A verse to remember it
Check yourself
You get 3 heads in 4 flips. What is the maximum likelihood estimate of p?
3 ÷ 4 = 0.75, the observed proportion.
Why can't you say "the likelihood of p = 0.7 is 0.267, so there is a 26.7% chance p is 0.7"?
Because likelihood is the probability of the data given p, not the probability of p given the data. Likelihoods over all p do not add up to 1. Turning them into probabilities of p requires a prior (Bayes' theorem).
What does a likelihood ratio of 100 mean?
The observed data are 100 times more probable under one parameter value (or hypothesis) than under the other.
Why do statisticians maximize the log-likelihood instead of the likelihood?
The log turns a product of many small probabilities into a sum, which is easier to calculate and differentiate and avoids numbers too small for computers. The maximum is at the same parameter value.