The detector says AI. Should you believe it?
Part 2 ended on a zero. Four phrases in the corpus start with my, none of them is my mat, and a model that trusts its counts calls my mat impossible. The fix is Bayes’ rule.
The clearest place to learn that rule is somewhere the stakes are higher. Our model now writes whole essays, some students hand them in, and schools run them through AI detectors. A detector is one more model that outputs a probability, and reading that probability the wrong way round gets students accused.
The 30-second version
About 1 in 6. Picture 10,000 essays. 100 are AI-written and the detector flags 99 of them. The other 9,900 are human, and 5% of those, 495, get flagged too. So of 594 flags only 99 are right. The 99% is the chance of a flag given an AI essay; the teacher needs the chance of an AI essay given a flag. Bayes' theorem turns one into the other, and the base rate, 1 in 100, does most of the work.Why does “99% accurate” answer the wrong question?
Because it is a statement about AI-written essays: of those, 99% get flagged. That is $p(\text{flag} \mid \text{AI}) = 0.99$. A teacher holding a flagged essay needs the reverse, $p(\text{AI} \mid \text{flag})$, and the two can be far apart.
Count it. Step 1 of the figure puts a class of 10,000 essays on a grid and runs the detector over it. Step 2 asks you the puzzle, then pulls every flagged essay into one pile.
Lay the counts out as a table, the way Part 2 laid out its phrases:
| flagged | not flagged | total | |
|---|---|---|---|
| AI-written | 99 | 1 | 100 |
| human | 495 | 9,405 | 9,900 |
| total | 594 | 9,406 | 10,000 |
“99% accurate” reads along the AI row: 99 of 100. The teacher’s question reads down the flagged column: 99 of 594, about 17%. Same counts, a different denominator, and the denominator is the whole story.
Where does Bayes’ theorem come from?
From Part 2’s product rule, written both ways round. Part 2 built the joint probability of the cat two ways and got 0.25 both times. Bayes’ theorem is that equality solved for one of its pieces (MML eqs. 6.24 to 6.26):
\[p(\text{AI}, \text{flag}) = p(\text{flag} \mid \text{AI})\,p(\text{AI}) = p(\text{AI} \mid \text{flag})\,p(\text{flag})\]Divide by $p(\text{flag})$ and every piece gets a name (MML eq. 6.23):
\[\underbrace{p(\text{AI} \mid \text{flag})}_{\text{posterior}} = \frac{\overbrace{p(\text{flag} \mid \text{AI})}^{\text{likelihood}}\;\overbrace{p(\text{AI})}^{\text{prior}}}{\underbrace{p(\text{flag})}_{\text{evidence}}} = \frac{0.99 \times 0.01}{0.0594} = \frac{1}{6}\]In words: the prior is what you believed before the flag, 1 in 100. The likelihood is how readily an AI essay produces a flag. The evidence is how often flags happen at all. MML calls the theorem “the probabilistic inverse”: it turns $p(y \mid x)$ into $p(x \mid y)$.
The evidence is the flagged column’s total, and Part 2’s sum rule computes it: flags from AI essays plus flags from human ones.
\[p(\text{flag}) = \underbrace{0.99 \times 0.01}_{\text{true alarms}} + \underbrace{0.05 \times 0.99}_{\text{false alarms}} = 0.0099 + 0.0495 = 0.0594\]The false alarms are five times the true ones. That 1 to 5 is the puzzle’s answer.
Why does the base rate decide?
Because a flag is news about something rare. When only 1 essay in 100 is AI-written, a small false-alarm rate on the other 99 swamps the true alarms.
Run the same detector in a class where 30% of essays are AI-written and a flag means 89%. Nothing about the detector changed. Only the prior did.
The dials make the lesson concrete. Push the catch rate to 99.9% and a flag still means 17%. Cut false alarms to 1% instead and it means 50%. For a rare event, the false-alarm rate is the number to watch, and it is the one a detector’s “accuracy” never mentions.
What does a second flag add?
Chaining Bayes is easiest in odds: the ratio of the two hypotheses. Before any flag the odds are 1 AI essay to 99 human ones. A flag multiplies them by the likelihood ratio, how much more often an AI essay is flagged than a human one:
\[\frac{p(\text{AI} \mid \text{flag})}{p(\text{human} \mid \text{flag})} = \frac{p(\text{AI})}{p(\text{human})} \times \frac{p(\text{flag} \mid \text{AI})}{p(\text{flag} \mid \text{human})} = \frac{1}{99} \times \frac{0.99}{0.05} = \frac{1}{5}\]The evidence cancels in the ratio, so there is nothing to sum. One flag gives 1 : 5, the puzzle’s 1 in 6. A second detector flags it too: multiply by 19.8 again, about 4 : 1, or 80%.
That second multiplication is honest only if the two detectors’ mistakes are independent once you know the truth, which is MML’s conditional independence (Definition 6.11). Two detectors trained on the same data tend to fail on the same essays, and then the second flag is worth far less than 19.8.
How does a model learn a probability from data?
Back to the zero. A model that trusts the count says $p(\textit{mat} \mid \textit{my}) = 0/4 = 0$. Bayes treats that unknown probability itself as the uncertain thing. Call it $\theta$.
Each my phrase is a coin flip that lands on mat with probability $\theta$. Four flips with no mat have probability $(1 - \theta)^4$: the likelihood, one number for every candidate $\theta$.
Four misses barely rule out small values. $\theta = 0.1$ explains them $0.9^4 = 0.66$ times as well as $\theta = 0$ does. The count keeps only the single best $\theta$ and throws the rest away.
Bayes keeps all of them. Start from a prior that calls every $\theta$ equally plausible, $\mathrm{Beta}(1, 1)$, and multiply by the likelihood. For coin flips the posterior is a Beta again, with the counts added to its two parameters (MML Example 6.12):
\[\mathrm{Beta}(1, 1) \;\xrightarrow{\;0 \text{ mats in } 4\;}\; \mathrm{Beta}(1 + 0,\; 1 + 4) = \mathrm{Beta}(1, 5)\]A prior whose posterior has its own form is called conjugate (MML Definition 6.13). Averaged over every plausible $\theta$, the chance that the next my phrase ends in mat is the posterior’s mean, $1/(1 + 5) = 1/6$. Not 0.
The prior acts like imagined data: one imagined mat and one imagined non-mat added to the real counts, $(0 + 1)/(4 + 2)$. That is add-one smoothing. MML’s warning about priors applies word for word: give nonzero probability to every plausible value, “even if they are very rare”. A zero, once in, survives any amount of data, because zero times any likelihood is zero.
What does this have to do with training a network?
Training finds parameters from data, millions of them instead of one $\theta$. Maximum likelihood keeps only the likelihood and takes its peak, the count; plain training does exactly this. Maximum a posteriori (MAP) estimation takes the posterior’s peak instead, prior included (MML §8.3.2):
\[\theta_{\text{MAP}} = \arg\max_\theta\; p(\text{data} \mid \theta)\,p(\theta) = \arg\min_\theta\; \big[-\log p(\text{data} \mid \theta) - \log p(\theta)\big]\]Put a Gaussian prior centred on 0 on every weight. Its $-\log$ is $\lVert\theta\rVert^2 / 2\sigma^2$ plus a constant: a penalty on large weights, which is weight decay. Every weight decay you set is a prior you chose.
How much a prior matters depends on how much it counts for. A $\mathrm{Beta}(\alpha, \beta)$ prior counts like $\alpha + \beta$ imagined observations.
A strong prior outweighs four phrases and fades against two hundred. MAP keeps only the peak of the posterior and throws away its spread; the full posterior keeps both, which is what gave my mat its 1/6.
The model is trained now, and its probabilities are sensible. The next question is the one every lab asks every week: is it any good? Measuring that is Part 4, and the measurement turns out to be random too.
Rapid fire: can you do these from memory?
- A detector flags 90% of AI essays and 10% of human ones. In a class where 10% use AI, what does a flag mean?
- In "the detector is 99% accurate", which conditional probability is that, and which one does a teacher need?
- Derive Bayes' theorem from the product rule in two lines, and name its four parts.
- When AI essays are rare, why does cutting false alarms help more than raising the catch rate?
- Start from odds 1 : 99 and add two flags, each with likelihood ratio 19.8. What are the odds now, and what did you assume about the two detectors?
- A Beta(1, 1) prior sees 2 mats in 8 phrases. What is the posterior, and the chance that the next phrase ends in mat?
- Why can no amount of data revive a probability that a raw count set to 0, and how does a prior prevent it?
- Which prior is weight decay, and what does MAP throw away that the full posterior keeps?
Where can you go deeper?
All of this is in Mathematics for Machine Learning by Deisenroth, Faisal and Ong (Cambridge University Press, 2020), free at mml-book.github.io:
- §6.3, Bayes’ theorem: eq. 6.23 with its prior, likelihood, posterior and evidence, derived from the product rule in eqs. 6.24 to 6.26. The evidence as an expected likelihood is eq. 6.27. This section also calls Bayes “the probabilistic inverse”, warns that a prior must be nonzero on every plausible value, and notes that a likelihood is a distribution over the data, not over the parameter.
- §6.4.5, Definition 6.11: conditional independence, the assumption behind multiplying two detectors’ likelihood ratios.
- §6.6.1, conjugacy: Definition 6.13, Example 6.11 (Beta-Binomial), Example 6.12 (Beta-Bernoulli, this part’s my mat), and Table 6.2.
- §8.3.2, maximum a posteriori estimation: eqs. 8.19 and 8.20, and Example 8.6’s Gaussian prior on the parameters. §8.2.3 is the same idea seen as regularization.