Models forget their priors

Every base model is, in some sense, a fossil — a snapshot of the prior distribution of a moment that has already passed. We use them as if they speak the present, but they speak a past we’ve stopped writing down.

This is the strange thing about working with them: they’re always slightly out of phase. The world moves; the weights don’t. The interesting question isn’t whether this matters, but where, and to whom, and how soon you notice.