I’m looking for an answer that would satisfy a reader who understands frequentist p-values but only understands the rudiments of Bayesian approaches to statistics.
At present google searches do not reveal a definition either on a Wikipedia page or any other commonly accepted resource.
This question seems related but isn’t really since it transpired that the user was not actually calculating Bayesian p-values. However, the accepted answer links to this Gelman paper in explanation of what Bayesian p-values are.
If I understand it correctly, then a Bayesian p-value is the comparison of a some metric calculated from your observed data with the same metric calculated from your simulated data (being generated with parameters drawn from the posterior distribution).
In Gelmans words: “From a Bayesian context, a posterior p-value is
the probability, given the data, that a future observation
is more extreme (as measured by some test variable)
than the data”
For example, the number of zeros generated from a poisson based model could be such a metric or test statistic, and you could calculate how many of your simulated datasets have a larger fraction of zeros than you actually observe in your real data. The closer this value to 0.5, the better the values calculated from your simulated data distribute around the real observation.