Accounting for discrete or binary parameters in Bayesian information criterion

BIC penalizes based on the number of parameters. What if some of the parameters are some sort of binary indicator variables? Do these count as full parameters? But I can combine m binary parameters into one discrete variable that takes values in {0,1,...,2m1}. Are these to be counted as m parameters or one parameter?

Answer

It is partly because of this imprecision in the “number of parameters” in BIC that DIC (the deviance information criterion) introduced an effective number of parameters as
pD(x)=E[D(θ)|x]D(E[θ|x])
where
D(θ)=2logf(x|θ)
and
DIC(x)=pD(x)+E[D(θ)|x]
Note that pD(x) is then data-dependent. (As discussed there, DIC also has problems of its own!)

Attribution
Source : Link , Question Author : highBandWidth , Answer Author : Xi’an

Leave a Comment