Conditional probability of continuous variable

“The concept of a conditional probability with regard to an isolated hypothesis whose probability equals 0 is inadmissible.” A. Kolmogorov

For continuous random variables, $X$ and $Y$ say, conditional distributions are defined by the property that they recover the original probability measure, that is, for all measurable sets $A\in\mathcal{A}(\mathbf{X})$, $B\in\mathcal{B}(\mathbf{Y})$, $$\mathbb{P}(X\in A,Y\in B)=\int_B \text{d}P_Y(y) \int_B \text{d}P_{X|Y}(x|y)$$ This implies that the conditional density is defined arbitrarily on sets of measure zero or, on other words, that the conditional density $p_{X|Y}(x|y)$ is defined almost everywhere. Since the set $\{5,6\}$ is of measure zero against the Lebesgue measure, this means that you can define both $p(5)$ and $p(6)$ in absolutely arbitrary manners and hence that the probability $$\mathbb{P}(U=5|U\in\{5,6\})$$ can take any value.

This does not mean you cannot define a conditional density by the ratio formula $$f(y|x)=f(x,y)\big/f(x)$$ as in the bivariate normal case but simply that the density is only defined almost everywhere for both $x$ and $y$.

“Many quite futile arguments have raged – between otherwise competent
probabilists – over which of these results is ‘correct’.” E.T. Jaynes

The fact that the limiting argument (when $\epsilon$ goes to zero) in the above answer seems to give a natural and intuitive answer is related with Borel’s paradox. The choice of the parametrisation in the limit matters, as shown by the following example I use in my undergrad classes.

Take the bivariate normal $$X,Y\stackrel{\text{i.i.d.}}{\sim}\mathcal{N}(0,1)$$ What is the conditional density of $X$ given that $X=Y$?

If one starts from the joint density $\varphi(x)\varphi(y)$, the “intuitive” answer is [proportional to] $\varphi(x)^2$. This can be obtained by considering the change of variable $$(x,t)=(x,y-x) \sim \varphi(x)\varphi(t+x)$$ where $T=Y-X$ has the density $\varphi(t/\sqrt{2})/\sqrt{2}$. Hence $$f(x|t)=\dfrac{\varphi(x)\varphi(t+x)}{\varphi(t/\sqrt{2})/\sqrt{2}}$$ and $$f(x|t=0)=\dfrac{\varphi(x)\varphi(x)}{\varphi(0/\sqrt{2})/\sqrt{2}}=\varphi(x)^2\sqrt{2}$$ However, if one considers instead the change of variable $$(x,r)=(x,y/x) \sim \varphi(x)\varphi(rx)|x|$$ the marginal density of $R=Y/X$ is the Cauchy density $\psi(r)=1/\pi\{1+r^2\}$ and the conditional density of $X$ given $R$ is $$f(x|r)=\varphi(x)\varphi(rx)|x| \times \pi \{1+r^2\}$$ Therefore, $$f(x|r=1)= \pi\varphi(x)^2|x|/2\,.$$
And here lies the “paradox”: the events $R=1$ and $T=0$ are the same as $X=Y$, but they lead to different conditional densities on $X$.