Proof of closeness of kernel functions under pointwise product

How can I prove that pointwise product of two kernel functions is a kernel function?

Answer

By point-wise product, I assume you mean that if k1(x,y),k2(x,y) are both valid kernel functions, then their product

kp(x,y)=k1(x,y)k2(x,y)

is also a valid kernel function.

Proving this property is rather straightforward when we invoke Mercer’s theorem. Since k1,k2 are valid kernels, we know (via Mercer) that they must admit an inner product representation. Let a denote the feature vector of k1 and b denote the same for k2.

k1(x,y)=a(x)Ta(y),a(z)=[a1(z),a2(z),aM(z)]k2(x,y)=b(x)Tb(y),b(z)=[b1(z),b2(z),bN(z)]

So a is a function that produces an M-dim vector, and b produces an N-dim vector.

Next, we just write the product in terms of a and b, and perform some regrouping.

kp(x,y)=k1(x,y)k2(x,y)=(Mm=1am(x)am(y))(Nn=1bn(x)bn(y))=Mm=1Nn=1[am(x)bn(x)][am(y)bn(y)]=Mm=1Nn=1cmn(x)cmn(y)=c(x)Tc(y)

where c(z) is an MN -dimensional vector, s.t. cmn(z)=am(z)bn(z).

Now, because we can write kp(x,y) as an inner product using the feature map c, we know kp is a valid kernel (via Mercer’s theorem). That’s all there is to it.

Attribution
Source : Link , Question Author : Gigili , Answer Author : Mike Hughes

Leave a Comment