What is the difference between the dot product and the element by element multiplication?

What is the different between the dot product” and the element-by-element multiplication notation in Statistics? I referred to Hamilton’s Time-Series Analysis, and these seem to have the same definition. For instance, for two n×1 vectors a and b, is
ab?ab=ni=1aibi

Answer

The difference operationally is the aggregation by summation. With the dot product, you multiply the corresponding components and add those products together. With the Hadamard product (element-wise product) you multiply the corresponding components, but do not aggregate by summation, leaving a new vector with the same dimension as the original operand vectors. And on that point, the dot product of two vectors gives a scalar number while the Hadamard product of two vectors gives a vector.

c=xTy=[x1x2xn]T[y1y2yn]=[x1,x2,,xn][y1y2yn]=ni=1xiyiR
z=xy=[x1x2xn][y1y2yn]=[x1y1x2y2xnyn]Rn

Note that x,y must have the same dimension. The Hadamard product is often defined in terms of matrices in many sources, but for general tensors it suffices that they have the same shape. For two vectors, this is tantamount to having the same dimension.

So in general xyxy.

Furthermore, and \|\vec{x} \cdot \vec{y}\| \leq \|\vec{x}\| \|\vec{y}\|. But since Pr\left( \|\vec{x} \cdot \vec{y}\| \leq \|\vec{x} \odot \vec{y}\| \right) \approx .67 for some choice of probability space (see link on \|\vec{x} \odot \vec{y}\| \leq \|\vec{x}\|\|\vec{y}\|), there isn’t a direct inequality between their norms.

Footnote: The dot product is also an example of an inner product while the Hadamard product is not. This is important for certain forms of rotational invariance, but that is not really a ‘statistics’ SE point.

Attribution
Source : Link , Question Author : Carl , Answer Author : DifferentialCovariance

Leave a Comment