How do sample weights work in classification models?

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain examples? I would love going into the details to unpack how these algorithms leverage weights.

If you look at the sklearn documentation for logistic regression, you can see that the fit function has an optional sample_weight parameter which is defined as an array of weights assigned to individual samples.

Answer

As Frans Rodenburg already correctly stated in his comment, in most cases instance or sample weights factor into the loss function that is being optimized by the method in question.

Consider the equation the documentation provides for the primal problem of the C-SVM

min

Here C is the same for each training sample, assigning equal ‘cost’ to each instance. In the case that there are sample weights passed to the fitting function

“The sample weighting rescales the C parameter, which means that the
classifier puts more emphasis on getting these points right.”

As this example puts it, which also provides a nice visualization, showing how instances represented by bigger circles (those with larger weight) influence the decision boundary.

enter image description here

Attribution
Source : Link , Question Author : Jane Sully , Answer Author : deemel

Leave a Comment