Dealing with ties, weights and voting in kNN

I am programming a kNN algorithm and would like to know the following:

Tie-breaks:

  1. What happens if there is no clear winner in the majority voting? E.g. all k nearest neighbors are from different classes, or for k=4 there are 2 neighbors from class A and 2 neighbors from class B?
  2. What happens if it is not possible to determine exactly k nearest neighbors because there are more neighbors which have the same distance? E.g. for the list of distances (x1;2), (x2;3.5), (x3;4.8), (x4;4.8), (x5;4.8), (x6;9.2) it would not be possible to determine the k=3 or k=4 nearest neighbors, because the 3rd to 5th neighbors all have same distance.

Weights:

  1. I read it is good to weight the k-nearest neighbors before selecting the winning class. How does that work? I.e. how are the neighbors weighted and how is then the class determined?

Majority vote alternatives:

  1. Are there other rules/strategies to determine the winning class other than majority vote?

Answer

The ideal way to break a tie for a k nearest neighbor in my view would be to decrease k by 1 until you have broken the tie. This will always work regardless of the vote weighting scheme, since a tie is impossible when k = 1. If you were to increase k, pending your weighting scheme and number of categories, you would not be able to guarantee a tie break.

Attribution
Source : Link , Question Author : Fletcher Duran , Answer Author : Nick Stauner

Leave a Comment