How we can draw an ROC curve for decision trees?

Normally we cannot draw an ROC curve for the discrete classifiers like decision trees.
Am I right?
Is there any way to draw an ROC curve for Dtrees?

Answer

If your classifier produces only factor outcomes (only labels), without scores, you still can draw a ROC curve. However this ROC curve is only a point. Considering the ROC space, this points is (x,y)=(FPR,TPR), where FPR – false positive rate and TPR – true positive rate.

See more on how this is computed on Wikipedia page.

You can extend this point to look like a ROC curve by drawing a line from (0,0) to your point, and from there to (1,1). Thus you have a curve.

However, for a decision tree is easy to extend from an label output to a numeric output. Note that when you predict with a decision tree you go down from the root node to a leaf node, where you predict with majority class. If instead of that class you would return the proportion of classes in that leaf node, you would have a score for each class. Suppose that you have two classes T and F, and in your leaf node you have 10 instances with T and 5 instances with F, you can return a vector of scores: (scoreT,scoreF)=(countTcountT+countF,countFcountT+countF)=(10/15,5/15)=(0.66,0.33). Take care that this is really note a proper scoring rule (this are not the best estimator for probabilities), but is better than nothing I believe, and this is how usually scores are retrieved for decision trees.

Attribution
Source : Link , Question Author : DataMiner , Answer Author : Alexey Grigorev

Leave a Comment