Margin-based Ranking and an Equivalence between AdaBoost and RankBoost
Author(s)Rudin, Cynthia; Schapire, Robert E.
MetadataShow full item record
We study boosting algorithms for learning to rank. We give a general margin-based bound for ranking based on covering numbers for the hypothesis space. Our bound suggests that algorithms that maximize the ranking margin will generalize well. We then describe a new algorithm, smooth margin ranking, that precisely converges to a maximum ranking-margin solution. The algorithm is a modification of RankBoost, analogous to “approximate coordinate ascent boosting.” Finally, we prove that AdaBoost and RankBoost are equally good for the problems of bipartite ranking and classification in terms of their asymptotic behavior on the training set. Under natural conditions, AdaBoost achieves an area under the ROC curve that is equally as good as RankBoost’s; furthermore, RankBoost, when given a specific intercept, achieves a misclassification error that is as good as AdaBoost’s. This may help to explain the empirical observations made by Cortes andMohri, and Caruana and Niculescu-Mizil, about the excellent performance of AdaBoost as a bipartite ranking algorithm, as measured by the area under the ROC curve.
DepartmentSloan School of Management
Journal of Machine Learning Research
Rudin, Cynthia, and Robert E. Schapire. “Margin-based Ranking and an Equivalence between AdaBoost and RankBoost.” Journal of Machine Learning Research 10 (2009): 2193-2232.
Final published version
area under the ROC curve, AdaBoost, generalization bounds, RankBoost, ranking