Learning Hierarchical Interactions at Scale: A Convex Optimization Approach
Name
hazimeh20a.pdf
Description
Published version
Size
489.64 KB
Format
Adobe PDF
Checksum (MD5)
4072b60718a5633f3c4c1c5b6e9b469e
Author(s)
Hazimeh, Hussein
Mazumder, Rahul
Date Issued
June 2020
Journal
Proceedings of Machine Learning Research
Publisher
International Machine Learning Society
Citation
Hazimeh, Hussein and Rahul Mazumder. “Learning Hierarchical Interactions at Scale: A Convex Optimization Approach.” Paper in the Proceedings of Machine Learning Research, 108, 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) 2020, Palermo,Italy, June 3-5 2020, International Machine Learning Society © 2020 The Author(s)
Version
Final published version
Abstract
In many learning settings, it is beneficial toaugment the main features with pairwise in-teractions. Such interaction models can beoften enhanced by performing variable selec-tion under the so-calledstrong hierarchycon-straint: an interaction is non-zero only if itsassociated main features are non-zero. Ex-isting convex optimization-based algorithmsface difficulties in handling problems wherethe number of main featuresp∼103(withtotal number of features∼p2). In this pa-per, we study a convex relaxation which en-forces strong hierarchy and develop a highlyscalable algorithm based on proximal gradi-ent descent. We introduce novel screeningrules that allow for solving the complicatedproximal problem in parallel. In addition,we introduce a specialized active-set strategywith gradient screening for avoiding costlygradient computations. The framework can handle problems having dense design matri-ces, withp= 50,000 (∼109interactions)—instances that are much larger than state ofthe art. Experiments on real and syntheticdata suggest that our toolkithierScaleout-performs the state of the art in terms of pre-diction and variable selection and can achieveover a 4900x speed-up.
MIT Department
Sloan School of Management
Massachusetts Institute of Technology. Operations Research Center
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://proceedings.mlr.press/v108/hazimeh20a.html
