Tight mixed-integer optimization formulations for prescriptive trees
Name
10994_2025_Article_6771.pdf
Size
3.83 MB
Format
Adobe PDF
Checksum (MD5)
5694bb661ce52618b4f9c0566b4f854a
Author(s)
Biggs, Max
Perakis, Georgia
Date Issued
May 29, 2025
Journal
Machine Learning
Publisher
Springer US
Citation
Biggs, M., Perakis, G. Tight mixed-integer optimization formulations for prescriptive trees. Mach Learn 114, 156 (2025).
Version
Final published version
Abstract
We focus on modeling the relationship between an input feature vector and the predicted outcome of a trained decision tree using mixed-integer optimization. This can be used in many practical applications where a decision tree or a tree ensemble is incorporated into an optimization problem to model the predicted outcomes of a decision. We propose novel tight mixed-integer optimization formulations for this problem. Existing formulations can be shown to have linear relaxations that have fractional extreme points, even for the simple case of modeling a single decision tree or a very large number of constraints, which leads to slow solve times in practice. A formulation we propose, based on a projected union of polyhedra approach, is ideal (i.e., the extreme points of the linear relaxation are integer when required) for a single decision tree. Although the formulation is generally not ideal for tree ensembles, it generally has fewer extreme points, leading to a faster time to solve. We also study formulations with a binary representation of the feature vector and present multiple approaches to tighten existing formulations. We show that fractional extreme points are removed when multiple splits are on the same feature. At an extreme, we prove that this results in an ideal formulation for a tree ensemble modeling a one-dimensional feature vector. Building on this result, we also show that these additional constraints result in significantly tighter linear relaxations when the feature vector is low dimensional.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10994-025-06771-8
