Fast Polyhedral Adaptive Conjoint Estimation

Abstract

We propose and test new "polyhedral" question design and estimation methods that use recent developments in mathematical programming. The methods are designed to offer accurate estimates after relatively few questions in problems involving many parameters. With polyhedral question design, each respondent's questions are adapted based upon prior answers by that respondent to reduce a feasible set of parameters as rapidly as possible. Polyhedral estimation provides estimates based on a centrality criterion (the "analytic center" of the feasible parameter set). The methods require computer support but can operate in both Internet and other computer-aided environments with no noticeable delay between questions.

We evaluate the proposed methods using two approaches. First, we use Monte Carlo simulations to compare the methods against established benchmarks in a variety of domains. In the simulations we compare polyhedral question design to three benchmarks: random selection, efficient Fixed designs, and Adaptive Conjoint Analysis (ACA). We compare polyhedral estimation to Hierarchical Bayes estimation for each question design method. The simulations evaluate the methods across different levels of respondent heterogeneity, response accuracy, and numbers of questions. For low numbers of questions, polyhedral question design does best (or is tied for best) for all domains. For high numbers of questions, efficient Fixed designs do better in some domains. The best estimation method depends on respondent heterogeneity and response accuracy. Polyhedral (analytic center) estimation shows particular promise for high heterogeneity and/or for low response errors.

The second evaluation employs a large-scale field test. The field test involved 330 respondents, who were randomly assigned to a question-design method and asked to complete a web-based conjoint exercise. Following the conjoint exercise, respondents were given $100 and allowed to make a purchase from a Pareto choice set of five new-to-the-market laptop computer bags. The respondents received their chosen bag together with the difference in cash between the price of their chosen bag and the $100. We compare the question-design and estimation methods on both internal validity (holdout tasks) and external validity (actual choice of a laptop bag). The field test findings are consistent with the simulation results and offer strong support for the polyhedral question design method. The preferred estimation method varied based on the question design method, although Hierarchical Bayes estimation consistently per-formed well in this domain.

The findings reveal a remarkable level of consistency across the validation tasks. They suggest that the proposed methods are sufficiently promising to justify further development. At the time of the test, the bags were prototypes. Based, in part, on the results of this study the bags were launched successfully and are now commercially available. Sales of the features of the laptop bags were consistent with conjoint-analysis predictions.