Objective comparison of design of experiments strategies in design and observations in practice

Abstract

Design of Experiments (DoE) strategies in robust engineering determine which prototypes and how many of each are created and tested. A better strategy is one that delivers a closer-to-optimal performance at a lower experimental cost. Prototype testers who may use statistical DoE, design-build-test, or one-at-a-time methods in a wide variety of industries were sought out and interviewed to examine the strategies used in practice and how they fit into a proposed five-layer process support model. From these interviews, we see that DoE are competently and widely practiced. Some improvements to the state of the practice may include: contracts to specify and reward quality engineering among suppliers to complex product systems and wider use in light of new computing power of system level mathematical models for experimentation on complex systems. This thesis also examines the relative value of strategies in a particular response surface using a software-based comparator. The data is modified to simulate data environments with other levels of repeatability and interactions, and the way that these variables effect the performance of strategies is examined. The concept of an optimal design of experiments strategy is developed by abstracting the characteristics of a generic strategy and letting it develop in a genetic algorithm in that comparator. The framework for the evaluation of DoE strategies is one significant output to come out of this work that may be of use in future research. Further, the particular abstraction chosen for DoE strategies is offered to other researchers as an exemplar of a particular perspective, to help engender dialogue about methods for optimizing prototype testing policy.