Modeling manufacturing on-time delivery

Abstract

Historically, Company X has undertaken multiple projects to improve On-time Delivery (OTD) goals but not all of these have resulted in significantly improved performance. As a government contractor, Company X is evaluated on OTD as a contractual performance metric. This project established the foundations for a real-time predictive and diagnostic machine learning model for OTD. A real-time machine learning model could correct negative impacts to the manufacturing system before they impact the customer. It would also provide an understanding of how corrective measures are expected to affect outcomes. To begin, candidate variables correlating to OTD were considered and the relevant datasets were gathered. These spanned the enterprise, including Bill of Materials (BOM), material resource planning, supply chain, production, quality, and test datasets.

Each dataset was evaluated to assess data quality, using a framework which considers accuracy, reliability, timeliness, completeness, comprehensiveness, accessibility, and availability. Datasets were also examined for missing data, with recommendations for new data collection that may improve future iterations of predictive models. The datasets were integrated to reflect the time-phased dependencies of the hierarchical BOMs. This dataset structure best represents the manufacturing reality. The integrated dataset was aggregated on the basis of BOM indenture-levels, due to data sparsity. To maximize understanding and interpretability of this proof-of-concept model, the machine learning methods considered were limited to decision trees. Given a total training and testing population of 207 deliveries, the model achieved an accuracy (F1 Score) of 86% and RMSE of 39%.

While preliminary modeling shows promise for future models, a number of issues need to be addressed: expansion of training and testing datasets, improvements in data quality, gathering missing data, and implementing IT systems better suited for accessing large datasets. Once improvements can be made in these areas, a true real-time predictive model for OTD may be a possible solution for Company X.