Transforming Unstructured Data into Actionable Insights: A Use Case of Generative AI in Operational Technology Problem Management

Abstract

The increasing availability and reduced cost of Generative AI applications for the general public have motivated organizations across all industries to implement AI-based solutions in their daily operations. Still, they struggle to determine the capabilities and limitations of this technology when implementing it in their specific context. This thesis addresses these challenges through a practical case study: deploying a text-based Generative AI system (using Large Language Models - LLMs) for automated downtime event characterization within a global industrial operational technology (OT) setting by transforming unstructured problem management reports into structured, actionable business insights. The developed software system contains a data pre-processing stage, followed by four LLM-based tasks (LLM-extraction, LLM-autoclassification, multi-aspect multi-level LLM-classification, and LLM-accuracy). We wrap everything in a well-structured and easy-to-understand evaluation framework that ensures the system’s output is format-reliable, accurate, and consistent. Through simple prompt engineering techniques and continuous failure modes analysis, we achieve high accuracy (>89%) and consistency (>79%) for downtime events characterization at 1% of the current cost. In the end, we prove that it is possible to implement an AI-based solution within current operational processes while properly communicating its capabilities and limitations and adapting its usage to the most added value purpose.