Statistical Verification using Surrogate Models and Conformal Inference and a Comparison with Risk-aware Verification
Author(s)
Qin, Xin
Xia, Yuan
Zutshi, Aditya
Fan, Chuchu
Deshmukh, Jyotirmoy
Journal
ACM Transactions on Cyber-Physical Systems
Publisher
ACM
Citation
Qin, Xin, Xia, Yuan, Zutshi, Aditya, Fan, Chuchu and Deshmukh, Jyotirmoy. "Statistical Verification using Surrogate Models and Conformal Inference and a Comparison with Risk-aware Verification." ACM Transactions on Cyber-Physical Systems.
Version
Final published version
Abstract
Uncertainty in safety-critical cyber-physical systems can be modeled using a finite number of parameters or parameterized input signals. Given a system specification in Signal Temporal Logic (STL), we would like to verify that for all (infinite) values of the model parameters/input signals, the system satisfies its specification. Unfortunately, this problem is undecidable in general. Statistical model checking (SMC) offers a solution by providing guarantees on the correctness of CPS models by statistically reasoning on model simulations. We propose a new approach for statistical verification of CPS models for user-provided distribution on the model parameters. Our technique uses model simulations to learn surrogate models, and uses conformal inference to provide probabilistic guarantees on the satisfaction of a given STL property. Additionally, we can provide prediction intervals containing the quantitative satisfaction values of the given STL property for any user-specified confidence level. We compare this prediction interval with the interval we get using risk estimation procedures. We also propose a refinement procedure based on Gaussian Process (GP)-based surrogate models for obtaining fine-grained probabilistic guarantees over sub-regions in the parameter space. This in turn enables the CPS designer to choose assured validity domains in the parameter space for safety-critical applications. Finally, we demonstrate the efficacy of our technique on several CPS models.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3635160
