Predicting problems caused by component upgrades

Abstract

This thesis presents a new, automatic technique to assess whether replacing a component of a software system by a purportedly compatible component may change the behavior of the system. The technique operates before integrating the new component into the system or running system tests, permitting quicker and cheaper identification of problems. It takes into account the system's use of the component, because a particular component upgrade may be desirable in one context but undesirable in another. No formal specifications are required, permitting detection of problems due either to errors in the component or to errors in the system. Both external and internal behaviors can be compared, enabling detection of problems that are not immediately reflected in the output. The technique generates an operational abstraction for the old component in the context of the system, and one for the new component in the context of its test suite. An operational abstraction is a set of program properties that generalizes over observed run-time behavior. Modeling a system as divided into modules, and taking into account the control and data flow between the modules, we formulate a logical condition to guarantee that the system's behavior is preserved across a component replacement. If automated logical comparison indicates that the new component does not make all the guarantees that the old one did, then the upgrade may affect system behavior and should not be performed without further scrutiny.

(cont.) We describe a practical implementation of the technique, incorporating enhancements to handle non-local state, non-determinism, and missing test suites, and to distinguish old from new incompatibilities. We evaluate the implementation in case studies using real-world systems, including the Linux C library and 48 Unix programs. Our implementation identified real incompatibilities among versions of the C library that affected some of the programs, and it approved the upgrades for other programs that were unaffected by the changes.