Kernel dependence analysis and graph structure morphing for novelty detection with high-dimensional small size data set

Abstract

In this study, we propose a new approach for novelty detection that uses kernel dependence techniques for characterizing the statistical dependencies of random variables (RV) and use this characterization as a basis for making inference. Considering the statistical dependencies of the RVs in multivariate problems is an important challenge in novelty detection. Ignoring these dependencies, when they are strong, may result in inaccurate inference, usually in the form of high false positive rates. Previously studied methods, such as graphical models or conditional classifiers, mainly use density estimation techniques as their main learning element to characterize the dependencies of the relevant RVs. Therefore, they suffer from the curse of dimensionality which makes them unable to handle high-dimensional problems. The proposed method, however, avoids using density estimation methods, and rather, employs a kernel method, which is robust with respect to dimensionality, to encode the dependencies and hence, it can handle problems with arbitrarily high-dimensional data. Furthermore, the proposed method does not need any prior information about the dependence structure of the RVs; thus, it is applicable to general novelty detection problems with no simplifying assumption. To test the performance of the proposed method, we apply it to realistic application problems for analyzing sensor networks and compare the results to those obtained by peer methods.