Adjusting for Autocorrelated Errors in Neural Networks for Time Series

Abstract

Time series are everywhere and exist in a wide range of domains. Electrical activities of manufacturing equipment, electrocardiograms, traffic occupancy rates, currency exchange rates, speech signals, and atmospheric measurements can all be seen as examples of time series. Modeling time series across different domains is difficult. In many cases, it requires enormous effort and a significant amount of prior knowledge to generate highly accurate models tailored to a particular time series domain. In response, an increasing body of research focuses on training neural networks on time series, such that the neural networks learn to model the time series. A common assumption in training neural networks on time series is that the errors at different time steps are uncorrelated. However, due to the temporality of the data, errors are actually autocorrelated in many cases, making the assumption inaccurate.

In this thesis, we propose to learn the autocorrelation coefficient jointly with the model parameters in order to adjust for autocorrelated errors and thus improve model performances on time series. We first develop our method for time series regression. Then, extensions are made to three other time series tasks: time series forecasting, time series classification, and anomaly detection. Large-scale experiments with various neural network architectures and datasets from the four time series tasks verify the effectiveness of our method. Results show that our method enhances performance across most of these time series modeling tasks.