Non-asymptotic Behavior in Massive Multiple Access and Streaming System Identification

Abstract

Non-asymptotic understanding of information theoretic and algorithmic limits of estimation in statistical problems is indispensable for practical applications in engineering. There are two broad approaches to this end. The first: tools and advances in modern probability theory aid in deriving fully non-asymptotic bounds for such problems. This approach, while useful, is sometimes insufficient as the bounds it yields usually have sub-optimal constants and unavoidable logarithmic factors. Consequently, in some situations where the primary goal is to obtain sharp characterizations, e.g. error exponents, it can be highly non-trivial to derive fully non-asymptotic results that serve this purpose, particularly in high dimensional problems of recent times. In aforesaid circumstances there is a second approach: recourse to asymptotics that can serve as a reasonable substitute for finite length behavior. In this thesis, we employ both these approaches. First we provide high dimensional asymptotic bounds for massive multiple access which is an important consideration in upcoming wireless networks. Then we turn towards streaming system identification where we develop a novel algorithm provide tight non-asymptotic bounds showing the optimality of our method.

Massive multiple access is an important problem in current and upcoming wireless networks. Also known as massive machine type communication (mMTC) in 5G, it envisions a scenario of a large number of transmitters (usually small sensors in IoT for instance) with small payloads communicating sporadically with a base station. Information theoretic understanding of such a problem is of paramount importance for evaluating existing multiple access schemes and developing new strategies that handle such drastic interference. To this end, many-user multiple access channel (MAC) is a crucial model that captures the new effects in massive multiple access. Previous works have focused on the additive white Gaussian noise (AWGN) many-user MAC. In this thesis, we aim to understand the fundamental limits of energy efficiency in the quasi-static Rayleigh fading many-user MAC. In particular, we provide tight achievability and converse bounds on the minimum energy-per-bit required to support a certain user density, fixed payload and target per-user error (in the limit as blocklength grows to infinity). Although asymptotic in nature, the results are expected to serve as a good proxy for true finite length behavior. We confirm the presence of the promising almost perfect multi-user interference cancellation, first observed in the AWGN setting, in the quasi-static case. Further we also provide a new achievability bound for the AWGN many-user MAC.

Next we turn towards problem of streaming or online system identification with the goal of designing optimal algorithms and providing non-asymptotic rates on the convergence. In particular, we consider a class of linear and generalized linear (nonlinear) parametric discrete time dynamical systems. Observing a single trajectory from such a system, the aim is recover the system parameters in a streaming fashion. Our work shows that one-pass forward stochastic gradient descent (SGD) algorithm where samples are read in order is sub-optimal compared to the offline ordinary least squares (OLS) estimator. More importantly, based on the observation that reading samples in reverse order mitigates the effect of temporal dependencies, we develop a novel algorithm called SGD with reverse experience replay (SGD-RER) and derive fully non-asymptotic bounds that show it to be near minimax optimal for both stable linear and generalized linear models. Furthermore, we consider a Quasi-Newton style offline algorithm for the generalized linear setting and show that is near optimal even when the process is unstable.