An algorithm for testing heavy-tailed distributions

Abstract

Efficiently testing whether distributions have certain properties is a problem important in many different disciplines. We seek to understand how many elements appear infrequently, that is, characterize the tail of the distribution. In particular, we develop an algorithm that determines whether the weight of the tail of a distribution is heavy (such as a Lomax) or exponential. Our algorithm makes use of an equal weight bucketing scheme that divides the support of the distribution according to how much probability mass lies between two points, allowing us to characterize parts of the distribution we may not see many samples from. We analyze the sample complexity of our test statistic.