Minimax Optimal Universal Codeword Sets

Abstract

In an interactive multi-user data-processing system a user knows the probabilities of his messages and must encode them into a fixed system-wide variable-length codeword set. He needs to receive the answer to his last message before selecting the next, so his encoding is one-shot. To minimize average codeword length he encodes his messages in order of decreasing probability into codewords in order of increasing length. I give an algorithm which, for each of several measures of performance, finds the codeword set best by that measure for the worst user, and some of the minimax optimal codeword sets the algorithm has found. Some of the results hold for all user distributions: others require e.g. that all users send exactly or at most m distinct messages, and/or that there is an integer k such that no user has a message of probability greater than 1/k.