Efficient cloud photo enhancement using compact transform recipes

Abstract

Cloud image processing is often proposed as a solution to the limited computing power and battery life of mobile devices: it allows complex algorithms to run on powerful servers with virtually unlimited energy supply. Unfortunately, this idyllic picture overlooks the cost of uploading the input and downloading the output images, which can be significantly higher, in both time and energy, than that of local computation. For most practical scenarios, the compression ratios required to make cloud image processing energy and time efficient would degrade an image so severely as to make the result unacceptable. Our work focuses on image enhancements that preserve the overall content of an image. Our key insight is that, in this case, the server can compute and transmit a description of the transformation from input to output, which we call a transform recipe. At equivalent quality, recipes are much more compact than jpeg images: this reduces the data to download. Furthermore, we can fit the parameters of a recipe from a highly-degraded input-output pair, which decreases the data to upload. The client reconstructs an approximation of the true output by applying the recipe to its local high quality input. We show on a dataset of 168 images spanning 10 applications, that recipes can handle a diverse set of image enhancements and, for the same amount of transmitted data, provide much higher fidelity compared to jpeg-compressing both the input and output. We further demonstrate their usefulness in a practical proof of concept system with which