Inferring program sequences for texture generation with learned abstractions

Abstract

Abstract interpretation of computer programs is an active area of research in program synthesis [11] that involves learning the behavior and characterization of programs given a sequence of inputs/ output examples for the program. One aims to learn as much about a program as possible with very little information on the inputs and outputs the program generates. Abstraction helps in understanding the semantics, which is the mathematical characterization of the behavior of a computer program. This project aims to model a composite set of encoder-decoder neural networks that can learn a sequence of transformations which convert a specfic program input to an output, and from that infer which sequence of transitions one can use to generate an output from a given input. A specfic use case we explore is to take an image or a specfic property of the image .eg. texture and apply a sequence of image transformations to produce an output image/texture, can one learn the sequence of transformations that converts the input image to the output image given only the input/output image pair? In the image domain of this abstract interpretation problem, we want to be able to learn the correct sequence that will transform an input image to its corresponding output image and from that infer which image transformations can be generated by training an encoded image through dierent blocks that correspond to dierent transformations. We expect that an image produced from a given sequence of transformations also has a high probability of being found in the set of images generated from the encoder-decoder network where each block is a neural network trained for a specfic image transformation on a set of images.