HyperSketch : Language for Implementing Generic Neuro-Symbolic Program Synthesizers

Abstract

Recent developments in neuro-symbolic learning, including program synthesis and deep learning have surprised in the rate of growth of scale, scope, and variety of models used and problems solved. However, existing frameworks are usually specialized for one style of model learning / synthesis and when integrating with other representational modalities requires substantial engineering effort.

In this paper we propose a sketch-based meta-programming framework for developing generic synthesis algorithms and integrating different learning modalities (particularly constraint solving and gradient descent) over a common intermediate representation.

We introduce HyperSketch, a high-level language that empowers developers in programming model-agnostic neuro-symbolic learning and synthesis algorithms. HyperSketch does this by providing the developer compact access to the internal representation and solvers of the Sketch (Solar-Lezama et al.) program synthesis system, allowing the user to manipulate sketches during runtime. With few primitives, we give the developer ability to encode prior knowledge about the order of concretization of unknown model structure, and offering the developer to choose which solvers (and hyperparameters) to use for solving different parts of the sketch. The core innovation of our language is a family of primitives for runtime manipulation of sketches that can perform the following tasks: (1) call Sketch’s solver on a particular sketch, (2) concretize a sketch function, (3) clone a sketch function, (4) rewire the call-graph of the sketch, (5) manipulate input files, and (6) evaluate sketches. We expose these primitives as part of an imperative dynamically typed standard language base with Python/C++-style syntax.

We use HyperSketch to implement several synthesis strategies with varying complexity for several problem domains: Boolean function synthesis, synthesis of Karel programs, and synthesis of timeseries programs. We particularly explore best-effort synthesis and synthesis-through-unification (STUN) in tutorial-style guide. We also demonstrate a case study of using HyperSketch to solve a problem in an industry research setting on a task of synthesizing predicates over labeled graphs. We demonstrate that using HyperSketch for implementing synthesizers can reduce developer effort by 3x-10x compared to previous approaches.