On Physics-Inspired Generative Models

Abstract

Physics-inspired generative models such as diffusion models constitute a powerful family of generative models. The advantages of models in this family come from relatively stable training process and high capacity. A number of possible improvements remain possible. In the thesis, we will first delve into the improved techniques for training and sampling in diffusion models. The training objectives of diffusion models exhibit high variance when the data distribution is multi-modal. To mitigate this, we propose a training objective that generalizes conventional denoising score-matching and significantly reduces variance in training targets. Alternatively, we introduce a training framework that integrates learnable discrete latents into continuous diffusion models. These latents simplify the learning of diffusion models’ complex noise-to-data mapping. On the other hand, the sampling process of diffusion models generally involves solving differential equations. To expedite the sampling process, we propose a new sampling algorithm that combines the best of previous ODE and SDE samplers, greatly boosting the performance of pre-trained diffusion models. Additionally, our research explores methods to promote diversity in finite samples by introducing mutual repulsion forces in the generative process. In the realm of physics-inspired generative models, many physical processes could be used to develop generative models. We will introduce a new family of generative models arising from electrostatic theory, termed Poisson Flow Generative Models (PFGM). PFGM rivals leading diffusion models while showcasing improved sampling robustness. The extended version, PFGM++, places diffusion models and PFGM under the same framework and introduces new, better models. We will further present a principled approach to convert physical processes into generative models.