Single-Model Any-Subgroup Equivariance via SymmetricPositional Encodings

Abstract

The inclusion of symmetries as an inductive bias, known as “equivariance”, often improves generalization on geometric data (e.g. grids, sets, and graphs). However, equivariant architectures are usually highly constrained, designed for pre-chosen symmetries, and cannot be applied to datasets with different symmetries. This work constructs a single model that is simultaneously equivariant to several groups, by simply regulating a certain input feature. Starting with a permutation-equivariant base model respecting the full Sₙ symmetry group, we can obtain subgroup G ⊆ Sₙ equivariance by using a symmetry-breaking input that is G-symmetric. Under mild conditions, the resultant network is only G-equivariant. But finding an input with automorphism group exactly G is computationally hard, which can be overcome by relaxing exact symmetry breaking to approximate symmetry breaking. This is done by leveraging the notion of 2-closure to derive fast algorithms. This method is validated on symmetry selection, multitask, and transfer learning settings, demonstrating that a single network equivariant to multiple permutation subgroups outperforms both separate equivariant models or a single non-equivariant model.