Self-Supervised Self-Supervision by Combining Deep Learning and Probabilistic Logic

Labeling training examples at scale is a perennial challenge in machine learning. Self-supervision methods compensate for the lack of direct supervision by leveraging prior knowledge to automatically generate noisy labeled examples. Deep probabilistic logic (DPL) unifying framework self-supervised learning that represents unknown labels as latent variables and incorporates diverse self-supervision using train deep neural network end-to-end variational EM. While DPL successful combining pre-specified self-supervision, manually crafting attain high accuracy may still be tedious challenging. In this paper, we propose Self-Supervised Self-Supervision (S4), which adds capability learn new automatically. Starting from an initial "seed," S4 iteratively uses self-supervision. These are either added directly (a form structured self-training) or verified human expert (as feature-based active learning). Experiments show able accurate can often nearly match supervised with tiny fraction effort.