Self-supervised Short-text Modeling through Auxiliary Context Generation

Short text is ambiguous and often relies predominantly on the domain context at hand in order to attain semantic relevance. Existing classification models perform poorly short due data sparsity inadequate context. Auxiliary context, which can provide sufficient background regarding domain, typically available several application scenarios. While some of existing works aim leverage real-world knowledge enhance short-text representations, they fail place appropriate emphasis auxiliary Such do not harness full potential sources. To address this challenge, we reformulate as a dual channel self-supervised learning problem (that leverages context) with generation network corresponding prediction model. We propose framework, Pseudo-Auxiliary Context for Short-text Modeling (PACS) , comprehensively it jointly learned an end-to-end manner. Our PACS model consists two sub-networks: Generation Network (CGN) that context’s distribution Prediction (PN) map features final class label. experimental results diverse datasets demonstrate outperforms formidable state-of-the-art baselines. also performance our cold-start scenarios (where contextual information non-existent) during prediction. Furthermore, interpretability ablation studies analyze various representational captured by individual contribution its modules overall PACS, respectively.