Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classification

arXiv:2405.11524v1 Announce Type: new
Abstract: Text classification is a crucial and fundamental task in natural language processing. Compared with the previous learning paradigm of pre-training and fine-tuning by cross entropy loss, the recently proposed supervised contrastive learning approach has received tremendous attention due to its powerful feature learning capability and robustness. Although several studies have incorporated this technique for text classification, some limitations remain. First, many text datasets are imbalanced, and the learning mechanism of supervised contrastive learning is sensitive …

abstract arxiv attention classification cs.cl entropy feature fine-tuning fundamental language language processing loss natural natural language natural language processing paradigm pre-training processing sampling simple text text classification training type