Study on Text Sentiment Classification Method Based on TRANSFORMER

It is widely used in some real scenes, such as film box office prediction, and has attracted much attention all the time. In order to explore the performance and defects of the current mainstream deep learning methods in text emotion classification tasks, this paper compares and evaluates several mainstream methods based on transformer, including Bert and its improved models: Roberta, distilbert and minilm.

After the experiment on IMDB film review emotion classification task, it is found that the current multilingual pre training method will reduce the classification performance of Bert model; Different from the visual model and the language model simplified by distillation, its classification ability will decline slightly; Roberta's training method is excellent and worthy of in-depth study. This paper provides a further improvement direction for Bert emotion classification.