Abstract: To accurately detect and identify hate speech, the dataset samples are expanded and balanced by fine-tuning the large language model. The RoBERTa-Attention-GRU-TextCNN model is constructed based on the pre-training model RoBERTa, leveraging the powerful feature capture and extraction capabilities of deep learning for the analysis and mining of text sequence data. Firstly, the RoBERTa model is used to extract features from the text data; then, the self-attention mechanism is used to obtain the dependencies between words; finally, the acquired feature matrix is input into the GRU-TextCNN layer to capture deeper semantic information and local features. Two publicly available datasets provided by TweetEval are used to evaluate the model effect, and the experimental results show that the model has a better detection effect compared to the traditional hate speech detection model.

Key words: large language model, hate detection, RoBERTa, pre-trained model, RoBERTa-Attention-GRU-TextCNN