20 July 2024
Volume 59 Issue 7
Review
Research on self-supervised pre-training for recommender systems
YANG Jiyuan, MA Muyang, REN Pengjie, CHEN Zhumin, REN Zhaochun, XIN Xin, CAI Fei, MA Jun
2024, 59(7):  1-26.  doi:10.6040/j.issn.1671-9352.1.2023.043
Abstract ( 45 )   PDF (7266KB) ( 21 )   Save
References | Related Articles | Metrics
Plenty of recent studies explores the application of pre-training techniques within the context of recommendation scenarios and the design of pre-training tasks in order to enhance the overall performance of recommendation. This paper extensively reviews the progress in research of recommendation models based on pre-training, classifies and compares different pre-training methods, and conducts extensive experiments and analyses on some representative models using three benchmark datasets for recommendation systems. The datasets and codes have been made open source. Finally, we summarize and prospect the future development trend of recommendation models based on pre-training.
Dimensionality reduction and retrieval algorithms for high dimensional data
SHAO Wei, ZHU Gaoyu, YU Lei, GUO Jiafeng
2024, 59(7):  27-43.  doi:10.6040/j.issn.1671-9352.1.2023.062
Abstract ( 32 )   PDF (1007KB) ( 17 )   Save
References | Related Articles | Metrics
At present, most studies use some dimensionality reduction methods to convert high-dimensional vectors into low-dimensional vector representations, and then apply related vector retrieval optimization technology to achieve fast similarity retrieval, thereby improving the application performance of large models. Currently, there are many and scattered dimensionality reduction methods for high-dimensional data, and the dimensionality reduction methods used in different research backgrounds are different. Similarly, there are also many different retrieval ideas and optimization methods in vector retrieval technology. By reviewing the main ideas and optimization methods of recent dimensionality reduction and retrieval algorithms, this paper helps to generate inspiring connections between the two and support the development and in-depth research of subsequent high-dimensional vector retrieval optimization algorithms.
Matrix product operator based sequential recommendation model
LIU Peiyu, YAO Bowen, GAO Zefeng, ZHAO Wayne Xin
2024, 59(7):  44-52.  doi:10.6040/j.issn.1671-9352.1.2023.042
Abstract ( 25 )   PDF (1254KB) ( 6 )   Save
References | Related Articles | Metrics
The task of sequential recommendation confronts challenges characterized by high complexity and substantial diversity. The paradigm of pre-training and fine-tuning is extensively employed for learning item representations based on sequential data in recommendation scenarios. However, prevalent approaches tend to disregard the potential underfitting and overfitting issues that may arise during model fine-tuning in new domains. To address this concern, a novel neural network architecture grounded in the framework of matrix product operator(MPO)is introduced, and two versatile fine-tuning strategies are presented. Firstly, a lightweight fine-tuning approach that involves updating only a subset of parameters is proposed to effectively mitigate the problem of overfitting during the fine-tuning process. Secondly, an over-parameterization fine-tuning strategy is introduced by augmenting the number of trainable parameters, robustly addressing the issue of underfitting during fine-tuning. Through extensive experimentation on well-established open-source datasets, the efficacy of the proposed approach is demonstrated by achieving performance achievements. This serves as a compelling testament to the effectiveness of the proposed approach in addressing the challenge of general item representation in recommendation systems.
A document-level event extraction method based on core arguments
SUN Chengjie, LI Zongwei, SHAN Lili, LIN Lei
2024, 59(7):  53-63.  doi:10.6040/j.issn.1671-9352.1.2023.080
Abstract ( 20 )   PDF (3665KB) ( 9 )   Save
References | Related Articles | Metrics
A document-level event extraction method based on core arguments(CA-DocEE)is proposed, which defines criteria for selecting core arguments based on their distributions in document-level events, uses heterogeneous graph convolutional neural networks to augment document contextual information for encoding argument entities, and captures deep semantic information in sentences based on machine reading comprehension methods for classifying the role of arguments. On the document-level event extraction dataset, the method proposed in this paper achieves a micro-average F1 value of 80.1%, which is comparable with the state-of-the-art methods.
Noise network alignment method integrating multiple features
XIAN Ning, FAN Yixing, LIAN Tao, GUO Jiafeng
2024, 59(7):  64-75.  doi:10.6040/j.issn.1671-9352.1.2023.102
Abstract ( 22 )   PDF (1975KB) ( 4 )   Save
References | Related Articles | Metrics
A multi-round iterative network alignment method is proposed to address the challenges of large structural differences and high noise sensitivity in anchor nodes in network alignment tasks. The method calculates node features of different dimensions using various heuristic approaches at each iteration, utilizing the combination of multiple features to assess the reliability of anchor nodes, filter potential noise, and enhance the confidence of each alignment round. Additionally, a graph neural network is employed to improve the consistency between nodes without attributes, mitigating the impact of structural differences in networks. Experimental results demonstrate that this method achieves high accuracy under high noise conditions, verifying its effectiveness.
Factual error detection in knowledge graphs based on dynamic neighbor selection
GUI Liang, XU Yao, HE Shizhu, ZHANG Yuanzhe, LIU Kang, ZHAO Jun
2024, 59(7):  76-84.  doi:10.6040/j.issn.1671-9352.1.2023.097
Abstract ( 27 )   PDF (1844KB) ( 30 )   Save
References | Related Articles | Metrics
The construction and updating of the knowledge graph(KG)usually depend on a wide range of web data and automated methods, inevitably resulting in factual inaccuracies in the modeled and acquired knowledge. To tackle this problem, a novelapproach for identifying factual inaccuracies within the knowledge graph is proposed. This method actively selects adjacent nodes of the facts to be checked, detecting errors by measuring the intricate associations linking the head and tail entities. More specifically, it first utilizes graph structure information to identify potential neighbors for each entity. Then, based on contextual information, it dynamically selects relevant neighbors and uses an efficient graph attention network to encode node features. Finally, by calculating the consistency of head and tail entity representations, it determines if the fact under consideration is erroneous. Experimental results on multiple public KG datasets demonstrate that this method outperforms existing approaches in error detection.
Category-wise knowledge probers for representation learning of graph neural networks
HUANG Xingyu, ZHAO Mingyu, LYU Ziyu
2024, 59(7):  85-94.  doi:10.6040/j.issn.1671-9352.1.2023.064
Abstract ( 19 )   PDF (3615KB) ( 4 )   Save
References | Related Articles | Metrics
In order to solve the problem that the graph neural network model lacks corresponding probes, a knowledge detection framework for graph neural network representation learning is proposed, and two kinds of class-aware knowledge probes are designed based on the category attributes of data in different domains, namely clustering probes and contrastive clustering probes. The two probe the characterization effect of different models and give corresponding scores. On 8 datasets in 3 neighborhoods, including reference networks, social networks and biological networks, the representation learning of 7 classical graph neural network models realizes systematic knowledge detection and evaluation experiments, and summarizes the detection and evaluation conclusions.
Probability distribution optimization model for learning to rank
ZHAO Fengxu, WANG Jian, LIN Yuan, LIN Hongfei
2024, 59(7):  95-104.  doi:10.6040/j.issn.1671-9352.1.2023.026
Abstract ( 22 )   PDF (1775KB) ( 3 )   Save
References | Related Articles | Metrics
Existing learning to rank(LTR)models rely on the scores output by models to represent the partial order among documents. Considering the limitation of treating scores as deterministic values, this paper proposes a probability distribution optimization method for the LTR model, which introduces the uncertainty of the ranking score. It smooths the scores in the form of probability distributions, thereby transforming the comparison of ranking scores into the probability estimation of score partial orders. The proposed method is applied to LTR models such as RankNet, LambdaRank, and LambdaMART. It effectively bridges the gap between the modeled probability and the target probability, leading to optimization of the LTR models. The paper conducts experiments on multiple large-scale real datasets, and the experimental results show that the optimized models outperform the original ones, which validates the effectiveness of the proposed method.
Multimodal conversation emotion recognition based on clustering and group normalization
LUO Qi, GOU Gang
2024, 59(7):  105-112.  doi:10.6040/j.issn.1671-9352.1.2023.055
Abstract ( 31 )   PDF (3409KB) ( 3 )   Save
References | Related Articles | Metrics
It is a challenge for multimodal emotion recognition task that the confusion of similar emotion categories recognition leads to a decrease in recognition effect. To address this problem, a neural network modeling approach for relational graphs is proposed based on clustering group normalization. Firstly, three modal features are extracted using three different feature extractors and spliced by incorporating speaker encoding, which enriches the feature representation and preserves the original information. Secondly, contextual information is extracted using Transformer. Finally, after the feature nodes are input into the relational graph convolutional neural network, the nodes are clustered and grouped by clustering and independently normalized to make similar nodes more similar,which alleviates the problem that similar emotions are difficult to delimit. Through experimental validation, the network model can reach an 86.34% F1-score on the IEMOCAP dataset four classification, which verifies the effectiveness of the method in this paper. At present, the model achieves the best performance on this dataset.
A prompt learning approach for telecom network fraud case classification
JI Jie, SUN Chengjie, SHAN Lili, SHANG Boyue, LIN Lei
2024, 59(7):  113-121.  doi:10.6040/j.issn.1671-9352.1.2023.040
Abstract ( 24 )   PDF (3845KB) ( 8 )   Save
References | Related Articles | Metrics
For the automatic classification technology of telecom fraud cases, a classification system of telecom network fraud based on situational analysis is formulated, the privacy protection method of case text de-identification is realized, and accuracy and F1-score of a classification method of telecom network fraud cases based on prompt learning is proposed. The experimental results show that the method is on average 1 to 2 percentage points higher than the BERT-based classification method on the data set constructed in the paper.
Chinese disease text classification model driven by medical knowledge
LI Chao, LIAO Wei
2024, 59(7):  122-130.  doi:10.6040/j.issn.1671-9352.0.2023.291
Abstract ( 22 )   PDF (3349KB) ( 8 )   Save
Related Articles | Metrics
This study proposes a Chinese disease text classification model that integrates knowledge graph. Firstly, by introducing structured knowledge from external medical knowledge graph, a knowledge enhanced disease text vector representation is obtained; Secondly, the global semantic features and local semantic features of the disease text are extracted by using bidirectional long short-term memory network and convolutional neural network respectively. At the same time, the joint attention mechanism improves the efficiency of the model in extracting effective features information; Finally, the extracted features are concatenated and fused, and a classifier is used to output the classification result. The experimental results on the Chinese disease text dataset show that the proposed model has a classification accuracy, recall, and the harmonic mean value F1 of 95.21%, 95.64%, and 95.42%, respectively, which shows better classification performance compared to other models.