Loading...

Table of Content

      
    20 July 2024
    Volume 59 Issue 7
    Review
    Research on self-supervised pre-training for recommender systems
    Jiyuan YANG,Muyang MA,Pengjie REN,Zhumin CHEN,Zhaochun REN,Xin XIN,Fei CAI,Jun MA
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  1-26.  doi:10.6040/j.issn.1671-9352.1.2023.043
    Abstract ( 1311 )   HTML ( 7 )   PDF (7266KB) ( 5130 )   Save
    Figures and Tables | References | Related Articles | Metrics

    Plenty of recent studies explores the application of pre-training techniques within the context of recommendation scenarios and the design of pre-training tasks in order to enhance the overall performance of recommendation. This paper extensively reviews the progress in research of recommendation models based on pre-training, classifies and compares different pre-training methods, and conducts extensive experiments and analyses on some representative models using three benchmark datasets for recommendation systems. The datasets and codes have been made open source. Finally, we summarize and prospect the future development trend of recommendation models based on pre-training.

    Dimensionality reduction and retrieval algorithms for high dimensional data
    Wei SHAO,Gaoyu ZHU,Lei YU,Jiafeng GUO
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  27-43.  doi:10.6040/j.issn.1671-9352.1.2023.062
    Abstract ( 1363 )   HTML ( 21 )   PDF (1007KB) ( 3955 )   Save
    Figures and Tables | References | Related Articles | Metrics

    At present, most studies use some dimensionality reduction methods to convert high-dimensional vectors into low-dimensional vector representations, and then apply related vector retrieval optimization technology to achieve fast similarity retrieval, thereby improving the application performance of large models. Currently, there are many and scattered dimensionality reduction methods for high-dimensional data, and the dimensionality reduction methods used in different research backgrounds are different. Similarly, there are also many different retrieval ideas and optimization methods in vector retrieval technology. By reviewing the main ideas and optimization methods of recent dimensionality reduction and retrieval algorithms, this paper helps to generate inspiring connections between the two and support the development and in-depth research of subsequent high-dimensional vector retrieval optimization algorithms.

    Matrix product operator based sequential recommendation model
    Peiyu LIU,Bowen YAO,Zefeng GAO,Wayne Xin ZHAO
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  44-52, 104.  doi:10.6040/j.issn.1671-9352.1.2023.042
    Abstract ( 628 )   HTML ( 4 )   PDF (930KB) ( 585 )   Save
    Figures and Tables | References | Related Articles | Metrics

    The task of sequential recommendation confronts challenges characterized by high complexity and substantial diversity. The paradigm of pre-training and fine-tuning is extensively employed for learning item representations based on sequential data in recommendation scenarios. However, prevalent approaches tend to disregard the potential underfitting and overfitting issues that may arise during model fine-tuning in new domains. To address this concern, a novel neural network architecture grounded in the framework of matrix product operator (MPO) is introduced, and two versatile fine-tuning strategies are presented. Firstly, a lightweight fine-tuning approach that involves updating only a subset of parameters is proposed to effectively mitigate the problem of overfitting during the fine-tuning process. Secondly, an over-parameterization fine-tuning strategy is introduced by augmenting the number of trainable parameters, robustly addressing the issue of underfitting during fine-tuning. Through extensive experimentation on well-established open-source datasets, the efficacy of the proposed approach is demonstrated by achieving performance achievements. This serves as a compelling testament to the effectiveness of the proposed approach in addressing the challenge of general item representation in recommendation systems.

    A document-level event extraction method based on core arguments
    Chengjie SUN,Zongwei LI,Lili SHAN,Lei LIN
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  53-63.  doi:10.6040/j.issn.1671-9352.1.2023.080
    Abstract ( 857 )   HTML ( 2 )   PDF (3665KB) ( 1092 )   Save
    Figures and Tables | References | Related Articles | Metrics

    A document-level event extraction method based on core arguments(CA-DocEE) is proposed, which defines criteria for selecting core arguments based on their distributions in document-level events, uses heterogeneous graph convolutional neural networks to augment document contextual information for encoding argument entities, and captures deep semantic information in sentences based on machine reading comprehension methods for classifying the role of arguments. On the document-level event extraction dataset, the method proposed in this paper achieves a micro-average F1 value of 80.1%, which is comparable with the state-of-the-art methods.

    Noise network alignment method integrating multiple features
    Ning XIAN,Yixing FAN,Tao LIAN,Jiafeng GUO
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  64-75.  doi:10.6040/j.issn.1671-9352.1.2023.102
    Abstract ( 589 )   HTML ( 4 )   PDF (1975KB) ( 831 )   Save
    Figures and Tables | References | Related Articles | Metrics

    A multi-round iterative network alignment method is proposed to address the challenges of large structural differences and high noise sensitivity in anchor nodes in network alignment tasks. The method calculates node features of different dimensions using various heuristic approaches at each iteration, utilizing the combination of multiple features to assess the reliability of anchor nodes, filter potential noise, and enhance the confidence of each alignment round. Additionally, a graph neural network is employed to improve the consistency between nodes without attributes, mitigating the impact of structural differences in networks. Experimental results demonstrate that this method achieves high accuracy under high noise conditions, verifying its effectiveness.

    Factual error detection in knowledge graphs based on dynamic neighbor selection
    Liang GUI,Yao XU,Shizhu HE,Yuanzhe ZHANG,Kang LIU,Jun ZHAO
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  76-84.  doi:10.6040/j.issn.1671-9352.1.2023.097
    Abstract ( 942 )   HTML ( 5 )   PDF (1844KB) ( 742 )   Save
    Figures and Tables | References | Related Articles | Metrics

    The construction and updating of the knowledge graph(KG) usually depend on a wide range of web data and automated methods, inevitably resulting in factual inaccuracies in the modeled and acquired knowledge. To tackle this problem, a novelapproach for identifying factual inaccuracies within the knowledge graph is proposed. This method actively selects adjacent nodes of the facts to be checked, detecting errors by measuring the intricate associations linking the head and tail entities. More specifically, it first utilizes graph structure information to identify potential neighbors for each entity. Then, based on contextual information, it dynamically selects relevant neighbors and uses an efficient graph attention network to encode node features. Finally, by calculating the consistency of head and tail entity representations, it determines if the fact under consideration is erroneous. Experimental results on multiple public KG datasets demonstrate that this method outperforms existing approaches in error detection.

    Category-wise knowledge probers for representation learning of graph neural networks
    Xingyu HUANG,Mingyu ZHAO,Ziyu LYU
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  85-94.  doi:10.6040/j.issn.1671-9352.1.2023.064
    Abstract ( 627 )   HTML ( 2 )   PDF (3615KB) ( 1727 )   Save
    Figures and Tables | References | Related Articles | Metrics

    In order to solve the problem that the graph neural network model lacks corresponding probes, a knowledge detection framework for graph neural network representation learning is proposed, and two kinds of class-aware knowledge probes are designed based on the category attributes of data in different domains, namely clustering probes and contrastive clustering probes. The two probe the characterization effect of different models and give corresponding scores. On 8 datasets in 3 neighborhoods, including reference networks, social networks and biological networks, the representation learning of 7 classical graph neural network models realizes systematic knowledge detection and evaluation experiments, and summarizes the detection and evaluation conclusions.

    Probability distribution optimization model for learning to rank
    Fengxu ZHAO,Jian WANG,Yuan LIN,Hongfei LIN
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  95-104.  doi:10.6040/j.issn.1671-9352.1.2023.026
    Abstract ( 626 )   HTML ( 4 )   PDF (1775KB) ( 748 )   Save
    Figures and Tables | References | Related Articles | Metrics

    Existing learning to rank (LTR) models rely on the scores output by models to represent the partial order among documents. Considering the limitation of treating scores as deterministic values, this paper proposes a probability distribution optimization method for the LTR model, which introduces the uncertainty of the ranking score. It smooths the scores in the form of probability distributions, thereby transforming the comparison of ranking scores into the probability estimation of score partial orders. The proposed method is applied to LTR models such as RankNet, LambdaRank, and LambdaMART. It effectively bridges the gap between the modeled probability and the target probability, leading to optimization of the LTR models. The paper conducts experiments on multiple large-scale real datasets, and the experimental results show that the optimized models outperform the original ones, which validates the effectiveness of the proposed method.

    Multimodal conversation emotion recognition based on clustering and group normalization
    Qi LUO,Gang GOU
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  105-112.  doi:10.6040/j.issn.1671-9352.1.2023.055
    Abstract ( 793 )   HTML ( 6 )   PDF (3409KB) ( 449 )   Save
    Figures and Tables | References | Related Articles | Metrics

    It is a challenge for multimodal emotion recognition task that the confusion of similar emotion categories recognition leads to a decrease in recognition effect. To address this problem, a neural network modeling approach for relational graphs is proposed based on clustering group normalization. Firstly, three modal features are extracted using three different feature extractors and spliced by incorporating speaker encoding, which enriches the feature representation and preserves the original information. Secondly, contextual information is extracted using Transformer. Finally, after the feature nodes are input into the relational graph convolutional neural network, the nodes are clustered and grouped by clustering and independently normalized to make similar nodes more similar, which alleviates the problem that similar emotions are difficult to delimit. Through experimental validation, the network model can reach an 86.34% F1-score on the IEMOCAP dataset four classification, which verifies the effectiveness of the method in this paper. At present, the model achieves the best performance on this dataset.

    A prompt learning approach for telecom network fraud case classification
    Jie JI,Chengjie SUN,Lili SHAN,Boyue SHANG,Lei LIN
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  113-121.  doi:10.6040/j.issn.1671-9352.1.2023.040
    Abstract ( 1479 )   HTML ( 17 )   PDF (3845KB) ( 1360 )   Save
    Figures and Tables | References | Related Articles | Metrics

    For the automatic classification technology of telecom fraud cases, a classification system of telecom network fraud based on situational analysis is formulated, the privacy protection method of case text de-identification is realized, and accuracy and F1-score of a classification method of telecom network fraud cases based on prompt learning is proposed. The experimental results show that the method is on average 1 to 2 percentage points higher than the BERT-based classification method on the data set constructed in the paper.

    Chinese disease text classification model driven by medical knowledge
    Chao LI,Wei LIAO
    JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE). 2024, 59(7):  122-130.  doi:10.6040/j.issn.1671-9352.0.2023.291
    Abstract ( 1010 )   HTML ( 7 )   PDF (3349KB) ( 587 )   Save
    Figures and Tables | References | Related Articles | Metrics

    This study proposes a Chinese disease text classification model that integrates knowledge graph. Firstly, by introducing structured knowledge from external medical knowledge graph, a knowledge enhanced disease text vector representation is obtained; Secondly, the global semantic features and local semantic features of the disease text are extracted by using bidirectional long short-term memory network and convolutional neural network respectively. At the same time, the joint attention mechanism improves the efficiency of the model in extracting effective features information; Finally, the extracted features are concatenated and fused, and a classifier is used to output the classification result. The experimental results on the Chinese disease text dataset show that the proposed model has a classification accuracy, recall, and the harmonic mean value F1 of 95.21%, 95.64%, and 95.42%, respectively, which shows better classification performance compared to other models.