JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2023, Vol. 58 ›› Issue (9): 71-80.doi: 10.6040/j.issn.1671-9352.4.2022.2743

Previous Articles     Next Articles

Context fusion-based knowledge graph completion

Yujia NA1(),Jun XIE1,*(),Haiyang YANG1,Xinying XU2   

  1. 1. College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, Shanxi, China
    2. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, Shanxi, China
  • Received:2022-08-17 Online:2023-09-20 Published:2023-09-08
  • Contact: Jun XIE E-mail:2223760642@qq.com;xiejun@tyut.edu.cn

RichHTML

2

PDF (PC)

90

Abstract:

A knowledge graph completion model integrating context is constructed. Firstly, the deep interactive embedding of a given head-tail entity pair is obtained through the Inception network. Secondly, two types of context information of a given entity pair are defined and coded: adjacency and path context; Next, the attention mechanism based on head-to-tail interaction embedding is used to respectively aggregate the adjacency and path context of a given entity pair; Finally, the full connection layer is used to fuse the two types of context information of a given entity pair and consequently predict the relationship between the given entity pairs. Compared with other mainstream models in datasets FB15K-237, WN18RR and NELL-995, the experimental results show that the proposed model is effective.

Key words: knowledge graph completion, Inception network, adjacency relation, relational path, attention mechanism

CLC Number: 

  • TP391

Fig.1

Knowledge graph of A Dream of Red Mansions"

Fig.2

Frame diagram of CFKGC model"

Fig.3

Inception network structure diagram"

Fig.4

Original path from head entity to tail entity"

Table 1

Statistics of the datasets used"

数据集 实体数 关系数 训练集三元组数 验证集三元组数 测试集三元组数
FB15K-237 14 541 237 272 115 17 535 20 466
WN18RR 40 943 11 86 835 3 034 3 134
NELL-995 63 917 198 137 645 5 000 5 000

Table 2

Prediction results of different model relationships"

模型FB15K-237 WN18RR NELL-995
MRR Hits@1/% Hits@3/% MRR Hits@1/% Hits@3/% MRR Hits@1/% Hits@3/%
TransE 0.966 94.6 98.4 0.784 66.9 87.0 0.841 78.1 88.9
ComplEx 0.924 87.9 97.0 0.840 77.7 88.0 0.703 62.5 76.5
DistMult 0.875 80.6 93.6 0.847 78.7 89.1 0.634 52.4 72.0
ConvE 0.893 85.4 96.1 0.832 75.4 87.7 0.696 65.8 77.5
RotatE 0.970 95.1 98.0 0.799 75.3 82.3 0.729 69.1 75.6
ContE 0.912 90.6 92.8 0.784 75.6 83.6 0.713 69.9 70.3
DRUM 0.959 90.5 95.8 0.854 77.8 91.2 0.715 64.0 74.0
PathCon 0.979 96.4 99.4 0.974 95.4 99.4 0.896 84.4 94.1
CFKGC 0.979 96.7 99.6 0.962 95.1 99.5 0.911 85.6 95.5

Fig.5

The Hits@1 relationship prediction results of each module on FB15k-237"

Fig.6

The Hits@3 relationship prediction results of each module on FB15k-237"

Fig.7

The Hits@1 relationship prediction results of each module on WN18RR"

Fig.8

The Hits@3 relationship prediction results of each module on WN18RR"

Table 3

Influence of attention mechanism on prediction results"

模型FB15K-237 WN18RR
MRR Hits@1/% Hits@3/% MRR Hits@1/% Hits@3/%
c+p 0.951 93.3 96.3 0.929 91.6 94.2
attc+p 0.967 94.8 97.9 0.946 93.1 97.5
c+attp 0.968 95.0 97.8 0.944 93.5 97.1
attc+attp 0.979 96.7 99.6 0.962 95.1 99.5
1 SU Yu, YANG Shengqi, HUAN Sun, et al. Exploiting relevance feedback in knowledge graph search[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1135-1144.
2 BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
3 LEHMANN J , ISELE R , JAKOB M , et al. Dbpedia-A large-scale, multilingual knowledge base extracted from wikipedia[J]. Semantic Web, 2015, 6 (2): 167- 195.
doi: 10.3233/SW-140134
4 SUCHANEK F M, KASNECI G, WEIKUM G. Yago: a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. New York: ACM, 2007: 697-706.
5 SUN Yusheng , CHANG Kaiyue , ZHU Lijun . Research on large-scale knowledge graph and its application[J]. Information Studies: Theory & Application, 2018, 41 (11): 138- 143.
6 KROMPA D, BAIER S, TRESP V. Type-constrained representation learning in knowledge graphs[C]//Proceedings of International Semantic Web Conference. Berlin: Springer, 2015: 640-655.
7 ROSSI A , BARBOSA D , FIRMANI D , et al. Knowledge graph embedding for link prediction: a comparative analysis[J]. ACM Transactions on Knowledge Discovery from Data, 2020, 15 (2): 1- 49.
8 BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of Annual Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2013: 2787-2795.
9 DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graphembeddings[C]//Proceedings of the National Conferenceon Artificial Intelligence. Menlo Park: AAAI, 2018: 1811-1818.
10 SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
11 刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53 (2): 247- 261.
LIU Zhiyuan , SUN Maosong , LIN Yankai , et al. Knowledge representation learning: a review[J]. Journal of Computer Research and Development, 2016, 53 (2): 247- 261.
12 BAI Luyi , MA Xiangnan , ZHAN Mingcheng , et al. TPmod: a tendency-guided prediction model for temporal knowledge graph completion[J]. ACM Transactions on Knowledge Discovery from Data, 2021, 15 (3): 1- 17.
13 WANG Zhen, ZHANG Jianwen, FENG Jianlin, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2014: 1112-1119.
14 LIN Yankai, LIU Zhiyuan, SUN Maosong, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the Twenty-ninth AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2015: 2181-2187.
15 JI Guoliang, HE Shizhu, XU Liheng, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 1: Long Papers). Stroudsburg: ACL, 2015: 687-696.
16 JI Guoliang, LIU Kang, HE Shizhu, et al. Knowledge graph completion with adaptive sparse transfer matrix[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2016: 985-991.
17 JIA Yantao, WANG Yuanzhuo, LIN Hailun, et al. Locally adaptive translation for knowledge graph embedding[C]//Thirtieth AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2016: 992-998.
18 NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning. New York: ACM, 2011: 809-816.
19 YANG Bishan , YIH W T , HE Xiaodong , et al. Embedding entities and relations for learning and inference in knowledge bases[J]. Computer Science, 2014, 1412 (2): 6575- 6587.
20 TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the International Conference on Machine Learning. New York: ACM, 2016: 2071-2080.
21 MIKOLOV T , CHEN Kai , CORRADO G , et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013, 1301 (3): 3781- 3792.
22 SUN Zhiqing , DENG Zhihong , NIE Jianyun , et al. RotaTE: knowledge graph embedding by relational rotation in complex space[J]. Computer Science, 2019, 1902 (1): 10197- 10215.
23 EBISU T , ICHISE R . Generalized translation-based embedding of knowledge graph[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 32 (5): 941- 951.
doi: 10.1109/TKDE.2019.2893920
24 XIONG Bo, ZHU Shichao, NAYYERI M, et al. Ultrahyperbolic knowledge graph embeddings[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington: ACM, 2022: 14-18.
25 MOON C, HARENBERG S, SLANKAS J, et al. Learning contextual embeddings for knowledge graph completion[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. New York: ACM, 2017: 2215-2218.
26 周泽华, 陈恒, 李冠宇. 基于图上下文的知识表示学习[J]. 计算机应用与软件, 2021, 38 (6): 120- 125.
ZHOU Zehua , CHEN Heng , LI Guanyu . Knowledge representation learning based on graph context[J]. Computer Applications and Software, 2021, 38 (6): 120- 125.
27 ZHANG Qianjin , WANG Ronggui , YANG Juan , et al. Structural context-based knowledge graph embedding for link prediction[J]. Neurocomputing, 2022, 470 (22): 109- 120.
28 NING Zhiyuan, QIAO Ziyue, DONG Hao, et al. LightCAKE: a lightweight framework for context-aware knowledge graph embedding[J/OL]. arXiv, 2021. https://arxiv.org/abs/2102.10826v2.
29 LIN Yankai, LIU Zhiyuan, LUAN Hanbo, et al. Modeling relation paths for representation learning of knowledge bases[J/OL]. arXiv, 2015. https://arxiv.org/pdf/1506.00379.pdf.
30 SADEGHIAN A , ARMANDPOUR M , DING P , et al. DRUM: end-to-end differentiable rule mining on knowledgegraphs[J]. Advances in Neural Information Processing Systems, 2019, 32, 55- 66.
31 WANG Hongwei, REN Hongyu, LESKOVEC J. Relational message passing for knowledge graph completion[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: ACM, 2021: 1697-1707.
32 VASHISHTH S , SANYAL S , NITIN V , et al. InteractE: improving convolution-based knowledge graph embeddings by increasing feature interactions[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (3): 3009- 3016.
doi: 10.1609/aaai.v34i03.5694
33 GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Cambridge: MIT Press, 2011: 315-323.
34 SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Computer Society, 2016: 2818-2826.
35 MAAS A L , HANNUN A Y , NG A Y . Rectifier nonlinearities improve neural network acoustic models[J]. International Conference on Machine Learning, 2013, 30 (1): 3.
36 TOUTANOVA K, CHEN Danqi, PANTEL P, et al. Representing text for joint embedding of text and knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2015: 1499-1509.
37 XIONG Wenhan , HOANG T , WANG W Y . Deeppath: a reinforcement learning method for knowledge graph reasoning[J]. Computer Science, 2017, 1707 (3): 48550- 48560.
38 TOUTANOVA K, CHEN Danqi. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. Stroudsburg: ACL Press, 2015: 57-66.
39 VU T, NGUYEN T D, NGUYEN D Q, et al. A capsule network-based embedding model for knowledge graph completion and search personalization[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers). Stroudsburg: ACL, 2019: 2180-2189.
[1] WANG Jing-hong, LIANG Li-na, LI Hao-kang, WANG Xi-zhao. Community discovery algorithm based on label attention mechanism [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(12): 1-12.
[2] BAO Liang, CHEN Zhi-hao, CHEN Wen-zhang, YE Kai, LIAO Xiang-wen. Dual co-matching network with multiway attention for opinion reading comprehension [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2021, 56(3): 44-53.
[3] TANG Guang-yuan, GUO Jun-jun, YU Zheng-tao, ZHANG Ya-fei,GAO Sheng-xiang. Method of recommendation based on knowledge driven by BERT and law [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2021, 56(11): 24-30.
[4] Chang-ying HAO,Yan-yan LAN,Hai-nan ZHANG,Jia-feng GUO,Jun XU,Liang PANG,Xue-qi CHENG. Dialogue generation model based on extended keywords information [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2019, 54(7): 68-76.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] MAO Ai-qin1,2, YANG Ming-jun2, 3, YU Hai-yun2, ZHANG Pin1, PAN Ren-ming1*. Study on thermal decomposition mechanism of  pentafluoroethane fire extinguishing agent[J]. J4, 2013, 48(1): 51 -55 .
[2] LI Yong-ming1, DING Li-wang2. The r-th moment consistency of estimators for a semi-parametric regression model for positively associated errors[J]. J4, 2013, 48(1): 83 -88 .
[3] DONG Li-hong1,2, GUO Shuang-jian1. The fundamental theorem for weak Hopf module in  Yetter-Drinfeld module categories[J]. J4, 2013, 48(2): 20 -22 .
[4] ZHAO Jun1, ZHAO Jing2, FAN Ting-jun1*, YUAN Wen-peng1,3, ZHANG Zheng1, CONG Ri-shan1. Purification and anti-tumor activity examination of water-soluble asterosaponin from Asterias rollestoni Bell[J]. J4, 2013, 48(1): 30 -35 .
[5] YANG Yong-wei1, 2, HE Peng-fei2, LI Yi-jun2,3. On strict filters of BL-algebras#br#[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(03): 63 -67 .
[6] LI Min1,2, LI Qi-qiang1. Observer-based sliding mode control of uncertain singular time-delay systems#br#[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(03): 37 -42 .
[7] Ming-Chit Liu. THE TWO GOLDBACH CONJECTURES[J]. J4, 2013, 48(2): 1 -14 .
[8] ZHAO Tong-xin1, LIU Lin-de1*, ZHANG Li1, PAN Cheng-chen2, JIA Xing-jun1. Pollinators and pollen polymorphism of  Wisteria sinensis (Sims) Sweet[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(03): 1 -5 .
[9] WANG Kai-rong, GAO Pei-ting. Two mixed conjugate gradient methods based on DY[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(6): 16 -23 .
[10] LUO Si-te, LU Li-qian, CUI Ruo-fei, ZHOU Wei-wei, LI Zeng-yong*. Monte-Carlo simulation of photons transmission at alcohol wavelength in  skin tissue and design of fiber optic probe[J]. J4, 2013, 48(1): 46 -50 .
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd