融合上下文的知识图谱补全方法

doi:10.6040/j.issn.1671-9352.4.2022.2743

摘要/Abstract

摘要：

构建了一种融合了上下文的知识图谱补全模型。首先通过Inception网络得到给定头尾实体对的深度交互嵌入；其次定义和编码给定实体对的2种上、下文信息：邻接关系上下文和路径上下文；然后使用基于头尾交互嵌入的注意力机制，分别聚合给定实体对的邻接关系上下文和路径上下文；最后利用全连接层来融合给定实体对的2种上下文信息，预测给定实体对之间的关系。在数据集FB15K-237、WN18RR和NELL-995中与其他主流模型对比，实验结果证实了所提补全模型的有效性。

关键词: 知识图谱补全, Inception网络, 邻接关系, 关系路径, 注意力机制

Abstract:

A knowledge graph completion model integrating context is constructed. Firstly, the deep interactive embedding of a given head-tail entity pair is obtained through the Inception network. Secondly, two types of context information of a given entity pair are defined and coded: adjacency and path context; Next, the attention mechanism based on head-to-tail interaction embedding is used to respectively aggregate the adjacency and path context of a given entity pair; Finally, the full connection layer is used to fuse the two types of context information of a given entity pair and consequently predict the relationship between the given entity pairs. Compared with other mainstream models in datasets FB15K-237, WN18RR and NELL-995, the experimental results show that the proposed model is effective.

Key words: knowledge graph completion, Inception network, adjacency relation, relational path, attention mechanism

中图分类号:

TP391

那宇嘉,谢珺,杨海洋,续欣莹. 融合上下文的知识图谱补全方法[J]. 《山东大学学报(理学版)》, 2023, 58(9): 71-80.

Yujia NA,Jun XIE,Haiyang YANG,Xinying XU. Context fusion-based knowledge graph completion[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2023, 58(9): 71-80.

图/表 11

图1

图2

图3

图4

表1

表2

图5

图6

图7

图8

表3

参考文献 39

1	SU Yu, YANG Shengqi, HUAN Sun, et al. Exploiting relevance feedback in knowledge graph search[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1135-1144.
2	BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
3	LEHMANN J , ISELE R , JAKOB M , et al. Dbpedia-A large-scale, multilingual knowledge base extracted from wikipedia[J]. Semantic Web, 2015, 6 (2): 167- 195. doi: 10.3233/SW-140134
4	SUCHANEK F M, KASNECI G, WEIKUM G. Yago: a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. New York: ACM, 2007: 697-706.
5	SUN Yusheng , CHANG Kaiyue , ZHU Lijun . Research on large-scale knowledge graph and its application[J]. Information Studies: Theory & Application, 2018, 41 (11): 138- 143.
6	KROMPA D, BAIER S, TRESP V. Type-constrained representation learning in knowledge graphs[C]//Proceedings of International Semantic Web Conference. Berlin: Springer, 2015: 640-655.
7	ROSSI A , BARBOSA D , FIRMANI D , et al. Knowledge graph embedding for link prediction: a comparative analysis[J]. ACM Transactions on Knowledge Discovery from Data, 2020, 15 (2): 1- 49.
8	BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of Annual Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2013: 2787-2795.
9	DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graphembeddings[C]//Proceedings of the National Conferenceon Artificial Intelligence. Menlo Park: AAAI, 2018: 1811-1818.
10	SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
11	刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53 (2): 247- 261.
	LIU Zhiyuan , SUN Maosong , LIN Yankai , et al. Knowledge representation learning: a review[J]. Journal of Computer Research and Development, 2016, 53 (2): 247- 261.
12	BAI Luyi , MA Xiangnan , ZHAN Mingcheng , et al. TPmod: a tendency-guided prediction model for temporal knowledge graph completion[J]. ACM Transactions on Knowledge Discovery from Data, 2021, 15 (3): 1- 17.
13	WANG Zhen, ZHANG Jianwen, FENG Jianlin, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2014: 1112-1119.
14	LIN Yankai, LIU Zhiyuan, SUN Maosong, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the Twenty-ninth AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2015: 2181-2187.
15	JI Guoliang, HE Shizhu, XU Liheng, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 1: Long Papers). Stroudsburg: ACL, 2015: 687-696.
16	JI Guoliang, LIU Kang, HE Shizhu, et al. Knowledge graph completion with adaptive sparse transfer matrix[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2016: 985-991.
17	JIA Yantao, WANG Yuanzhuo, LIN Hailun, et al. Locally adaptive translation for knowledge graph embedding[C]//Thirtieth AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2016: 992-998.
18	NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning. New York: ACM, 2011: 809-816.
19	YANG Bishan , YIH W T , HE Xiaodong , et al. Embedding entities and relations for learning and inference in knowledge bases[J]. Computer Science, 2014, 1412 (2): 6575- 6587.
20	TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the International Conference on Machine Learning. New York: ACM, 2016: 2071-2080.
21	MIKOLOV T , CHEN Kai , CORRADO G , et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013, 1301 (3): 3781- 3792.
22	SUN Zhiqing , DENG Zhihong , NIE Jianyun , et al. RotaTE: knowledge graph embedding by relational rotation in complex space[J]. Computer Science, 2019, 1902 (1): 10197- 10215.
23	EBISU T , ICHISE R . Generalized translation-based embedding of knowledge graph[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 32 (5): 941- 951. doi: 10.1109/TKDE.2019.2893920
24	XIONG Bo, ZHU Shichao, NAYYERI M, et al. Ultrahyperbolic knowledge graph embeddings[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington: ACM, 2022: 14-18.
25	MOON C, HARENBERG S, SLANKAS J, et al. Learning contextual embeddings for knowledge graph completion[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. New York: ACM, 2017: 2215-2218.
26	周泽华, 陈恒, 李冠宇. 基于图上下文的知识表示学习[J]. 计算机应用与软件, 2021, 38 (6): 120- 125.
	ZHOU Zehua , CHEN Heng , LI Guanyu . Knowledge representation learning based on graph context[J]. Computer Applications and Software, 2021, 38 (6): 120- 125.
27	ZHANG Qianjin , WANG Ronggui , YANG Juan , et al. Structural context-based knowledge graph embedding for link prediction[J]. Neurocomputing, 2022, 470 (22): 109- 120.
28	NING Zhiyuan, QIAO Ziyue, DONG Hao, et al. LightCAKE: a lightweight framework for context-aware knowledge graph embedding[J/OL]. arXiv, 2021. https://arxiv.org/abs/2102.10826v2.
29	LIN Yankai, LIU Zhiyuan, LUAN Hanbo, et al. Modeling relation paths for representation learning of knowledge bases[J/OL]. arXiv, 2015. https://arxiv.org/pdf/1506.00379.pdf.
30	SADEGHIAN A , ARMANDPOUR M , DING P , et al. DRUM: end-to-end differentiable rule mining on knowledgegraphs[J]. Advances in Neural Information Processing Systems, 2019, 32, 55- 66.
31	WANG Hongwei, REN Hongyu, LESKOVEC J. Relational message passing for knowledge graph completion[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: ACM, 2021: 1697-1707.
32	VASHISHTH S , SANYAL S , NITIN V , et al. InteractE: improving convolution-based knowledge graph embeddings by increasing feature interactions[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (3): 3009- 3016. doi: 10.1609/aaai.v34i03.5694
33	GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Cambridge: MIT Press, 2011: 315-323.
34	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Computer Society, 2016: 2818-2826.
35	MAAS A L , HANNUN A Y , NG A Y . Rectifier nonlinearities improve neural network acoustic models[J]. International Conference on Machine Learning, 2013, 30 (1): 3.
36	TOUTANOVA K, CHEN Danqi, PANTEL P, et al. Representing text for joint embedding of text and knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2015: 1499-1509.
37	XIONG Wenhan , HOANG T , WANG W Y . Deeppath: a reinforcement learning method for knowledge graph reasoning[J]. Computer Science, 2017, 1707 (3): 48550- 48560.
38	TOUTANOVA K, CHEN Danqi. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. Stroudsburg: ACL Press, 2015: 57-66.
39	VU T, NGUYEN T D, NGUYEN D Q, et al. A capsule network-based embedding model for knowledge graph completion and search personalization[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers). Stroudsburg: ACL, 2019: 2180-2189.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

数据集	实体数	关系数	训练集三元组数	验证集三元组数	测试集三元组数
FB15K-237	14 541	237	272 115	17 535	20 466
WN18RR	40 943	11	86 835	3 034	3 134
NELL-995	63 917	198	137 645	5 000	5 000

模型	FB15K-237			WN18RR			NELL-995
模型	MRR	Hits@1/%	Hits@3/%	MRR	Hits@1/%	Hits@3/%	MRR	Hits@1/%	Hits@3/%
TransE	0.966	94.6	98.4	0.784	66.9	87.0	0.841	78.1	88.9
ComplEx	0.924	87.9	97.0	0.840	77.7	88.0	0.703	62.5	76.5
DistMult	0.875	80.6	93.6	0.847	78.7	89.1	0.634	52.4	72.0
ConvE	0.893	85.4	96.1	0.832	75.4	87.7	0.696	65.8	77.5
RotatE	0.970	95.1	98.0	0.799	75.3	82.3	0.729	69.1	75.6
ContE	0.912	90.6	92.8	0.784	75.6	83.6	0.713	69.9	70.3
DRUM	0.959	90.5	95.8	0.854	77.8	91.2	0.715	64.0	74.0
PathCon	0.979	96.4	99.4	0.974	95.4	99.4	0.896	84.4	94.1
CFKGC	0.979	96.7	99.6	0.962	95.1	99.5	0.911	85.6	95.5

模型	FB15K-237			WN18RR
模型	MRR	Hits@1/%	Hits@3/%	MRR	Hits@1/%	Hits@3/%
c+p	0.951	93.3	96.3	0.929	91.6	94.2
attc+p	0.967	94.8	97.9	0.946	93.1	97.5
c+attp	0.968	95.0	97.8	0.944	93.5	97.1
attc+attp	0.979	96.7	99.6	0.962	95.1	99.5

[1]	王静红,梁丽娜,李昊康,王熙照. 基于标记注意力机制的社区发现算法[J]. 《山东大学学报(理学版)》, 2022, 57(12): 1-12.
[2]	鲍亮,陈志豪,陈文章,叶锴,廖祥文. 基于双重多路注意力匹配的观点型阅读理解[J]. 《山东大学学报(理学版)》, 2021, 56(3): 44-53.
[3]	唐光远,郭军军,余正涛,张亚飞,高盛祥. 基于BERT与法条知识驱动的法条推荐方法[J]. 《山东大学学报(理学版)》, 2021, 56(11): 24-30.
[4]	郝长盈,兰艳艳,张海楠,郭嘉丰,徐君,庞亮,程学旗. 基于拓展关键词信息的对话生成模型[J]. 《山东大学学报(理学版)》, 2019, 54(7): 68-76.