一种基于文本语义扩展的记忆网络查询建议模型

doi:10.6040/j.issn.1671-9352.2.2022.484

摘要/Abstract

摘要：

提出了一种新的基于文本语义扩展的记忆网络模型, 用于生成环境感知的查询建议。采用基于注意力机制的分层编码器-解码器模型, 利用外部记忆网络, 生成查询与查询相关文档之间的神经注意力向量。模型融合了查询层、会话层和文档层语义信息, 与目前的研究方法相比, 能生成具有更高相关性的环境感知查询建议。使用真实的商业搜索引擎查询日志进行了实验, 实验结果表明了该模型的有效性。

关键词: 查询建议, 文本语义扩展, 环境感知, 记忆网络, 编码器-解码器模型

Abstract:

A novel memory network model based on the semantic expansion of text is proposed to generate context-aware query suggestions. An attention-based hierarchical encoder-decoder model is employed, utilizing an external memory network to generate the neural attention vector between the query and the related document. The model fuses query-layer, session-layer, and document-layer semantic information. Compared with state-of-the-art approaches, our model can generate context-aware query suggestions with higher relevance. Extensive experiments using real commercial search engine query logs demonstrate the effectiveness of the proposed model.

Key words: query suggestion, semantic expansion of text, context-aware, memory network, encoder-decoder model

中图分类号:

TP391

张乃洲,曹薇. 一种基于文本语义扩展的记忆网络查询建议模型[J]. 《山东大学学报(理学版)》, 2023, 58(12): 10-21.

Naizhou ZHANG,Wei CAO. A memory network model based on semantic expansion of text for query suggestion[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2023, 58(12): 10-21.

图/表 8

图1

图2

表1

表2

表3

表4

表5

表6

各类模型生成的查询建议例子"

#	查询环境+尾查询	不同模型生成的查询建议
#	查询环境+尾查询	ADJ	HRED	Transformer	SETMN+Transformer	SETMN+HRED+Att
1		verizoncom	verizon wireless	verizon wireless	verizon wireless	verizon wireless
		verizonwirelesscom	verizon phone service	verizon phone	next com	best buy
		verizon wireless	verizon home phone	singular wireless	verizon ringtones	circuit city
		wwwverizoncom	verizon central com	aol phone	verizon phone	comcast net
2		frontier airlines com	frontier airlines com	frontier airlines	southwest airlines	continental airlines
		frontier airlines	america west airlines	northwest airlines	delta airlines	frontier airlines
		unitedairlinescom	united airlines	delta airlines	northwest airlines	southwest airlines
		southwestairlinescom	southwest airlines	alaska airlines	airline tickets	delta airlines
3		aol browser	aol media player	internet explorer	aol browser	aol browser
		aol cookes	aol browser	aol browser	windows media	browser settings
		google	aol live help	aol explorer	aol spyware	web unlock
		setup	windows media player	aol upgrade	aol antivirus	microsoft internet browser
4		baseball bats	baseball bats	baseball scores	sporting goods	pottery barn
		training aids for baseball	baseball glasses	baseball bats	baseball bats	baseball tickets
		batting gloves	baseball warehouse com	sports authority	baseball hats	white sox
		animated bats	baseball tools com	baseball shoes	baseball players	dickssportinggoods com
5		enterprise	national car rental	car rental	kelly rentacar	toyota financial
		hertz	enterprise car rental	avis car	car rental	boston city toyota
		budget	car rental rates	avis rental	auto repair	car rentals
		alamo	budget car rental	enterprise rental	auto insurance	leader toyota
6			map of australia	us map	www map	united states map of america
			maps of nigra falls	united airlines	united airlines	map of canada
			united states map	www map	weather channel	list of hotels
			world atlas	cheap flights	washington state	falls of canada
7			balls fitness	american idol	total fitness	gold gym fitness
			fitness equipment	american eagle	american eagle	weekend black fitness
			fitness clubs	best buy	fitness equipment	german wet spa
			fitness arts outlet	home depot	total gym	tv fitness for children
8		toyota cars	new toyota cars	mercedes benz	used cars	toyota dealers
		nissan cars	nissan dealers	used cars	toyota corolla	honda dealers
		new cars	new dodge	toyota suv	toyota cars	used toyota dealers
		toyota deals	nissan cars	toyota dealers	nissan dealers	top car dealers

表6

参考文献 24

1	ARAMPATZIS Avi, KAMPS Jaap. A study of query length[C]//Proceedings of the 31th Annual International ACM SIGIR Conference (SIGIR). Singapore: ACM, 2008: 811-812.
2	SORDONI Alessandro, BENGIO Yoshua, VAHABI Hossein, et al. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM). Melbourne: ACM, 2015: 553-562.
3	MEI Qiaozhu, ZHOU Dengyong, CHURCH Kenneth. Query suggestion using hitting time[C]//Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM). California: ACM, 2008: 469-478.
4	CAO Huanhuan, JIANG Daxin, PEI Jian, et al. Context-aware query suggestion by mining click-through and session data[C]//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(KDD). Las Vegas: ACM, 2008: 875-883.
5	HU Hao, ZHANG Mingxi, HE Zhenying, et al. Diversifying query suggestions by using topics from wikipedia[C]//ACM International Conferences on Web Intelligence(WI). Atlanta: ACM, 2013: 139-146.
6	JAIN Alpa, OZERTEM Umut, VELIPASAOGLU Emre. Synthesizing high utility suggestions for rare web search queries[C]//Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). Beijing: ACM, 2011: 805-814.
7	SHOKOUHI Milad. Learning to personalize query auto-completion[C]//Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). Dublin: ACM, 2013: 103-112.
8	JIANG J Y, KE Y Y, CHIEN P Y, et al. Learning user reformulation behavior for query auto-completion[C]//Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). Gold Coast: ACM, 2014: 445-454.
9	HE Qi, JIANG Daxin, LIAO Zhen, et al. Web query recommendation via sequential query prediction[C]//Proceedings of the 25th International Conference on Data Engineering(ICDE). Shanghai: IEEE, 2009: 1443-1454.
10	BAHDANAU Dzmitry, CHO Kyunghyun, BENGIO Yoshua. Neural machine translation by jointly learning to align and translate[C]//The 3rd International Conference on Learning Representations(ICLR). San Diego: ICLR, 2015.
11	VASWANI Ashish, SHAZEER Noam, PARMAR Niki, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30(NIPS). Long Beach: MIT, 2017: 5998-6008.
12	AHMAD Uddin Wasi, CHANG Kaiwei, WANG Hongning. Context attentive document ranking and query suggestion[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). Paris: ACM, 2019: 385-394
13	HAN F X, NIU D, LAI K F, et al. Inferring search queries from web documents via a graph-augmented sequence to attention network[C]//Proceedings of the 2019 World Wide Web Conference on World Wide Web (WWW). San Francisco: ACM, 2019: 2792-2798.
14	ZHONG Jianling, GUO Weiwei, GAO Huiji, et al. Personalized query suggestions[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). Paris: ACM, 2020: 1645-1648.
15	MUSTAR Agnès, LAMPRIER Sylvain, PIWOWARSKI Benjamin. On the study of transformers for query suggestion[J]. ACM Transactions on Information Systems, 2022, 40(1): 18: 1-18: 27.
16	YU Shi, LIU Jiahua, YANG Jingqin, et al. Few-shot generative conversational query rewriting[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). [S.l.]: ACM, 2020: 1933-1936.
17	NOGUEIRA Rodrigo, YANG Wei, LIN Jimmy, et al. Document expansion by query prediction[J/OL]. arXiv. 2019. http://arxiv.org/abs/1904.08375.
18	DAI Zhuyun, CALLAN Jamie. Deeper text understanding for IR with contextual neural language modeling[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR). Paris: ACM, 2019: 985-988.
19	YANG Wei, ZHANG Haotian, LIN Jimmy. Simple applications of BERT for ad hoc document retrieval[J/OL]. arXiv. 2019. https://arxiv.org/abs/1903.10972.
20	WESTON Jason, CHOPRA Sumit, BORDES Antoine. Memory networks[C]//The 3rd International Conference on Learning Representations (ICLR). San Diego: ICLR, 2015.
21	CHEN Peng, SUN Zhongqian, BING Lidong, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing(EMNLP). Copenhagen Copenhagen: [s. n. ], 2017: 452-461.
22	TANG Duyu, QIN Bing, LIU Ting. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin: MIT, 2016: 214-224.
23	PAPINENI Kishore, ROUKOS Salim, WARD Todd, et al. Bleu: a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002). Philadelphia: MIT, 2002: 311-318.
24	HAGEN Matthias, POTTHAST Martin, BEYER Anna, et al. Towards optimum query segmentation: in doubt without[C]//Proceedings of the 21st ACM International on Conference on Information and Knowledge Management (CIKM). Maui: ACM, 2012: 1015-1024.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

数据集	划分时间区间(年-月-日)	数据集大小
ModelTrainSet	2006-03-01~2006-04-30	2 184 095
L2RTrainSet	2006-05-01~2006-05-14	638 362
ValidSet	2006-05-15~2006-05-24	328 924
TestSet	2006-05-25~2006-05-31	219 890

模型	BLEU-1	BLEU-2	BLEU-3	BLEU-4
HRED^[2]	32.605	9.698	10.278	8.554
Transformer	31.068	7.408	8.947	7.663
SETMN+Transformer	32.102	8.326	9.311	7.832
SETMN+HRED+Att	36.005	11.815	10.071	9.685

模型	BLEU-1	BLEU-2	BLEU-3	BLEU-4
HRED^[2]	39.126	18.426	17.417	13.685
Transformer	36.039	14.075	14.315	13.793
SETMN+Transformer	38.522	17.485	16.994	14.231
SETMN+HRED+Att	43.566	25.048	17.117	16.185

数据集	数据集大小
L2RTrainSet^*	19 108
ValidSet^*	13 189
TestSet^*	10 288

模型	MRR
ADJ^[2]	0.511 4
Baseline Ranker(BR)^[2]	0.545 9
HRED+BR^[2]	0.553 3
Transformer+BR	0.548 6
SETMN+Transformer+BR	0.550 1
SETMN+HRED+Att+BR	0.566 8