您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（理学版）》

山东大学学报（理学版） ›› 2017, Vol. 52 ›› Issue (6): 24-31.doi: 10.6040/j.issn.1671-9352.5.2016.023

• • 上一篇    下一篇

面向序列迁移学习的似然比模型选择方法

孙世昶1,2,林鸿飞1,孟佳娜2*,刘洪波3   

  1. 1.大连理工大学计算机学院, 辽宁 大连 116023;2.大连民族大学计算机学院, 辽宁 大连116600;3. 大连海事大学信息科学技术学院, 辽宁 大连 116026
  • 收稿日期:2016-10-11 出版日期:2017-06-20 发布日期:2017-06-21
  • 通讯作者: 孟佳娜(1972— ),女,博士,教授,研究方向为文本挖掘和信息检索. E-mail:mengjn@dlnu.edu.cn E-mail:ssc@dlnu.edu.cn
  • 作者简介:孙世昶(1979— ),男,博士研究生,讲师,研究方向为机器学习与文本挖掘. E-mail:ssc@dlnu.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(61472058,61572102);辽宁省自然科学基金引导计划项目(201602195,201700334);中央高校自主基金资助项目(DC201502030202)

Model selection with likelihood ratio for sequence transfer learning

SUN Shi-chang1,2, LIN Hong-fei1, MENG Jia-na2*, LIU Hong-bo3   

  1. 1. School of Computer, Dalian University of Technology, Dalian 116023, Liaoning, China;
    2. School of Computer, Dalian Minzu University, Dalian 116600, Liaoning, China;
    3. Information Science and Technology College, Dalian Maritime University, Dalian 116026, Liaoning, China
  • Received:2016-10-11 Online:2017-06-20 Published:2017-06-21

PDF (PC)

875

摘要： 为了解决迁移学习的欠适配问题,将粒模型作为候选模型的集合,通过模型选择的方式引入目标域的辅助模型中包含的标注规则,提出粒模型推断中基于似然比的模型选择方法(likelihood ratio model selection, LRMS),实现了辅助模型与粒模型的融合。LRMS保持基于Viterbi算法的标注模型对整条序列进行计算的模式,避免了候选标注器对上下文关系的破坏。通过大量词性标注实验表明LRMS在每个迁移学习任务中都有准确率的提高,从而证明似然比模型选择是一种有效的解决欠适配问题的方法。

关键词: 迁移学习, 似然比, 词性标注, 模型选择

Abstract: To solve the under-adaptation problem of transfer learning,in this paper the granular model is used as a set of candidate models, and labeling rules contained in minor for target domain models is introduced by a model selection method. We propose a Likelihood Ratio based Model Selection method(LRMS)for the inference of granular model, which implements the fusion of minor models with the granular model. LRMS keeps the single-path calculating of Viterbi-based sequence labeling model, which avoid the violation of contextual connections. In empirical experiments on part-of-speech tagging, LRMS improves the accuracy in every transfer learning task, therefore, the effectiveness of LRMS in solving the under-adaptation problem is verified.

Key words: transfer learning, likelihood ratio, part-of-speech tagging, model selection

中图分类号: 

  • TP391
[1] PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.
[2] PEDRYCZ W, RUSSO B, SUCCI G. Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity[J]. Applied Soft Computing, 2012, 12(8):1985-1995.
[3] 龙明盛. 迁移学习问题与方法研究[D]. 北京:清华大学, 2014. LONG Mingsheng. Transfer learning: problems and methods[J].Beijing: Tsinghua University, 2014.
[4] LOPER E, BIRD S. NLTK: the natural language toolkit[C] // Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics. Stroudsburg: Association for Computational Linguistics, 2006: 69-72.
[5] RABINER L. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2):257-286.
[6] SUTTON C, MCCALLUM A. Composition of conditional random fields for transfer learning[C] // Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2005: 748-754.
[7] BRANTS T. TnT: a statistical part-of-speech tagger[C] // Proceedings of the 6th Conference on Applied Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2000: 224-231.
[8] AIT-MOHAND K, PAQUET T, RAGOT N. Combining structure and parameter adaptation of HMMs for printed text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(9):1716-1732.
[9] KIM N S, SUNG J S, HONG D H. Factored MLLR adaptation[J]. Signal Processing Letters, 2011, 18(2):99-102.
[10] SONG M, PEDRYCZ W. Granular neural networks: concepts and development schemes[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(4):542-553.
[11] LIM C H, VATS E, CHAN C S. Fuzzy human motion analysis: a review[J]. Pattern Recognition, 2015, 48(5):1773-1796.
[12] CAI Q, ZHANG D, ZHENG W, et al. A new fuzzy time series forecasting model combined with ant colony optimization and auto-regression[J]. Knowledge-Based Systems, 2015, 74:61-68.
[13] SUN B, GUO H, KARIMI H R, et al. Prediction of stock index futures prices based on fuzzy sets and multivariate fuzzy time series[J]. Neurocomputing, 2015, 151:1528-1536.
[14] SIDDIQI S M, GORDON G J, MOORE A W. Fast state discovery for HMM model selection and learning[J] //AISTATS. 2007, 2:492-499.
[15] WANG S, ZHAO Y. Online Bayesian tree-structured transformation of HMMs with optimal model selection for speaker adaptation[J]. IEEE Transactions on Speech and Audio Processing, 2001, 9(6):663-677.
[16] ESCALANTE H J, MONTES M, SUCAR L E. Particle swarm model selection[J]. Journal of Machine Learning Research, 2009, 10(Feb):405-440.
[17] GOLDWATER S, GRIFFITHS T. A fully Bayesian approach to unsupervised part-of-speech tagging[C] // Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2007, 45:744-751.
[18] PEDRYCZ W. Granular computing: analysis and design of intelligent systems [M]. Boca Raton: CRC Press, 2013.
[19] GAUVAIN J L, LEE C H. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains[J]. IEEE Transactions on Speech and Audio Processing, 1994, 2(2):291-298.
[1] 余传明,冯博琳,田鑫,安璐. 基于深度表示学习的多语言文本情感分析[J]. 山东大学学报（理学版）, 2018, 53(3): 13-23.
[2] 马云艳,栾贻会*. 基于局部LRS方法的稀疏信号片段检测[J]. J4, 2012, 47(12): 1-5.
[3] 黄贤立，罗冬梅. 倾向性文本迁移学习中的特征重要性研究[J]. J4, 2010, 45(7): 13-17.
[4] . 具有AR(2)误差非线性回归模型的联合检验[J]. J4, 2009, 44(7): 38-43.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《山东大学学报(理学版)》编辑部　
地址: 山东省济南市山大南路27号山东大学中心校区（250100）　电话: +86-0531-88366917　E-mail: xblxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发