JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2021, Vol. 56 ›› Issue (7): 91-102.doi: 10.6040/j.issn.1671-9352.0.2020.588

Previous Articles    

Multi-label feature selection based on manifold structure and flexible embedding

ZHANG Yao, MA Ying-cang*, YAND Xiao-fei, ZHU Heng-dong, YANG Ting   

  1. School of Science, Xian Polytechnic University, Xian 710600, Shaanxi, China
  • Published:2021-07-19

PDF (PC)

213

Abstract: The linear regression model and manifold structure are combined to form a weak linear multi-label feature selection framework. Firstly, the least square loss function is used to learn the regression coefficient matrix; secondly, the label manifold structure is used to learn the weight matrix of data features; thirdly, L2,1-norm is used to constrain the regression coefficient matrix and feature weight matrix, which can guide the sparsity and facilitate feature selection. In addition, an iterative updating algorithm with convergence is designed and proved to solve the above problems. Finally, the proposed method is verified on several classical multi-label datasets, and the experimental results show the effectiveness of the proposed algorithm.

Key words: multi-label learning, feature selection, logistic regression, L2,1-norm, manifold structure

CLC Number: 

  • TP181
[1] CAI Jie, LUO Jiawei, WANG Shulin, et al. Feature selection in machine learning: a new perspective[J]. Neurocomputing, 2018, 300:70-79.
[2] BERMINGHAM M L, PONG-WONG R, SPILIOPOULOU A, et al. Application of high-dimensional feature selection: evaluation for genomic prediction in man[J]. Scientific Reports, 2015, 5:10312.
[3] HASTIE T, TIBSHIRANI R, FRIEDMAN J. The elements of statistical learning: data mining, inference, and prediction[J]. The Mathematical Intelligencer, 2005, 27(2):83-85.
[4] SUN Xin, LIU Yanheng, LI Jin, et al. Using cooperative game theory to optimize the feature selection problem[J]. Neurocomputing, 2012, 97:86-93.
[5] ZHANG Rui, NIE Feiping, LI Xuelong, et al. Feature selection with multi-view data: a survey[J]. Information Fusion, 2019(50):158-167.
[6] DING Chuancang, ZHAO Ming, LIN Jing, et al. Multi-objective iterative optimization algorithm based optimal wavelet filter selection for multi-fault diagnosis of rolling element bearings[J]. ISA Trans, 2019, 82:199-215.
[7] LABANI M, MORADI P, AHMADIZAR F, et al. A novel multivariate filter method for feature selection in text classification problems[J]. Engineering Applications of Artificial Intelligence, 2018, 70:25-37.
[8] YAO Chao, LIU Yafeng, JIANG Bo, et al. LLE score: a new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition[J]. IEEE Transactions on Image Processing, 2017, 26(11):5257-5269.
[9] GONZALEZ J, ORTEGA J, DAMAS M, et al. A new multi-objective wrapper method for feature selection: Accuracy and stability analysis for BCI[J]. Neurocomputing, 2019, 333:407-418.
[10] SWATI J, HONGMEI H, KARL J. Information gain directed genetic algorithm wrapper feature selection for credit rating[J]. Appl Soft Comput, 2018(69):541-553.
[11] MALDONADO S, LÓPEZ J. Dealing with high-dimensional class-imbalanced datasets: embedded feature selection for SVM classification[J]. Appl Soft Comput, 2018(67):94-105.
[12] KONG Yunchuan, YU Tianwei. A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data[J]. Bioinformatics, 2018, 34(21):3727-3737.
[13] SUN Zhenqiang,ZHANG Jia, DAI Liang, et al. Mutual information based multi-label feature selection via constrained convex optimization[J]. Neurocomputing, 2019, 329:447-456.
[14] ZHANG Ping, LIU Guixia, GAO Wanfu. Distinguishing two types of labels for multilabel feature selection[J]. Pattern Recognition. 2019, 95:72-82.
[15] CHEN Linlin, CHEN Degang. Alignment based feature selection for multi-label learning[J]. Neural Processing Letters. 2019, 50(2/3):28-36.
[16] CHEN Sibao, ZHANG Yumei, CHRIS H Q, et al. Extended adaptive Lasso for multi-class and multi-label feature selection[J]. Knowledge-Based Systems, 2019, 173:28-36.
[17] ZHANG Jia, LUO Zhiming, LI Candong, et al. Manifold regularized discriminative feature selection for multi-label learning[J]. Pattern Recognition. 2019, 95(1):136-150.
[18] 蔡志铃, 祝峰. 非负稀疏表示的多标签特征选择[J]. 计算机科学与探索, 2017, 11(7):1175-1182. CAI Zhiling, ZHU William. Multi-label feature selection via non-negative sparse representation[J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(7):1175-1182.
[19] GU Quanquan, ZHOU Jie. Co-clustering on manifolds [C] //ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2009: 359-368.
[20] NIE Feiping, HUANG Heng, CAI Xiao, et al. Efficient and robust feature selection via joint L21-norms minimization [C] //Proceedings of the 23rd International Conference on Neural Information Processing Systems-Volume 2. New York: Curran Associates Inc, 2010: 1813-1821.
[21] ZHANG Minling, ZHOU Zhihua. ML-KNN: a lazy learning approach to multi-label learning[J]. Pattern Recognit, 2007, 40(7):2038-2048.
[22] LEE J, KIM D W. SCLS: multi-label feature selection based on scalable criterion for large label set[J]. Pattern Recognition. 2017, 66(1):342-352.
[23] LIN Yaojin, HU Qinghua, LIU Jinghua, et al. Multi-label feature selection based on max-dependency and min-redundancy[J]. Neurocomputing. 2015, 168(30):92-103.
[24] LEE J, KIM D W. Feature selection for multi-label classification using multivariate mutual information[J]. Pattern Recognit Lett, 2013, 34(3):349-357
[25] LEE J, KIM D W. Fast multi-label feature selection based on information-theoretic feature ranking[J]. Pattern Recognition, 2015, 48(9):2761-2771.
[26] DOUGHERTY J, KOHAVI R, SAHAMI M, et al. Supervised and unsupervised discretization of continuous features[C] //Proceedings of the Twelfth International Conference on Machine Learning. Tahoe: Elsevier Inc, 1995: 194-202.
[1] Ying YU,Xin-nian WU,Le-wei WANG,Ying-long ZHANG. A multi-label three-way classification algorithm based on label correlation [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2020, 55(3): 81-88.
[2] HUANG Tian-yi, ZHU William. Cost-sensitive feature selection via manifold learning [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(3): 91-96.
[3] WAN Zhong-ying, WANG Ming-wen, ZUO Jia-li, WAN Jian-yi. Feature selection combined with the global and local information(GLFS) [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(5): 87-93.
[4] LI Zhao,SUN Zhan-,LI Xiao,LI Cheng,. Study on feature selection method based on information loss [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(11): 7-12.
[5] GAN Xin-jun, YANG Wei-qiang. Weight of evidence method and credit risk control [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(12): 55-59.
[6] XIA Meng-nan, DU Yong-ping, ZUO Ben-xin. Micro-blog opinion analysis based on syntactic dependency and feature combination [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 22-30.
[7] ZHENG Yan, PANG Lin, BI Hui, LIU Wei, CHENG Gong. Feature selection algorithm based on sentiment topic model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 74-81.
[8] PAN Qing-qing, ZHOU Feng, YU Zheng-tao, GUO Jian-yi, XIAN Yan-tuan. Recognition method of Vietnamese named entity based on#br# conditional random fields [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(1): 76-79.
[9] YU Ran 1,2, LIU Chun-yang3*, JIN Xiao-long 1, WANG Yuan-zhuo 1, CHENG Xue-qi 1. Chinese spam microblog filtering based on the fusion of
multi-angle features [J]. J4, 2013, 48(11): 53-58.
[10] FENG Xin-ying1,2, JI Hua1,2, ZHANG Hua-xiang1,2. Multi-label RBF neural networks learning algorithm  based on clustering optimization [J]. J4, 2012, 47(5): 63-67.
[11] YI Chao-qun, LI Jian-ping, ZHU Cheng-wen. A kind of feature selection based on classification accuracy of SVM [J]. J4, 2010, 45(7): 119-121.
[12] YANG Yu-Zhen, LIU Pei-Yu, SHU Zhen-Fang, QIU Ye. Research of an improved information gain methodusing distribution information of terms [J]. J4, 2009, 44(11): 48-51.
[13] YUAN Xiao-hang,DU Xiao-yong . iRIPPER: an improved rule-based text categorization algorithm [J]. J4, 2007, 42(11): 66-68 .
[14] YU Jun-ying,WANG Ming-wen,SHENG Jun . Class information feature selection method for text classification [J]. J4, 2006, 41(3): 144-148 .
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd