JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2024, Vol. 59 ›› Issue (3): 61-70.doi: 10.6040/j.issn.1671-9352.7.2023.1073

•   • Previous Articles     Next Articles

Hierarchical feature selection algorithm based on instance correlations

Chunyu SHI1,2(),Yu MAO1,2,*(),Haoyang LIU1,2,Yaojin LIN1,2   

  1. 1. School of Computer Science, Minnan Normal University, Zhangzhou 363000, Fujian, China
    2. Key Laboratory of Data Science and Intelligence Application(Minnan Normal University), Zhangzhou 363000, Fujian, China
  • Received:2023-04-29 Online:2024-03-20 Published:2024-03-06
  • Contact: Yu MAO E-mail:shichunyuuu@163.com;maoyu_bit@163.com

RichHTML

0

PDF (PC)

122

Abstract:

A hierarchical feature selection algorithm based on instance correlations (HFSIC) is proposed to further improve the performance of the hierarchical feature selection algorithm. After using sparse regularization items to remove irrelevant features, the parent-child relationship in the hierarchical structure with the reconstruction relationship between samples in the feature space are combined. The correlation of samples of each category under the same subtree are learned. Recursive regularization to optimize the output features weight matrix is used. When measuring the sample correlation, the reconstructed coefficient matrix is integrated into the training model, and the norm is used to remove irrelevant and redundant features. The optimization problem of the proposed model is solved using the accelerated proximal gradient method, and the superiority of the proposed algorithm is evaluated under multiple evaluation metrics. The experimental results show that the proposed method outperforms the other algorithms on five datasets. The test verifies the effectiveness of the proposed algorithm.

Key words: feature selection, hierarchical structure, instance correlation, recursive regularization

CLC Number: 

  • TP391

Table 1

Data set description"

序号 数据集 训练集数 测试集数 特征数 节点数 叶子节点数 层数
1 DD 3 020 605 473 32 27 3
2 F194 7 105 1 420 473 202 194 3
3 VOC 7 178 5 105 1 000 30 20 5
4 CLEF 8 368 939 80 88 63 4
5 ILSVRC65 12 346 11 845 4 096 65 57 4

Table 2

Standard TIE results of different feature selection algorithms on different data sets(↓)"

数据集 TTIE
HierFSNM HiermRMR Hier-FS HiRRfam-FS HFSDK HFSIC
F194 0.212 3(6) 0.180 0(5) 0.174 6(3) 0.173 0(2) 0.175 2(4) 0.166 0(1)
DD 0.088 6(5) 0.091 9(6) 0.085 0(3) 0.083 6(1) 0.086 3(4) 0.083 9(2)
ILSVRC65 0.035 0(5) 0.033 5(6) 0.032 8(2) 0.032 8(2) 0.032 9(4) 0.032 6(1)
VOC 0.214 4(5) 0.2188(6) 0.214 3(3) 0.214 3(3) 0.212 6(2) 0.208 7(1)
CLEF 0.207 7(6) 0.182 5(3) 0.182 6(4) 0.182 6(4) 0.174 5(2) 0.173 5(1)
平均排名 5.4 5.2 3 2.4 3.2 1.2

Table 3

Hierarchical-F1 measure results of different feature selection algorithms on different data sets(↑)"

数据集 FH
HierFSNM HiermRMR Hier-FS HiRRfam-FS HFSDK HFSIC
F194 0.646 2(6) 0.700 0(5) 0.708 9(3) 0.711 2(2) 0.707 5(4) 0.712 7(1)
DD 0.852 4(5) 0.846 8(6) 0.858 4(3) 0.860 6(1) 0.859 0(2) 0.858 4(3)
ILSVRC65 0.956 3(6) 0.958 1(5) 0.959 1(2) 0.958 8(4) 0.958 9(3) 0.959 2(1)
VOC 0.673 9(5) 0.666 9(6) 0.675 4(3) 0.675 8(3) 0.677 2(2) 0.682 0(1)
CLEF 0.739 6(6) 0.763 5(3) 0.763 1(4) 0.762 3(5) 0.774 2(2) 0.775 5(1)
平均排名 5.6 5 3 3 2.6 1.4

Fig.1

Comparing the performance of HFSIC algorithm with other algorithms through Bonferroni-Dunn test"

Fig.2

Ablation results based on F194 and VOC data sets"

Fig.3

Parameter sensitivity analysis based on F194 data set"

Fig.4

Parameter sensitivity analysis based on VOC data set"

Fig.5

Convergence curve of objective function values"

1 王忠伟, 陈叶芳, 钱江波, 等. 基于LSH的高维大数据k近邻搜索算法[J]. 电子学报, 2016, 44 (4): 906- 912.
doi: 10.3969/j.issn.0372-2112.2016.04.022
WANG Zhongwei , CHEN Yefang , QIAN Jiangbo , et al. LSH-based algorithm for k nearest neighbor search on bigdata[J]. Acta Electronica Sinica, 2016, 44 (4): 906- 912.
doi: 10.3969/j.issn.0372-2112.2016.04.022
2 胡清华, 王煜, 周玉灿, 等. 大规模分类任务的分层学习方法综述[J]. 中国科学(信息科学), 2018, 48 (5): 487- 500.
HU Qinghua , WANG Yu , ZHOU Yucan , et al. A review on hierarchical learning methods for large scale classification task[J]. Sci Sin Inform, 2018, 48 (5): 487- 500.
3 DUDA R O , HART P E , STORK D G . Pattern classification[M]. Hoboken: Wiley, 2000.
4 LIU Xinxin , ZHOU Yucan , ZHAO Hong . Robust hierarchical feature selection driven by data and knowledge[J]. Information Sciences, 2021, 551, 341- 357.
doi: 10.1016/j.ins.2020.11.003
5 WANG Jian , ZHANG Huaqing , WANG Junze , et al. Feature selection using a neural network with group lasso regularization and controlled redundancy[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32 (3): 1110- 1123.
6 林耀进, 白盛兴, 赵红, 等. 基于标签关联性的分层分类共有与固有特征选择[J]. 软件学报, 2022, 33 (7): 2667- 2682.
LIN Yaojin , BAI Shengxing , ZHAO Hong , et al. A label correlation based common and specific feature selection for large-scale hierarchical classification[J]. Journal of Software, 2022, 33 (7): 2667- 2682.
7 FREEMAN C, KULIC D, BASIR O. Joint feature selection and hierarchical classifier design[C]//2011 IEEE International Conference on Systems, Man and Cybernetics. Waterloo: IEEE, 2011: 1728-1734.
8 FREEMAN C , KULIC D , BASIR O , et al. Feature-selected tree-based classification[J]. IEEE Transactions on Cybernetics, 2013, 43 (6): 1990- 2004.
doi: 10.1109/TSMCB.2012.2237394
9 GRIMAUDO L, MELLIA M, BARALIS E. Hierarchical learning for fine grained internet traffic classification[C]//2012 8th International Wireless Communications and Mobile Computing Conference (IWCMC). Copenhagen: IEEE, 2012: 463-468.
10 ZHAO Hong , HU Qinghua , ZHU Pengfei , et al. A recursive regularization based feature selection framework for hierarchical classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33 (7): 2833- 2846.
doi: 10.1109/TKDE.2019.2960251
11 TUO Qianjuan , ZHAO Hong , HU Qinghua . Hierarchical feature selection with subtree based graph regularization[J]. Knowledge-Based Systems, 2018, 163 (1): 996- 1008.
12 DE ABREU I B M, MANTOVANI R G, CERRI R. Incorporating instance correlations in multi-label classification via label-space[C]//2017 International Joint Conference on Neural Networks (IJCNN). Anchorage: IEEE, 2017: 581-588.
13 HUANG Shengjun, ZHOU Zhihua. Multi-label learning by exploiting label correlations locally[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Toronto, Ontario: AAAI, 2012, 26(1): 949-955.
14 HUANG Jun , LI Guorong , HUANG Qingming , et al. Joint feature selection and classification for multilabel learning[J]. IEEE Transactions on Cybernetics, 2018, 48 (3): 876- 889.
doi: 10.1109/TCYB.2017.2663838
15 LI Junlong , LI Peipei , HU Xuegang , et al. Learning common and label-specific features for multi-label classification with correlation information[J]. Pattern Recognition, 2022, 121, 108- 259.
16 LI Jundong , CHENG Kewei , WANG Suhang , et al. Feature selection: a data perspective[J]. ACM Computing Surveys (CSUR), 2017, 50 (6): 1- 45.
17 刘浩阳, 林耀进, 刘景华, 等. 由粗到细的分层特征选择[J]. 电子学报, 2022, 50 (11): 2778- 2789.
LIU Haoyang , LIN Yaojin , LIU Jinghua , et al. Hierarchical feature selection from coarse to fine[J]. Acta Electronica Sinica, 2022, 50 (11): 2778- 2789.
18 LIN Zhouchen , GANESH A , WRIGHT J , et al. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix[J]. Computational Advances, 2009, 10, 1- 18.
19 DEKEL O, KESHET J, SINGER Y. Large margin hierarchical classification[C]//Proceedings of the Twenty-first International Conference on Machine Learning. New York: ACM, 2004: 1-8.
20 SILLA C N , FREITAS A A . A survey of hierarchical classification across different application domains[J]. Data Mining & Knowledge Discovery, 2011, 22 (1/2): 31- 72.
21 NIE Feiping, HUANG Heng, CAI Xiao, et al. Efficient and robust feature selection via joint ℓ2, 1-norms minimization[C]//Proceedings of the 23rd International Conference on Neural Information Processing Systems. Kyoto: IEEE, 2010: 1813-1821.
22 PENG Hanchuan , LONG Fuhui , DING C . Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27 (8): 1226- 1238.
doi: 10.1109/TPAMI.2005.159
23 DEMIAR J , SCHUURMAMS D . Statistical comparisons of classifiers over multiple data sets[J]. Journal of Machine Learning Research, 2006, 7 (1): 1- 30.
[1] ZHANG Zhi-hao, LIN Yao-jin, LU Shun, WU Yi-lin, WANG Chen-xi. Multi-label feature selection with streaming and missing labels [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(8): 39-52.
[2] SUN Lin, CHEN Yu-sheng, XU Jiu-cheng. Multilabel feature selection algorithm based on improved ReliefF [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(4): 1-11.
[3] SUN Lin, LIANG Na, XU Jiu-cheng. Feature selection using adaptive neighborhood mutual information and spectral clustering [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2022, 57(12): 13-24.
[4] ZHANG Yao, MA Ying-cang, YAND Xiao-fei, ZHU Heng-dong, YANG Ting. Multi-label feature selection based on manifold structure and flexible embedding [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2021, 56(7): 91-102.
[5] HUANG Tian-yi, ZHU William. Cost-sensitive feature selection via manifold learning [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(3): 91-96.
[6] WAN Zhong-ying, WANG Ming-wen, ZUO Jia-li, WAN Jian-yi. Feature selection combined with the global and local information(GLFS) [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(5): 87-93.
[7] LI Zhao,SUN Zhan-,LI Xiao,LI Cheng,. Study on feature selection method based on information loss [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(11): 7-12.
[8] ZHENG Yan, PANG Lin, BI Hui, LIU Wei, CHENG Gong. Feature selection algorithm based on sentiment topic model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 74-81.
[9] XIA Meng-nan, DU Yong-ping, ZUO Ben-xin. Micro-blog opinion analysis based on syntactic dependency and feature combination [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 22-30.
[10] PAN Qing-qing, ZHOU Feng, YU Zheng-tao, GUO Jian-yi, XIAN Yan-tuan. Recognition method of Vietnamese named entity based on#br# conditional random fields [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(1): 76-79.
[11] YU Ran 1,2, LIU Chun-yang3*, JIN Xiao-long 1, WANG Yuan-zhuo 1, CHENG Xue-qi 1. Chinese spam microblog filtering based on the fusion of
multi-angle features [J]. J4, 2013, 48(11): 53-58.
[12] YI Chao-qun, LI Jian-ping, ZHU Cheng-wen. A kind of feature selection based on classification accuracy of SVM [J]. J4, 2010, 45(7): 119-121.
[13] YANG Yu-Zhen, LIU Pei-Yu, SHU Zhen-Fang, QIU Ye. Research of an improved information gain methodusing distribution information of terms [J]. J4, 2009, 44(11): 48-51.
[14] YUAN Xiao-hang,DU Xiao-yong . iRIPPER: an improved rule-based text categorization algorithm [J]. J4, 2007, 42(11): 66-68 .
[15] YU Jun-ying,WANG Ming-wen,SHENG Jun . Class information feature selection method for text classification [J]. J4, 2006, 41(3): 144-148 .
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] GUO Qiao-jin, DING Yi, LI Ning. A context based method for ROI detection in digitized mammograms[J]. J4, 2010, 45(7): 70 -75 .
[2] FU Hai-yan,LU Chang-jing,SHI Kai-quan . (F,F-)-law inference and law mining[J]. J4, 2007, 42(7): 54 -57 .
[3] HU Ming-Di, SHE Yan-Hong, WANG Min. Topological properties of  three-valued   logic  metric space[J]. J4, 2010, 45(6): 86 -90 .
[4] ZHANG Hui . Generalized recursive preference in incomplete information[J]. J4, 2006, 41(1): 62 -68 .
[5] LIU Hong-hua . The alternating group iterative method for the dispersive equation[J]. J4, 2007, 42(1): 19 -23 .
[6] LIU Kun-lun. Application of variable structure pair copula model in the analysis of financial contagion[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(6): 104 -110 .
[7] TANG Xiao-hong1, HU Wen-xiao2*, WEI Yan-feng2, JIANG Xi-long2, ZHANG Jing-ying2, SHAO Xue-dong3. Screening and biological characteristics studies of wide wine-making yeasts[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(03): 12 -17 .
[8] YUAN Rui-qiang,LIU Guan-qun,ZHANG Xian-liang,GAO Hui-wang . Features of hydrogen and oxygen isotopes in groundwater ofthe shallow part of Yellow River Delta[J]. J4, 2006, 41(5): 138 -143 .
[9] GUO Wen-juan, YANG Gong-ping*, DONG Jin-li. A review of fingerprint image segmentation methods[J]. J4, 2010, 45(7): 94 -101 .
[10] ZHANG Wen, ZHANG Hua-xiang*, LI Ming-fang, JI Hua. A decision tree construction approach: two-step forward is better than one[J]. J4, 2010, 45(7): 114 -118 .
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd