JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2025, Vol. 60 ›› Issue (7): 1-12.doi: 10.6040/j.issn.1671-9352.4.2024.126

   

EEG-MFNet: a lightweight multi-branch fusion network for electroencephalogram signal analysis

YE Xiaoya1,2,3, WANG Xiuqing1,2,3*, MA Haibin1,2,3, ZHANG Nuofei1,2,3   

  1. 1. College of Computer and Cyber Security Hebei Normal University, Shijiazhuang 050024, Hebei, China;
    2. Hebei Provincial Key Laboratory of Network &
    Information Security(Hebei Normal University), Shijiazhuang 050024, Hebei, China;
    3. Hebei Provincial Engineering Research Center for Supply China Big Data Analytics &
    Data Security(Hebei Normal University), Shijiazhuang 050024, Hebei, China
  • Published:2025-07-01

PDF (PC)

153

Abstract: In order to solve the problems of decoding efficiency caused by low resolution, insufficient data volume and individual differences of subjects, multi-branch fusion network for electroencephalogram signal(EEG-MFNet)model suitable for EEG signal analysis is proposed. Multi-level spatiotemporal features of EEG data are extracted through multi-scale spatiotemporal convolutional modules, and further applied by multi-scale temporal convolution to extract more advanced time-space-frequency domain features. Applying a sliding window to the feature data of the input classifier significantly enhances the effective features of the data. The average classification accuracy and standard deviation of the EEG-MFNet model are improved by more than 3.19% and 22.86% compared with the comparison model, respectively. Model inference time is reduced by more than 16.87%. The experimental results show that the proposed method improves the stability of the model and significantly improves the training efficiency. This work provides a more efficient decoding scheme for EEG signal analysis based on motor imagery.

Key words: multi-scale convolution, slide window, motor imagery, brain-computer interface, electroencephalogram signal

CLC Number: 

  • TP391
[1] JIAO Runhai, LI Chengyang, XUN Gangyi, et al. A context-aware multi-event identification method for nonintrusive load monitoring[J]. IEEE Transactions on Consumer Electronics, 2023, 69(2):194-204.
[2] ZHANG Yu, LI Penghai, CHENG Longlong, et al. Attention-based multiscale spatial-temporal convolutional network for motor imagery EEG decoding[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1):2423-2434.
[3] AHMED I, JEON G, PICCIALLI F. From artificial intelligence to explainable artificial intelligence in industry 4.0: a survey on what, how, and where[J]. IEEE Transactions on Industrial Informatics, 2022, 18(8):5031-5042.
[4] WILLIAMS S C, HORSFALL H L, FUNNELL J P, et al. Neurosurgical team acceptability of brain-computer interfaces: a two-stage international cross-sectional survey[J]. World Neurosurgery, 2022, 164:884-898.
[5] LI Hongli, LIU Haoyu, CHEN Hongyu, et al. Multi-scale feature extraction and classification of motor imagery electroencephalography based on time series data enhancement[J]. Journal of Biomedical Engineering, 2023, 40(3):418-425.
[6] CHUNDURI V, AOUNDNI Y, KHAN S, et al. Multi-scale spatiotemporal attention network for neuron based motor imagery EEG classification[J]. Journal of Neuroscience Methods, 2024, 406:110-128.
[7] GAO Dongrui, YANG Wen, LI Pengrui, et al. A multiscale feature fusion network based on attention mechanism for motor imagery EEG decoding[J]. Applied Soft Computing, 2024, 151:111-129.
[8] LI Hongli, CHEN Hongyu, JIA Ziyu, et al. A parallel multi-scale time-frequency block convolutional neural network based on channel attention module for motor imagery classification[J]. Biomedical Signal Processing and Control, 2023, 79:104066.
[9] LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of neural engineering, 2018, 15(5):056013.
[10] ZHANG Yu, WANG Yu, ZHOU Guoxu, et al. Multi-kernel extreme learning machine for EEG classification in brain-computer interfaces[J]. Expert Systems with Applications, 2018, 96:302-310.
[11] ANG K K, CHIN Z Y, ZHANG H, et al. Filter bank common spatial pattern(FBCSP)in brain-computer interface[C] //2008 IEEE International Joint Conference on Neural Networks, Hong Kong: IEEE, 2390-2397.
[12] 吕璐,程虎,朱鸿泰,等. 基于深度学习的目标检测研究与应用综述[J]. 电子与封装,2022,22(1):72-80. LV Lu, CHENG Hu, ZHU Hongtai, et al. Review of research and application of target detection based on deep learning[J]. Electrons and Packaging, 2022, 22(1):72-80.
[13] INGOLFSSON T M, HERSCHE M, WANG X, et al. EEG-TCNet: an accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces[C] //2020 IEEE International Conference on Systems, Man, and Cybernetics(SMC). Toronto: IEEE, 2020:2958-2965.
[14] ALTAHERI H, MUHAMMAD G, ALSULAIMAN M. Dynamic convolution with multilevel attention for EEG-based motor imagery decoding[J]. IEEE Internet of Things Journal, 2023, 10(21):18579-18588.
[15] ALTUWAIJRI G, MUHAMMAD G. A multibranch of convolutional neural network models for electroencephalogram-based motor imagery classification[J]. Biosensors, 2022, 12(1):22.
[16] 王蒙昊,方慧娟,龚亨翔,等. 应用多尺度混合卷积网络的脑电信号特征提取与识别[J]. 华侨大学学报(自然科学版),2023,44(5):628-635. WANG Menghao, FANG Huijuan, GONG Hengxiang, et al. Feature extraction and identification of EEG signals using multi-scale hybrid convolutional networks[J]. Journal of Huaqiao University(Natural Science Edition), 2023, 44(5):628-635.
[17] TANG Xianlun, YANG Caiquan, SUN Xia, et al. Motor imagery EEG decoding based on multi-scale hybrid networks and feature enhancement[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31:1208-1218.
[18] GUENNEC A L, MALINOWSKI S, TAVENARD R. Data augmentation for time series classification using convolutional neural networks[C] //European Conference on Principles of Data Mining and Knowledge Discovery. Riva del Garda, Italy: IEEE, 2016.
[19] KIM S K, KIRCHNER E A, STEFES A, et al. Intrinsic interactive reinforcement learning-using error-related potentials for real world human-robot interaction[J]. Scientific Reports, 2017, 7(1):1-16.
[20] LOTTE F. Signal processing approaches to minimize or suppress calibration time in oscillatory activity-based brain-computer interfaces[J]. Proceedings of the IEEE, 2015, 103(6):871-890.
[21] WANG Fang, ZHONG Shenhua, PENG Jianfen, et al. Data augmentation for EEG-based emotion recognition with deep convolutional neural networks[C] //MultiMedia Modeling: 24th International Conference, MMM 2018. Tokyo: Springer, 2018.
[22] LI Youjun, HUANG Jiajin, ZHOU Haiyan, et al. Human emotion recognition with electroencephalographic multidimensional features by hybrid deep neural networks[J]. Applied Sciences, 2017, 7(10):1060.
[23] BRUNNER C, LEEB R, MULLER-PUTZ G et al. BCI competition 2008-Graz data set A[DB/OL].(2008-07-03)[2014-01-17]. https://ieee-dataport.org/documents/bci-competition-2008.
[24] ZHANG C, KIM Y K, ESKANDARIAN A. EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification[J]. Journal of Neural Engineering, 2021, 18(4):046014.
[25] SALAMI A, ANDREU-PEREZ J, GILLMEISTER H. EEG-ITNet: an explainable inception temporal convolutional network for motor imagery classification[J]. IEEE Access, 2022, 10:36672-36685.
[26] 刘近贞,叶方方,熊慧. 基于卷积神经网络的多类运动想象脑电信号识别[J]. 浙江大学学报(工学版),2021,55(11):2054-2066. LIU Jinzhen, YE Fangfang, XIONG Hui. Multi-class motor imagery EEG signal recognition based on convolutional neural networks[J]. Journal of Zhejiang University(Engineering Edition), 2021, 55(11):2054-2066.
[27] KIRANYAZ S, AVCI O, ABDELJABER O, et al. 1D convolutional neural networks and applications: a survey[J]. Mechanical Systems and Signal Processing, 2021, 151:107398.
[28] SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[C] //2017 IEEE Signal Processing in Medicine and Biology Symposium(SPMB). Los Angeles: IEEE, 2017:1-7.
[29] YANG Jun, MA Zhengmin, SHEN Tao. Multi-time and multi-band CSP motor imagery EEG feature classification algorithm [J]. Applied Sciences, 2021, 11(21):10294.
[30] ALTUWAIJRI G A, MUHAMMAD G. Electroencephalogram-based motor imagery signals classification using a multi-branch convolutional neural network model with attention blocks[J]. Bioengineering(Basel), 2022, 9(7):323.
[31] MA Weifeng, XUE Haojie, SUN Xiaoyong, et al. A novel multi-branch hybrid neural network for motor imagery EEG signal classification [J]. Biomedical Signal Processing and Control, 2022, 77:103718.
[1] Xia LIANG,Jie GUO. A method of online teaching platform selection based on online reviews [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(9): 108-118.
[2] Chao LI,Wei LIAO. Chinese disease text classification model driven by medical knowledge [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 122-130.
[3] Jie JI,Chengjie SUN,Lili SHAN,Boyue SHANG,Lei LIN. A prompt learning approach for telecom network fraud case classification [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 113-121.
[4] Qi LUO,Gang GOU. Multimodal conversation emotion recognition based on clustering and group normalization [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 105-112.
[5] Fengxu ZHAO,Jian WANG,Yuan LIN,Hongfei LIN. Probability distribution optimization model for learning to rank [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 95-104.
[6] Xingyu HUANG,Mingyu ZHAO,Ziyu LYU. Category-wise knowledge probers for representation learning of graph neural networks [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 85-94.
[7] Liang GUI,Yao XU,Shizhu HE,Yuanzhe ZHANG,Kang LIU,Jun ZHAO. Factual error detection in knowledge graphs based on dynamic neighbor selection [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 76-84.
[8] Ning XIAN,Yixing FAN,Tao LIAN,Jiafeng GUO. Noise network alignment method integrating multiple features [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 64-75.
[9] Chengjie SUN,Zongwei LI,Lili SHAN,Lei LIN. A document-level event extraction method based on core arguments [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 53-63.
[10] Peiyu LIU,Bowen YAO,Zefeng GAO,Wayne Xin ZHAO. Matrix product operator based sequential recommendation model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 44-52, 104.
[11] Wei SHAO,Gaoyu ZHU,Lei YU,Jiafeng GUO. Dimensionality reduction and retrieval algorithms for high dimensional data [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 27-43.
[12] Jiyuan YANG,Muyang MA,Pengjie REN,Zhumin CHEN,Zhaochun REN,Xin XIN,Fei CAI,Jun MA. Research on self-supervised pre-training for recommender systems [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(7): 1-26.
[13] Haisu CHEN,Jiachun LIAO,Sicheng YAO. Identification and statistical analysis methods of personal information disclosure in open government data [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(3): 95-106.
[14] Xin WEN,Deyu LI. The ML-KNN method based on attribute weighting [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(3): 107-117.
[15] Xueqiang ZENG,Yu SUN,Ye LIU,Zhongying WAN,Jiali ZUO,Mingwen WANG. Emoji embedded representation based on emotion distribution [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2024, 59(3): 81-94.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd