关键词: Transformer, 霜冰优化算法, 多尺度特征编码, 可学习位置编码, 注意力池化

Abstract: To address the shortcomings of the traditional Transformer in long-term sequence modeling and computational efficiency, an improved Transformer model is proposed. This model first introduces a multi-scale convolutional structure in the feature extraction stage. Parallel convolution kernels capture both short-term fluctuations and long-term trends at different scales, thereby enhancing the representation of multi-level temporal patterns. Subsequently, the model employs learnable positional encoding instead of fixed sinusoidal encoding to better address the challenges posed by non-stationary data and irregular time intervals. During global dependency modeling, the improved encoder leverages a multi-head self-attention mechanism to establish feature interactions across time steps and dynamically assign moment weights to focus on key segments, effectively reducing the computational complexity of long-term sequence modeling. Furthermore, the model incorporates the rime optimization algorithm RIME for efficient search and optimization in a high-dimensional hyperparameter space, thereby improving the models convergence speed and generalization ability. Experiments on three real-world complex datasets demonstrate that the RIME-Transformer outperforms mainstream methods across multiple metrics. These results validate the effectiveness and superiority of the proposed model for complex time series prediction tasks.

Key words: Transformer, rime optimization algorithm, multi-scale feature encoding, learnable positional encoding, attention pooling