Efficient Lead-lag Cross(X) Attention Time Series Transformer
Multivariate time series forecasting is essential in fields like finance and energy management, yet it poses significant challenges to capture inter-variate relationships and temporal dynamics. However, existing models struggle to reflect dynamic relationships between variates and temporal lead-lag patterns simultaneously, due to the computational complexity.
This study proposes ElxaTST, a novel framework that incorporates the Efficient Lead-lag Cross(X) Attention. Elxa identifies the most relevant variates for each target variate based on similarity and samples temporal patches from these variates, enabling efficient modeling of inter-variate interactions and lead-lag temporal relationships. This approach reduces computational overhead while effectively capturing complex temporal dependencies, significantly outperforming existing baseline models in long-term forecasting benchmarks.
$ conda create -n tslib python=3.8
$ conda activate tslib
$ pip3 install torch==1.8.2 --extra-index-url https://download.pytorch.org/whl/lts/1.8/cu111
$ pip install -e .
$ ./scripts/long_term_forecast/ETT_script/ElxaTST_ETTh1_96_96.sh
$ ./scripts/long_term_forecast/ETT_script/ElxaTST_ETTh2_96_96.sh
$ ./scripts/long_term_forecast/ETT_script/ElxaTST_ETTm1_96_96.sh
$ ./scripts/long_term_forecast/ETT_script/ElxaTST_ETTm2_96_96.sh
$ ./scripts/long_term_forecast/Weather_script/ElxaTST_Weather_96_96.sh
Note
Due to the use of a function, such as F.grid_sample, which exhibits nondeterministic behavior in its backward pass, it may not be possible to reproduce the exact results reported in the paper.
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2023R1A2C200337911 and No. RS-2023-00220762).