Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM

Xuebo Jin; Tianxiao Sun; Wei Chen; Huijun Ma; Yeqing Wang; Yusen Zheng

doi:10.62762/TIS.2024.137329

Article Information

Published in ICCK Transactions on Intelligent Systematics

Volume/Issue Volume 1, Issue 1, 2024

Pages 40-48

Cited by 18 (Web of Science) 18 (Scopus) 20 (Google Scholar)

Abstract

Nowadays, state estimation is widely used in fields such as autonomous driving and drone navigation. However, in practical applications, it is difficult to obtain accurate target motion models and noise covariance.This leads to a decrease in the estimation accuracy of traditional Kalman filters. To address this issue, this paper proposes an adaptive model free state estimation method based on attention parameter learning module. This method combines Transformer's encoder with Long Short Term Memory Network (LSTM), and obtains the system's operational characteristics through offline learning of measurement data without modeling the system dynamics and measurement characteristics. In addition, based on the output of the attention learning module, the expectation maximization (EM) algorithm is used to estimate the system model parameters online, and a Kalman filter is used to obtain state estimation. This paper was validated using the GPS trajectory path dataset, and the experimental results showed that the proposed parameter adaptive model free state estimation method has better estimation accuracy than other models, providing an effective method for using deep learning networks for state estimation.

Graphical Abstract

Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM

Keywords

state estimation kalman filter transformer LSTM

Data Availability Statement

Data will be made available on request.

Funding

This work was supported without any funding.

Conflicts of Interest

The authors declare no conflicts of interest.

Ethical Approval and Consent to Participate

Not applicable.

References

Jason, S., & Marilyn, Wolf. (2015). Tailoring design for embedded computer vision applications. Computer, 48(5), 58-62.
[CrossRef] [Google Scholar]
Freitas, E. J. D. R., Guimarães, M. P., Ramos, J. V., & Júnior, C. D. D. S. (2023, October). A Robotics Club in High School: an experience report. In 2023 Latin American Robotics Symposium (LARS), 2023 Brazilian Symposium on Robotics (SBR), and 2023 Workshop on Robotics in Education (WRE) (pp. 683-688). IEEE.
[CrossRef] [Google Scholar]
Okuyama, T., Gonsalves, T., & Upadhay, J. (2018, March). Autonomous driving system based on deep q learnig. In 2018 International conference on intelligent autonomous systems (ICoIAS) (pp. 201-205). IEEE.
[CrossRef] [Google Scholar]
Kalman, R. (1960). A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82D, 35-45.
[CrossRef] [Google Scholar]
Kalman, R. E., & Bucy, R. S. (1961). New results in linear filtering and prediction theory.
[CrossRef] [Google Scholar]
Xiong, K., Chan, C. W., & Zhang, H. Y. (2005). Unscented Kalman filter for fault detection. IFAC Proceedings Volumes, 38(1), 113-118.
[CrossRef] [Google Scholar]
Arasaratnam, I., & Haykin, S. (2011). Cubature kalman smoothers. Automatica, 47(10), 2245-2250.
[CrossRef] [Google Scholar]
Ghahramani, Z., & Hinton, G. (1996). Parameter Estimation for Linear Dynamical Systems.
[Google Scholar]
Yunita, A., Pratama, M. I., Almuzakki, M. Z., Ramadhan, H., Akhir, E. A. P., Mansur, A. B. F., & Basori, A. H. (2025). Performance analysis of neural network architectures for time series forecasting: A comparative study of RNN, LSTM, GRU, and hybrid models. MethodsX, 15, 103462.
[CrossRef] [Google Scholar]
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
[Google Scholar]
Wang, H., & Tu, M. (2020, December). Enhancing attention models via multi-head collaboration. In 2020 International Conference on Asian Language Processing (IALP) (pp. 19-23). IEEE.
[CrossRef] [Google Scholar]
Wang, J., Zhang, W., & Yang, H. (2021). Visual Analytics for RNN-Based Deep Reinforcement Learning. IEEE Transactions on Visualization and Computer Graphics, 28(12), 4141-4155.
[CrossRef] [Google Scholar]
Zhou, X., Shi, J., Gong, K., Zhu, C., Hua, J., & Xu, J. (2021). A Novel Quench Detection Method Based on CNN-LSTM Model. IEEE Transactions on Applied Superconductivity, 31(5), 1-5.
[CrossRef] [Google Scholar]
Huang, H., Zeng, Z., Yao, D., Pei, X., & Zhang, Y. (2021). Spatial-temporal ConvLSTM for vehicle driving intention prediction. Tsinghua Science and Technology, 27(3), 599-609.
[CrossRef] [Google Scholar]
Jin, X. B., Gong, W. T., Kong, J. L., Bai, Y. T., & Su, T. L. (2022). PFVAE: a planar flow-based variational auto-encoder prediction model for time series data. Mathematics, 10(4), 610.
[CrossRef] [Google Scholar]

Cite This Article

APA Style

Jin, X., Sun, T., Chen, W., Ma, H., Wang, Y., & Zheng, Y. (2024). Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM. ICCK Transactions on Intelligent Systematics, 1(1), 40–48. https://doi.org/10.62762/TIS.2024.137329

Export Citation

RIS Format

Compatible with EndNote, Zotero, Mendeley, and other reference managers

TY  - JOUR
AU  - Jin, Xuebo
AU  - Sun, Tianxiao
AU  - Chen, Wei
AU  - Ma, Huijun
AU  - Wang, Yeqing
AU  - Zheng, Yusen
PY  - 2024
DA  - 2024/05/29
TI  - Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM
JO  - ICCK Transactions on Intelligent Systematics
T2  - ICCK Transactions on Intelligent Systematics
JF  - ICCK Transactions on Intelligent Systematics
VL  - 1
IS  - 1
SP  - 40
EP  - 48
DO  - 10.62762/TIS.2024.137329
UR  - https://www.icck.org/article/abs/TIS.2024.137329
KW  - state estimation
KW  - kalman filter
KW  - transformer
KW  - LSTM
AB  - Nowadays, state estimation is widely used in fields such as autonomous driving and drone navigation. However, in practical applications, it is difficult to obtain accurate target motion models and noise covariance.This leads to a decrease in the estimation accuracy of traditional Kalman filters. To address this issue, this paper proposes an adaptive model free state estimation method based on attention parameter learning module. This method combines Transformer's encoder with Long Short Term Memory Network (LSTM), and obtains the system's operational characteristics through offline learning of measurement data without modeling the system dynamics and measurement characteristics. In addition, based on the output of the attention learning module, the expectation maximization (EM) algorithm is used to estimate the system model parameters online, and a Kalman filter is used to obtain state estimation. This paper was validated using the GPS trajectory path dataset, and the experimental results showed that the proposed parameter adaptive model free state estimation method has better estimation accuracy than other models, providing an effective method for using deep learning networks for state estimation.
SN  - 3068-5079
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -

BibTeX Format

Compatible with LaTeX, BibTeX, and other reference managers

@article{Jin2024Parameter,
  author = {Xuebo Jin and Tianxiao Sun and Wei Chen and Huijun Ma and Yeqing Wang and Yusen Zheng},
  title = {Parameter Adaptive Non-Model-Based State Estimation Combining Attention Mechanism and LSTM},
  journal = {ICCK Transactions on Intelligent Systematics},
  year = {2024},
  volume = {1},
  number = {1},
  pages = {40-48},
  doi = {10.62762/TIS.2024.137329},
  url = {https://www.icck.org/article/abs/TIS.2024.137329},
  abstract = {Nowadays, state estimation is widely used in fields such as autonomous driving and drone navigation. However, in practical applications, it is difficult to obtain accurate target motion models and noise covariance.This leads to a decrease in the estimation accuracy of traditional Kalman filters. To address this issue, this paper proposes an adaptive model free state estimation method based on attention parameter learning module. This method combines Transformer's encoder with Long Short Term Memory Network (LSTM), and obtains the system's operational characteristics through offline learning of measurement data without modeling the system dynamics and measurement characteristics. In addition, based on the output of the attention learning module, the expectation maximization (EM) algorithm is used to estimate the system model parameters online, and a Kalman filter is used to obtain state estimation. This paper was validated using the GPS trajectory path dataset, and the experimental results showed that the proposed parameter adaptive model free state estimation method has better estimation accuracy than other models, providing an effective method for using deep learning networks for state estimation.},
  keywords = {state estimation, kalman filter, transformer, LSTM},
  issn = {3068-5079},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations

Google Scholar

20

Crossref

19

Scopus

18

Web of Science

18

Views

2972

PDF Downloads

560

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

Institute of Central Computation and Knowledge (ICCK) or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

ICCK Transactions on Intelligent Systematics

ISSN: 3068-5079 (Online) | ISSN: 3069-003X (Print)

[email protected]

Preserved at
Portico

User

Unlimited Downloads

Complete Library Access

Membership Eligibility

Community Leadership Opportunities