Chinese Journal of Information Fusion Home About Editors Current Issue
Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems
Research Article  ·  Published: 27 May 2024
Issue cover
Chinese Journal of Information Fusion
Volume 1, Issue 1, 2024: 16-32
Research Article Open Access

Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems

by
Gongjian Zhou Gongjian Zhou 1 * ORCID
Shizhe Bu Shizhe Bu 1 ORCID
Thiagalingam Kirubarajan Thiagalingam Kirubarajan 2 ORCID
1 School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China
2 Department of Electrical and Computer Engineering, McMaster University, Ontario, Canada
* Corresponding Author: Gongjian Zhou, [email protected]
Volume 1, Issue 1
Volume 1, Issue 1 2024
Received: 17 February 2024 Accepted: 23 May 2024 Published: 27 May 2024 CrossMark
Download PDF 2.57 MB HTML XML Cite Metrics
DOI: 10.62762/CJIF.2024.361881
ARK: ark:/57805/cjif.2024.361881

Article Information

Published in Chinese Journal of Information Fusion
Volume/Issue Volume 1, Issue 1, 2024
Pages 16-32
Cited by 12 (Crossref) 9 (Scopus)

Abstract

Bias estimation of sensors is an essential prerequisite for accurate data fusion. Neglect of temporal bias in general real systems prevents the existing algorithms from successful application. In this paper, both spatial and temporal biases in asynchronous multisensor systems are investigated and two novel methods for simultaneous spatiotemporal bias compensation and data fusion are presented. The general situation that the sensors sample at different times with different and varying periods is explored, and unknown time delays may exist between the time stamps and the true measurement times. Due to the time delays, the time stamp interval of the measurements from different sensors may be different from their true measurement interval, and the unknown difference between them is considered as the temporal bias and augmented into the state vector to be estimated. Multisensor measurements are collected in batch processing or sequential processing schemes to estimate the augmented state vector, results in two spatiotemporal bias compensation methods. In both processing schemes, the measurements are formulated as functions of both target states and spatiotemporal biases according to the time difference between the measurements and the states to be estimated. The Unscented Kalman Filter is used to handle the nonlinearity of the measurements and produce spatiotemporal bias and target state estimates simultaneously. The posterior Cramer-Rao lower bound (PCRLB) for spatiotemporal bias and state estimation is presented and simulations are conducted to demonstrate the effectiveness of the proposed methods.

Graphical Abstract

Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems

Keywords

spatiotemporal bias state estimation multisensor data fusion asynchronous sensors

Data Availability Statement

Data will be made available on request.

Funding

This work was supported by the National Natural Science Foundation of China under Grant 61671181.

Conflicts of Interest

The authors declare no conflicts of interest.

Ethical Approval and Consent to Participate

Not applicable.

References

  1. Bar-Shalom, Y., Willett, P. K., & Tian, X. (2011). Tracking and data fusion (Vol. 11). Storrs, CT, USA:: YBS publishing.
    [Google Scholar]
  2. Bar-Shalom, Y., Li, X. R., & Kirubarajan, T. (2004). Estimation with applications to tracking and navigation: theory algorithms and software. John Wiley & Sons.
    [Google Scholar]
  3. Ge, Q., Shao, T., Yang, Q., Shen, X., & Wen, C. (2016). Multisensor nonlinear fusion methods based on adaptive ensemble fifth-degree iterated cubature information filter for biomechatronics. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 46(7), 912-925.
    [CrossRef] [Google Scholar]
  4. Khaleghi, B., Khamis, A., Karray, F. O., & Razavi, S. N. (2013). Multisensor data fusion: A review of the state-of-the-art. Information Fusion, 14(1), 28-44.
    [CrossRef] [Google Scholar]
  5. Chen, B., Hu, G., Ho, D. W., Zhang, W. A., & Yu, L. (2016). Distributed robust fusion estimation with application to state monitoring systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(11), 2994-3005.
    [CrossRef] [Google Scholar]
  6. Yu, D., Xia, Y., Li, L., Xing, Z., & Zhu, C. (2019). Distributed covariance intersection fusion estimation with delayed measurements and unknown inputs. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(8), 5165-5173.
    [CrossRef] [Google Scholar]
  7. Lin, X., Pan, X., Sun, W., Liu, L., & Chen, X. (2022). Multi-scale asynchronous fusion algorithm for multi-sensor integrated navigation system. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 236(9), 1709-1723.
    [CrossRef] [Google Scholar]
  8. Lin, H., & Sun, S. (2017). Distributed fusion estimator for multisensor multirate systems with correlated noises. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 48(7), 1131-1139.
    [CrossRef] [Google Scholar]
  9. Burke, J. (1966). The SAGE real quality control fraction and its interface with BUIC II/BUIC III. MITRE Corporation, Technical report 308.
    [Google Scholar]
  10. Leung, H., Blanchette, M. & Harrison, C. (1994). A least squares fusion of multiple radar data. In Proceedings of Radar 94, Paris, France, (pp. 498–508.)
    [Google Scholar]
  11. Zhou, Y., Leung, H., & Blanchette, M. (1999). Sensor alignment with earth-centered earth-fixed (ECEF) coordinate system. IEEE Transactions on Aerospace and Electronic systems, 35(2), 410-418.
    [CrossRef] [Google Scholar]
  12. Zheng, Z. W., & Zhu, Y. S. (2004). New least squares registration algorithm for data fusion. IEEE Transactions on Aerospace and Electronic Systems, 40(4), 1410-1416.
    [CrossRef] [Google Scholar]
  13. Zhou, Y., Leung, H., & Yip, P. C. (1997). An exact maximum likelihood registration algorithm for data fusion. IEEE Transactions on Signal Processing, 45(6), 1560-1573.
    [CrossRef] [Google Scholar]
  14. Okello, N., & Ristic, B. (2003). Maximum likelihood registration for multiple dissimilar sensors. IEEE Transactions on Aerospace and Electronic Systems, 39(3), 1074-1083.
    [CrossRef] [Google Scholar]
  15. Rafati, A., Moshiri, B., & Rezaei, J. (2007, July). A new algorithm for general asynchronous sensor bias estimation in multisensor-multitarget systems. In 2007 10th International Conference on Information Fusion (pp. 1-8). IEEE.
    [CrossRef] [Google Scholar]
  16. Helmick, R. E., & Rice, T. R. (1993). Removal of alignment errors in an integrated system of two 3-D sensors. IEEE Transactions on Aerospace and Electronic systems, 29(4), 1333-1343.
    [CrossRef] [Google Scholar]
  17. Nabaa, N., & Bishop, R. H. (1999). Solution to a multisensor tracking problem with sensor registration errors. IEEE Transactions on Aerospace and Electronic systems, 35(1), 354-363.
    [CrossRef] [Google Scholar]
  18. Song, Q. He, Y. & Yang, J. (2007). An augmented state target tracking algorithm with systematic errors based on the unscented Kalman filter. Journal of Projectiles, Rockets, Missiles and Guidance, no. 3, (pp. 311–316).
    [Google Scholar]
  19. Liu, J., Zuo, Y., & Xue, A. (2013, July). An ASEKF algorithm for 2D and 3D radar registration. In Proceedings of the 32nd Chinese Control Conference (pp. 4758-4761). IEEE.https://ieeexplore.ieee.org/abstract/document/6640261
    [Google Scholar]
  20. Furgale, P., Rehder, J., & Siegwart, R. (2013, November). Unified temporal and spatial calibration for multi-sensor systems. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1280-1286). IEEE.
    [CrossRef] [Google Scholar]
  21. Rehder, J., Beardsley, P., Siegwart, R., & Furgale, P. (2014, September). Spatio-temporal laser to visual/inertial calibration with applications to hand-held, large scale scanning. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 459-465). IEEE.
    [CrossRef] [Google Scholar]
  22. Rehder, J., Siegwart, R., & Furgale, P. (2016). A general approach to spatiotemporal calibration in multisensor systems. IEEE Transactions on Robotics, 32(2), 383-398.
    [CrossRef] [Google Scholar]
  23. Nocedal, J., & Wright, S. J. (Eds.). (1999). Numerical optimization. New York, NY: Springer New York.
    [Google Scholar]
  24. You, H., Hongwei, Z., & Xiaoming, T. (2013). Joint systematic error estimation algorithm for radar and automatic dependent surveillance broadcasting. IET Radar, Sonar & Navigation, 7(4), 361-370.
    [CrossRef] [Google Scholar]
  25. Li, S., Cheng, Y., Brown, D., Tharmarasa, R., Zhou, G., & Kirubarajan, T. (2019). Comprehensive time-offset estimation for multisensor target tracking. IEEE Transactions on Aerospace and Electronic Systems, 56(3), 2351-2373.
    [CrossRef] [Google Scholar]
  26. Li, M., & Mourikis, A. I. (2013, May). 3-D motion estimation and online temporal calibration for camera-IMU systems. In 2013 IEEE International Conference on Robotics and Automation (pp. 5709-5716). IEEE.
    [CrossRef] [Google Scholar]
  27. Li, W., & Leung, H. (2004). Simultaneous registration and fusion of multiple dissimilar sensors for cooperative driving. IEEE Transactions on Intelligent Transportation Systems, 5(2), 84-98.
    [CrossRef] [Google Scholar]
  28. Li, W., Leung, H., & Zhou, Y. (2004). Space-time registration of radar and ESM using unscented Kalman filter. IEEE Transactions on Aerospace and Electronic Systems, 40(3), 824-836.
    [CrossRef] [Google Scholar]
  29. Huang, D., & Leung, H. (2005). An expectation-maximization-based interacting multiple model approach for cooperative driving systems. IEEE Transactions on Intelligent Transportation Systems, 6(2), 206-228.
    [CrossRef] [Google Scholar]
  30. Bu, S., & Zhou, G.(2016). Spatiotemporal registration for multi-sensor fusion systems. In Proceedings of the International Conference on Artificial Intelligence and Computer Science, Guilin, China (pp. 333–339).
    [Google Scholar]
  31. Bu, S., Zhou, C., & Zhou, G. (2019). Simultaneous spatiotemporal bias and state estimation for asynchronous multi‐sensor system. The Journal of Engineering, 2019(19), 5824-5828.
    [CrossRef] [Google Scholar]
  32. Tichavsky, P., Muravchik, C. H., & Nehorai, A. (1998). Posterior Cramér-Rao bounds for discrete-time nonlinear filtering. IEEE Transactions on Signal Processing, 46(5), 1386-1396.
    [CrossRef] [Google Scholar]
  33. Gongjian, Z. H. O. U., Changjun, Y. U., Naigang, C. U. I., & Taifan, Q. U. A. N. (2012). A tracking filter in spherical coordinates enhanced by de-noising of converted Doppler measurements. Chinese Journal of Aeronautics, 25(5), 757-765.
    [CrossRef] [Google Scholar]
  34. Blair, W. D. (2008, June). Design of nearly constant velocity track filters for tracking maneuvering targets. In 2008 11th International Conference on Information Fusion (pp. 1-7). IEEE.
    [Google Scholar]
  35. You, H., Jianjuan, X., & Xin, G. (2016). Radar data processing with applications. John Wiley & Sons.
    [Google Scholar]
  36. Angrisani, L., D'Apuzzo, M., & Moriello, R. S. L. (2005, May). The unscented transform: a powerful tool for measurement uncertainty evaluation. In Proceedings of the 2005 IEEE International Workshop on Advanced Methods for Uncertainty Estimation in Measurement, 2005. (pp. 27-32). IEEE.
    [CrossRef] [Google Scholar]
  37. Challa, S. (2011). Fundamentals of object tracking. Cambridge University Press.
    [Google Scholar]
  38. Lerro, D., & Bar-Shalom, Y. (1993). Tracking with debiased consistent converted measurements versus EKF. IEEE Transactions on Aerospace and Electronic systems, 29(3), 1015-1022.
    [CrossRef] [Google Scholar]
  39. Longbin, M., Xiaoquan, S., Yiyu, Z., Kang, S. Z., & Bar-Shalom, Y. (1998). Unbiased converted measurements for tracking. IEEE Transactions on Aerospace and Electronic Systems, 34(3), 1023-1027.
    [CrossRef] [Google Scholar]
  40. Duan, Z., Han, C., & Li, X. R. (2004). Comments on" unbiased converted measurements for tracking". IEEE Transactions on Aerospace and Electronic systems, 40(4), 1374.
    [CrossRef] [Google Scholar]
  41. Van Trees, H. L. (2004). Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory. John Wiley & Sons.
    [Google Scholar]

Cited By (12)

  1. Zhengju Jia, Ming Hui, Jinshu Huang, Zufeng Fu, Tingxiao Pan, Jinghong Yao. FMAF-YOLO: fine-grained multi-modal adaptive fusion network for multispectral object detection. Measurement Science and Technology, 2026 , 37 (23).
    [CrossRef]
  2. Abby Scaletta, Jyoti Mandhani, Jonathan D. Regehr. A cordon count program to monitor pedestrian and bicycle exposure in an urban activity area: case study of a university campus in a winter city. Canadian Journal of Civil Engineering, 2025 , 52 (8).
    [CrossRef]
  3. Jie Lian, Choi Kyung Hwan, Yongqing Hu. Modified framework for sports event brand building evaluation with 2-tuple linguistic neutrosophic number group decision-making. Soft Computing, 2025 , 29 (11-12).
    [CrossRef]
  4. Jie Lian, Han Guo, Choi Kyung Hwan, Ying Meng. Enhanced fuzzy combined techniques for group decision-making with 2-tuple linguistic neutrosophic number and applications to sports event brand building evaluation. Scientific Reports, 2025 , 15 (1).
    [CrossRef]
  5. Jing Jiang, Sicheng Zhao, Jiankun Zhu, Wenbo Tang, Zhaopan Xu, Jidong Yang, Guoping Liu, Tengfei Xing, Pengfei Xu, Hongxun Yao. Multi-source domain adaptation for panoramic semantic segmentation. Information Fusion, 2025 , 117 .
    [CrossRef]
  6. Yong Wang, Xiaoming Dong, Jianfeng Zeng, Pengfei Du. Intelligent Driving Based on Multi-Objective Shifting Decision Method for Vehicle Transmission System. IEEE Transactions on Intelligent Transportation Systems, 2025 , 26 (10).
    [CrossRef]
  7. Jun Zhao. Virtualization resource scheduling and optimization method based on swarm intelligent systems. Intelligent Systems with Applications, 2025 , 25 .
    [CrossRef]
  8. Huiyu Yan, Jili Zhang, Liangdong Ma. A Time Registration Method for Asynchronous Building Electricity Consumption Data Based on the A-PCHIP-iKF Algorithm. Buildings, 2025 , 15 (22).
    [CrossRef]
  9. Hengrong Ju, Jing Guo, Weiping Ding, Xibei Yang. D3WC: Deep three-way clustering with granular evidence fusion. Information Fusion, 2025 , 114 .
    [CrossRef]
  10. Xin Xiong, Xiangyu Wang, Suorong Yang, Furao Shen, Jian Zhao. GMNI: Achieve good data augmentation in unsupervised graph contrastive learning. Neural Networks, 2025 , 181 .
    [CrossRef]
  11. Guanglong Bao, Beihe Weng, Huan Xv. Application of deep neural network identification models based on garden environment perception in plant protection. Crop Protection, 2024 , 184 .
    [CrossRef]
  12. Fei Zhu, Cuiping Shi, Liguo Wang, Kaijie Shi. A Greedy Strategy Guided Graph Self-Attention Network for Few-Shot Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 2024 , 62 .
    [CrossRef]
* Citation data provided by Crossref Cited-by.

Cite This Article

APA Style
Zhou, G., Bu, S., & Kirubarajan, T. (2024). Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems. Chinese Journal of Information Fusion, 1(1), 16–32. https://doi.org/10.62762/CJIF.2024.361881
Export Citation
RIS Format
Compatible with EndNote, Zotero, Mendeley, and other reference managers
TY  - JOUR
AU  - Zhou, Gongjian
AU  - Bu, Shizhe
AU  - Kirubarajan, Thiagalingam
PY  - 2024
DA  - 2024/05/27
TI  - Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems
JO  - Chinese Journal of Information Fusion
T2  - Chinese Journal of Information Fusion
JF  - Chinese Journal of Information Fusion
VL  - 1
IS  - 1
SP  - 16
EP  - 32
DO  - 10.62762/CJIF.2024.361881
UR  - https://www.icck.org/article/abs/CJIF.2024.361881
KW  - spatiotemporal bias
KW  - state estimation
KW  - multisensor data fusion
KW  - asynchronous sensors
AB  - Bias estimation of sensors is an essential prerequisite for accurate data fusion. Neglect of temporal bias in general real systems prevents the existing algorithms from successful application. In this paper, both spatial and temporal biases in asynchronous multisensor systems are investigated and two novel methods for simultaneous spatiotemporal bias compensation and data fusion are presented. The general situation that the sensors sample at different times with different and varying periods is explored, and unknown time delays may exist between the time stamps and the true measurement times. Due to the time delays, the time stamp interval of the measurements from different sensors may be different from their true measurement interval, and the unknown difference between them is considered as the temporal bias and augmented into the state vector to be estimated. Multisensor measurements are collected in batch processing or sequential processing schemes to estimate the augmented state vector, results in two spatiotemporal bias compensation methods. In both processing schemes, the measurements are formulated as functions of both target states and spatiotemporal biases according to the time difference between the measurements and the states to be estimated. The Unscented Kalman Filter is used to handle the nonlinearity of the measurements and produce spatiotemporal bias and target state estimates simultaneously. The posterior Cramer-Rao lower bound (PCRLB) for spatiotemporal bias and state estimation is presented and simulations are conducted to demonstrate the effectiveness of the proposed methods.
SN  - 2998-3371
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -
BibTeX Format
Compatible with LaTeX, BibTeX, and other reference managers
@article{Zhou2024Simultaneo,
  author = {Gongjian Zhou and Shizhe Bu and Thiagalingam Kirubarajan},
  title = {Simultaneous Spatiotemporal Bias Compensation and Data Fusion for Asynchronous Multisensor Systems},
  journal = {Chinese Journal of Information Fusion},
  year = {2024},
  volume = {1},
  number = {1},
  pages = {16-32},
  doi = {10.62762/CJIF.2024.361881},
  url = {https://www.icck.org/article/abs/CJIF.2024.361881},
  abstract = {Bias estimation of sensors is an essential prerequisite for accurate data fusion. Neglect of temporal bias in general real systems prevents the existing algorithms from successful application. In this paper, both spatial and temporal biases in asynchronous multisensor systems are investigated and two novel methods for simultaneous spatiotemporal bias compensation and data fusion are presented. The general situation that the sensors sample at different times with different and varying periods is explored, and unknown time delays may exist between the time stamps and the true measurement times. Due to the time delays, the time stamp interval of the measurements from different sensors may be different from their true measurement interval, and the unknown difference between them is considered as the temporal bias and augmented into the state vector to be estimated. Multisensor measurements are collected in batch processing or sequential processing schemes to estimate the augmented state vector, results in two spatiotemporal bias compensation methods. In both processing schemes, the measurements are formulated as functions of both target states and spatiotemporal biases according to the time difference between the measurements and the states to be estimated. The Unscented Kalman Filter is used to handle the nonlinearity of the measurements and produce spatiotemporal bias and target state estimates simultaneously. The posterior Cramer-Rao lower bound (PCRLB) for spatiotemporal bias and state estimation is presented and simulations are conducted to demonstrate the effectiveness of the proposed methods.},
  keywords = {spatiotemporal bias, state estimation, multisensor data fusion, asynchronous sensors},
  issn = {2998-3371},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations
Crossref
12
Scopus
9
Views
6483
PDF Downloads
1121

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

CC BY Copyright © 2024 by the Author(s). Published by Institute of Central Computation and Knowledge. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
Chinese Journal of Information Fusion
Chinese Journal of Information Fusion
ISSN: 2998-3371 (Online) | ISSN: 2998-3363 (Print)
Portico
Preserved at
Portico

Article QR Code

Article QR Code
Scan to read this article
Submit Manuscript Edit a Special Issue

Academic Editor

You He
You He
Tsinghua University, China

Popular Articles

Share This Article