Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training

Vijay Kumar; Arti Saxena; Leena Chawla

doi:10.62762/TMI.2025.317970

Volume 2, Issue 1, ICCK Transactions on Machine Intelligence

Volume 2, Issue 1, 2026

Submit Manuscript Edit a Special Issue

Article QR Code

Scan the QR code for reading

Popular articles

Case Studies on Integrating Artificial Intelligence in Finance to Transform Decision Making and Risk Management for Enhanced Financial Outcomes Reservoir Science: A Multi-Coupling Communication Platform to Promote Energy Transformation, Climate Change and Environmental Protection Reinforcement Learning for Prompt Optimization in Language Models: A Comprehensive Survey of Methods, Representations, and Evaluation Challenges From CO$_2$ Sequestration to Hydrogen Storage: Further Utilization of Depleted Gas Reservoirs AI and the Future of Education: Advancing Personalized Learning and Intelligent Tutoring Systems Effects of Crosslinking Agents and Reservoir Conditions on the Propagation of Fractures in Coal Reservoirs During Hydraulic Fracturing The Influence of Geological Factors and Transmission Fluids on the Exploitation of Reservoir Geothermal Resources: Factor Discussion and Mechanism Analysis YOLOv8-Lite: A Lightweight Object Detection Model for Real-time Autonomous Driving Systems Plant Disease Detection Using Deep Learning Techniques Current Status and Development Prospects of Carbon Capture, Utilization, and Storage (CCUS) in China: Technical, Policy, and Market Perspectives

ICCK Transactions on Machine Intelligence, Volume 2, Issue 1, 2026: 12-27

Free to Read | Research Article | 06 January 2026

Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training

Vijay Kumar 1 *

Arti Saxena 1

Leena Chawla 2

1 Manav Rachna International Institute of Research and Studies, Faridabad, Haryana 121010, India

2 DPG Institute of Technology and Management, Gurugram, Haryana, India

* Corresponding Author: Vijay Kumar, [email protected]

DOI: 10.62762/TMI.2025.317970

ARK: ark:/57805/tmi.2025.317970

Received: 14 November 2025, Accepted: 23 November 2025, Published: 06 January 2026

PDF (792.12 KB)

Article Metrics Cite This Article

Abstract

Adaptation learning is a data-driven technique that gives instructions based on the experiences made during data analysis. It plays an integral role in providing engineering solutions based on specific needs. Researchers have used the second-order statistics criterion for decades to conceptualize the optimality criteria using Shannon and Renyis information-theoretic measures. Some gaps have been identified in this research work, and useful findings have been proved with generalized information-theoretic measures of Renyis as Tsallis entropy of order $\alpha$ and Kapur entropy of order $\alpha$ and type $\beta$ using the Parzen-Rosenblatt window. This work explored the problem of constructing kernel density estimators and their application in adaptive systems training.

Graphical Abstract

Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training

Keywords

generalized entropies

adaptive learning

machine learning

kernel density estimation

information theoretic measures

Data Availability Statement

Data will be made available on request.

Funding

This work was supported without any funding.

Conflicts of Interest

The authors declare no conflicts of interest.

Ethical Approval and Consent to Participate

Not applicable.

References

Wiener, N. (1949). Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications. The MIT press.
[CrossRef] [Google Scholar]
Shannon, C. E. (1948). A mathematical theory of communication. The Bell system technical journal, 27(3), 379-423.
[CrossRef] [Google Scholar]
Rényi, A. (1961, January). On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics (Vol. 4, pp. 547-562). University of California Press.
[Google Scholar]
Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Journal of statistical physics, 52(1), 479-487.
[CrossRef] [Google Scholar]
Kapur, J. N. (1969, April). Some properties of entropy of order $\alpha$ and type $\beta$. In Proceedings of the Indian Academy of Sciences-Section A (Vol. 69, No. 4, pp. 201-211). New Delhi: Springer India.
[CrossRef] [Google Scholar]
Barlow, H. B., Kaushal, T. P., & Mitchison, G. J. (1989). Finding minimum entropy codes. Neural computation, 1(3), 412-423.
[CrossRef] [Google Scholar]
Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing?. Network: Computation in neural systems, 3(2), 213-251.
[CrossRef] [Google Scholar]
Intrator, N. (1991). Feature extraction using an unsupervised neural network. In Connectionist Models (pp. 310-318). Morgan Kaufmann.
[CrossRef] [Google Scholar]
Olshausen, B. A., & Field, D. J. (1996). Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7(2), 333.
[CrossRef] [Google Scholar]
Lazo, A. V., & Rathie, P. (1978). On the entropy of continuous probability distributions (corresp.). IEEE Transactions on Information Theory, 24(1), 120-122.
[CrossRef] [Google Scholar]
Ahmed, N. A., & Gokhale, D. V. (1989). Entropy expressions and their estimators for multivariate distributions. IEEE Transactions on Information Theory, 35(3), 688-692.
[CrossRef] [Google Scholar]
Havrda, J., & Charvat, F. (1967). Quantification method of classification processes. Concept of structural $\alpha$-entropy. Kybernetika, 3(1), 30-35. http://hdl.handle.net/10338.dmlcz/125526
[Google Scholar]
Kapur, J. N. (1994). Measures of information and their applications. Wiley. Retrieved from https://cir.nii.ac.jp/crid/1971430859819604036 (accessed on 05 January 2026).
[Google Scholar]
Principe, J. C., Xu, D., Zhao, Q., & Fisher Iii, J. W. (2000). Learning from examples with information theoretic criteria. Journal of VLSI signal processing systems for signal, image and video technology, 26(1), 61-77.
[CrossRef] [Google Scholar]
Sahoo, P., Wilkins, C., & Yeager, J. (1997). Threshold selection using Renyi's entropy. Pattern recognition, 30(1), 71-84.
[CrossRef] [Google Scholar]
Mittal, D. P. (1975). On additive and non-additive entropies. Kybernetika, 11(4), 271-276. http://hdl.handle.net/10338.dmlcz/125031
[Google Scholar]
Campbell, L. L. (1965). A coding theorem and Rényi's entropy. Information and control, 8(4), 423-429.
[CrossRef] [Google Scholar]
Bassat, M. B., & Raviv, J. (1978). Renyi's entropy and the probability of error. IEEE Transactions on Information Theory, 24(3), 324-331.
[CrossRef] [Google Scholar]
Cachin, C. (1997). Smooth entropy and Renyi entropy. In International Conference on the Theory and Applications of Cryptographic Techniques (pp. 193-208). Springer.
[CrossRef] [Google Scholar]
Principe, J. C. (2010). Information theoretic learning: Renyi's entropy and kernel perspectives. Springer Science & Business Media.
[CrossRef] [Google Scholar]
Xu, D. (1998). Energy, entropy and information potential for neural computation. University of Florida.
[Google Scholar]
Bowman, A. W., & Azzalini, A. (1997). Applied smoothing techniques for data analysis: the kernel approach with S-Plus illustrations (Vol. 18). OUP Oxford.
[Google Scholar]
Scott, D. W. (1992). Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons.
[CrossRef] [Google Scholar]
Silverman, B. W. (2018). Density estimation for statistics and data analysis. Routledge.
[CrossRef] [Google Scholar]
Härdle, W. (2004). Nonparametric and semiparametric models. Springer Science & Business Media.
[CrossRef] [Google Scholar]
Horová, I., Koláček, J., & Zelinka, J. (2012). Kernel Smoothing in MATLAB: theory and practice of kernel smoothing. World scientific.
[CrossRef] [Google Scholar]
Devroye, L., & Krzyzak, A. (1999). On the Hilbert kernel density estimate. Statistics & Probability Letters, 44(3), 299-308.
[CrossRef] [Google Scholar]
Duong, T., & Hazelton, M. L. (2008). Feature significance for multivariate kernel density estimation. Computational Statistics & Data Analysis, 52(9), 4225-4242.
[CrossRef] [Google Scholar]
Lake, D. E. (2009). Nonparametric entropy estimation using kernel densities. Methods in enzymology, 467, 531-546.
[CrossRef] [Google Scholar]
Gyorfi, L., & Van der Meulen, E. C. (1991). On the nonparametric estimation of the entropy functional. In G. Roussas (Ed.), Nonparametric Functional Estimation and Related Topics (pp. 81-95). Springer.
[CrossRef] [Google Scholar]
Ul Hassan, R., Yan, J., & Liu, Y. (2022). Security risk assessment of wind integrated power system using Parzen window density estimation. Electrical Engineering, 104(4), 1997-2008.
[CrossRef] [Google Scholar]
Beirlant, J., Dudewicz, E. J., Gyorfi, L., & Van der Meulen, E. C. (1997). Nonparametric entropy estimation: An overview. International Journal of Mathematical and Statistical Sciences, 6(1), 17-39.
[Google Scholar]
Ahmad, I., & Lin, P. E. (1976). A nonparametric estimation of the entropy for absolutely continuous distributions. IEEE Transactions on Information Theory, 22(3), 372-375.
[CrossRef] [Google Scholar]
Arimoto, S. (1971). Information-theoretical considerations on estimation problems. Information and control, 19(3), 181-194.
[CrossRef] [Google Scholar]
Parzen, E. (1962). On estimation of a probability density function and mode. The Annals of Mathematical Statistics, 33(3), 1065-1076.
[Google Scholar]
Loftsgaarden, D. O., & Quesenberry, C. P. (1965). A nonparametric estimate of a multivariate density function. The Annals of Mathematical Statistics, 36(3), 1049-1051.
[CrossRef] [Google Scholar]
Breiman, L., Meisel, W., & Purcell, E. (1977). Variable kernel estimates of multivariate densities. Technometrics, 19(2), 135-144.
[Google Scholar]
Davis, R. A., Lii, K. S., & Politis, D. N. (2011). Remarks on some nonparametric estimates of a density function. In Selected Works of Murray Rosenblatt (pp. 95-100). New York, NY: Springer New York.
[CrossRef] [Google Scholar]
Hardle, W. (1991). Smoothing techniques: With implementation in S. Springer.
[CrossRef] [Google Scholar]
Hall, P., Hu, T. C., & Marron, J. S. (1995). Improved variable window kernel estimates of probability densities. The Annals of Statistics, 23(1), 1-10.
[Google Scholar]
Wand, M. P., & Jones, M. C. (1994). Kernel smoothing. CRC Press.
[Google Scholar]
Stute, W. (1982). A law of the logarithm for kernel density estimators. The Annals of Probability, 10(2), 414-422.
[Google Scholar]
Stute, W. (1982). The oscillation behavior of empirical processes. The Annals of Probability, 10(1), 86-107.
[Google Scholar]
Einmahl, U., & Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. The Annals of Statistics, 33(3), 1380-1403.
[CrossRef] [Google Scholar]
Wied, D., & Weibbach, R. (2012). Consistency of the kernel density estimator: A survey. Statistical Papers, 53(1), 1-21.
[CrossRef] [Google Scholar]
Nadaraja, E. A. (1965). On non-parametric estimates of density functions and regression curves. Theory of Probability & Its Applications, 10(1), 186-190.
[CrossRef] [Google Scholar]
Erdogmus, D., & Principe, J. C. (2001, September). Convergence analysis of the information potential criterion in adaline training. In Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No. 01TH8584) (pp. 123-132). IEEE.
[CrossRef] [Google Scholar]
Erdogmus, D., & Principe, J. C. (2001). An on-line adaptation algorithm for adaptive system training with minimum error entropy: Stochastic information gradient. In Proceedings of the 2001 International Conference on Independent Component Analysis and Signal Separation (pp. 7-12).
[Google Scholar]

Cite This Article

APA Style

Kumar, V., Saxena, A., & Chawla, L. (2026). Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training. ICCK Transactions on Machine Intelligence, 2(1), 12–27. https://doi.org/10.62762/TMI.2025.317970

Export Citation

RIS Format

Compatible with EndNote, Zotero, Mendeley, and other reference managers

RIS format data for reference managers

TY  - JOUR
AU  - Kumar, Vijay
AU  - Saxena, Arti
AU  - Chawla, Leena
PY  - 2026
DA  - 2026/01/06
TI  - Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training
JO  - ICCK Transactions on Machine Intelligence
T2  - ICCK Transactions on Machine Intelligence
JF  - ICCK Transactions on Machine Intelligence
VL  - 2
IS  - 1
SP  - 12
EP  - 27
DO  - 10.62762/TMI.2025.317970
UR  - https://www.icck.org/article/abs/TMI.2025.317970
KW  - generalized entropies
KW  - adaptive learning
KW  - machine learning
KW  - kernel density estimation
KW  - information theoretic measures
AB  - Adaptation learning is a data-driven technique that gives instructions based on the experiences made during data analysis. It plays an integral role in providing engineering solutions based on specific needs. Researchers have used the second-order statistics criterion for decades to conceptualize the optimality criteria using Shannon and Renyis information-theoretic measures. Some gaps have been identified in this research work, and useful findings have been proved with generalized information-theoretic measures of Renyis as Tsallis entropy of order $\alpha$ and Kapur entropy of order $\alpha$ and type $\beta$ using the Parzen-Rosenblatt window. This work explored the problem of constructing kernel density estimators and their application in adaptive systems training.
SN  - 3068-7403
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -

BibTeX Format

Compatible with LaTeX, BibTeX, and other reference managers

BibTeX format data for LaTeX and reference managers

@article{Kumar2026Adaptive,
  author = {Vijay Kumar and Arti Saxena and Leena Chawla},
  title = {Adaptive Learning Density Estimators for Tsallis Entropy and Kapur Entropy with Applications in System Training},
  journal = {ICCK Transactions on Machine Intelligence},
  year = {2026},
  volume = {2},
  number = {1},
  pages = {12-27},
  doi = {10.62762/TMI.2025.317970},
  url = {https://www.icck.org/article/abs/TMI.2025.317970},
  abstract = {Adaptation learning is a data-driven technique that gives instructions based on the experiences made during data analysis. It plays an integral role in providing engineering solutions based on specific needs. Researchers have used the second-order statistics criterion for decades to conceptualize the optimality criteria using Shannon and Renyis information-theoretic measures. Some gaps have been identified in this research work, and useful findings have been proved with generalized information-theoretic measures of Renyis as Tsallis entropy of order \$\alpha\$ and Kapur entropy of order \$\alpha\$ and type \$\beta\$ using the Parzen-Rosenblatt window. This work explored the problem of constructing kernel density estimators and their application in adaptive systems training.},
  keywords = {generalized entropies, adaptive learning, machine learning, kernel density estimation, information theoretic measures},
  issn = {3068-7403},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations:

Google Scholar

Crossref

Scopus

Web of Science

Article Access Statistics:

PDF Downloads: 62

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

Institute of Central Computation and Knowledge (ICCK) or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

ICCK Transactions on Machine Intelligence

ISSN: 3068-7403 (Online)

Email: [email protected]

Portico

All published articles are preserved here permanently:
https://www.portico.org/publishers/icck/

User

Unlimited Downloads

Complete Library Access

Membership Eligibility

Community Leadership Opportunities

Google Scholar

Crossref

Scopus

Web of Science