IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges

Baleegha Fatima; Bilal Mushtaq; Muhammad Ammar Iqbal; Abbas Ahmed

doi:10.62762/TIOT.2024.723193

Volume 2, Issue 4, ICCK Transactions on Internet of Things

Volume 2, Issue 4, 2024

Submit Manuscript Edit a Special Issue

Academic Editor

Jinchao Chen

Northwestern Polytechnical University, China

Article QR Code

Scan the QR code for reading

Popular articles

Case Studies on Integrating Artificial Intelligence in Finance to Transform Decision Making and Risk Management for Enhanced Financial Outcomes Reservoir Science: A Multi-Coupling Communication Platform to Promote Energy Transformation, Climate Change and Environmental Protection From CO$_2$ Sequestration to Hydrogen Storage: Further Utilization of Depleted Gas Reservoirs Reinforcement Learning for Prompt Optimization in Language Models: A Comprehensive Survey of Methods, Representations, and Evaluation Challenges AI and the Future of Education: Advancing Personalized Learning and Intelligent Tutoring Systems Effects of Crosslinking Agents and Reservoir Conditions on the Propagation of Fractures in Coal Reservoirs During Hydraulic Fracturing The Influence of Geological Factors and Transmission Fluids on the Exploitation of Reservoir Geothermal Resources: Factor Discussion and Mechanism Analysis YOLOv8-Lite: A Lightweight Object Detection Model for Real-time Autonomous Driving Systems Plant Disease Detection Using Deep Learning Techniques Modeling Brain Functional Networks Using Graph Neural Networks: A Review and Clinical Application

ICCK Transactions on Internet of Things, Volume 2, Issue 4, 2024: 74-82

Free to Read | Research Article | 19 November 2024

IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges

Baleegha Fatima 1

Bilal Mushtaq 1 *

Muhammad Ammar Iqbal 1

Abbas Ahmed 1

1 Beaconhouse International College, Islamabad, Pakistan

* Corresponding Author: Bilal Mushtaq, [email protected]

DOI: 10.62762/TIOT.2024.723193

ARK: ark:/57805/tiot.2024.723193

Received: 02 September 2024, Accepted: 04 November 2024, Published: 19 November 2024

PDF (3.20 MB)

Article Metrics Cite This Article

Abstract

This paper reviews advancements in assistive technology for deaf and hard of hearing individuals, highlighting the Internet of Things' (IoTs) pivotal role in enhancing their daily lives. Despite progress in sign language technologies, communication barriers persist. To address these gaps, we developed a video-based American Sign Language (ASL) identification system. Our approach utilizes MediaPipe for hand tracking, OpenCV for image normalization, and Gesture Control Convolutional Neural Network (CNN) for gesture localization. Implemented in Python, the system records video streams, filters hand regions, and recognizes ASL letter gestures with high accuracy. Leveraging computer vision and machine learning, our system enhances user experience, breaks communication barriers, promotes inclusivity, and supports accessible technologies. This innovative solution empowers deaf and hard of hearing individuals to fully participate in their communities, contributing to a more inclusive and accessible environment.

Graphical Abstract

IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges

Keywords

convolutional neural networks

sign language

artificial intelligence

machine learning

American sign language

image processing

internet of things

home automation

Funding

This work was supported without any funding.

References

Yang, L. I., Huang, J., Feng, T. I. A. N., Hong-An, W. A. N. G., & Guo-Zhong, D. A. I. (2019). Gesture interaction in virtual reality. Virtual Reality & Intelligent Hardware, 1(1), 84-112.
[Google Scholar]
Kheratkar, N., Bhavani, S., Jarali, A., Pathak, A., & Kumbhar, S. (2020, May). Gesture controlled home automation using CNN. In 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 620-626). IEEE.
[Google Scholar]
Tomar, D., Nauni, D., Zaidi, A. M., & Kaur, M. (2023, November). Gesture-Controlled Home Automation for the Differently Abled: Enhanced Accessibility and Independence. In 2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS) (pp. 1560-1564). IEEE.
[Google Scholar]
Okokpujie, K., Jacinth, D., James, G. A., Okokpujie, I. P., & Vincent, A. A. (2023). An IoT-based multimodal real-time home control system for the physically challenged: Design and implementation. Inf. Dyn. Appl, 2(2), 90-100.
[Google Scholar]
Kurian, B., Regi, J., John, D., Hari, P., & Mahesh, T. Y. (2023, May). Visual Gesture-Based Home Automation. In 2023 3rd International Conference on Advances in Computing, Communication, Embedded and Secure Systems (ACCESS) (pp. 286-290). IEEE.
[Google Scholar]
Yang, Y. (2016). Gesture controlled user interface for elderly people (Master’s thesis, Oslo and Akershus University College of Applied Sciences).
[Google Scholar]
Davis, J., & Shah, M. (1994). Recognizing hand gestures. In Computer Vision—ECCV’94: Third European Conference on Computer Vision Stockholm, Sweden, May 2–6, 1994 Proceedings, Volume I 3 (pp. 331-340). Springer Berlin Heidelberg.
[Google Scholar]
Wu, Y., & Huang, T. S. (2001). Hand modeling, analysis and recognition. IEEE Signal Processing Magazine, 18(3), 51-60.
[Google Scholar]
Park, G., Chandrasegar, V. K., & Koh, J. (2023). Accuracy enhancement of hand gesture recognition using CNN. IEEE Access, 11, 26496-26501.
[Google Scholar]
Sun, J. H., Ji, T. T., Zhang, S. B., Yang, J. K., & Ji, G. R. (2018, December). Research on the hand gesture recognition based on deep learning. In 2018 12th International symposium on antennas, propagation and EM theory (ISAPE) (pp. 1-4). IEEE.
[Google Scholar]
Licsár, A., & Szirányi, T. (2004, August). Dynamic training of hand gesture recognition system. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. (Vol. 4, pp. 971-974). IEEE.
[Google Scholar]
Hendricks, D. (2014). The history of smart homes. IoT Evolution World.
[Google Scholar]
Varriale, L., Briganti, P., & Mele, S. (2020). Disability and home automation: insights and challenges within organizational settings. In Exploring Digital Ecosystems: Organizational and Human Challenges (pp. 47-66). Springer International Publishing.
[Google Scholar]
Mushtaq, B., Rehman, M. A., Hussain, A., & Abbass, M. J. (2023, March). A highly selective dual bandpass filter using couple line resonator for modern wireless communication systems. In 2023 4th International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1-5). IEEE.
[Google Scholar]
Asadullah, M., & Raza, A. (2016, November). An overview of home automation systems. In 2016 2nd international conference on robotics and artificial intelligence (ICRAI) (pp. 27-31). IEEE.
[Google Scholar]
Mushtaq, B., Rehman, M. A., Khalid, S., & Alhaisoni, M. (2023). Design of Tri-Band Bandpass Filter Using Modified X-Shaped Structure for IoT-Based Wireless Applications. IEEE Embedded Systems Letters, 16(2), 194-197.
[Google Scholar]
Kim, Y., & Toomajian, B. (2016). Hand gesture recognition using micro-Doppler signatures with convolutional neural network. IEEE Access, 4, 7125-7130.
[Google Scholar]
Abdul Rehman, M., Khalid, S., Mushtaq, B., & Idrees, M. (2022). Design of a novel compact highly selective wideband bandstop RF filter using dual path lossy resonator for next generation applications. Plos one, 17(10), e0273514.
[Google Scholar]
Katre, S. R., & Rojatkar, D. V. (2017). Home automation: past, present and future. International research journal of engineering and technology, 4(10), 343-346.
[Google Scholar]
Ahmed, A., Botsinis, P., Won, S., Yang, L. L., & Hanzo, L. (2018). EXIT Chart Aided Convergence Analysis of Recursive Soft $ m $-Sequence Initial Acquisition in Nakagami-m Fading Channels. IEEE Transactions on Vehicular Technology, 67(5), 4655-4660.
[Google Scholar]
Ahmed, A. (2019). Iterative initial synchronization in wireless communications (Doctoral dissertation, University of Southampton).
[Google Scholar]
Mushtaq, B., & Khalid, S. (2023). Design of miniaturized single and dual-band bandpass filters using diamond-shaped coupled line resonator for next-generation wireless systems. International Journal of Microwave and Wireless Technologies, 15(3), 375-383.
[Google Scholar]
Mushtaq, B., Khalid, S., & Rehman, M. A. (2022). Design of a compact novel stub loaded pentaband bandpass filter for next generation wireless RF front ends. IEEE Access, 10, 109919-109924.
[Google Scholar]
Ahmed, A., Ahmed, Q. Z., Almogren, A., Haider, S. K., & Rehman, A. U. (2021). Hybrid precoding aided fast frequency-hopping for millimeter-wave Communication. IEEE Access, 9, 149596-149608.
[Google Scholar]
UN DESA. 2023. The Sustainable Development Goals Report 2023: Special Edition. New York, USA: UN DESA. © UN DESA.
[Google Scholar]
Khokher, A., Mushtaq, B., Rehman, M. A., & Abbas, M. J. (2024). RF Planning And Optimization Of 5G On The City Campus (MUST) of Mirpur, Pakistan. IECE Transactions on Sensing, Communication, and Control, 1(1), 52-59.
[Google Scholar]
Rehman, M. A., Mushtaq, B., Khalid, S., & Iqbal, J. (2023, August). Design and Analysis of Broadband Bandstop Filter using Dual Path Capacitive Coupled Resonator for 4G and 5G Applications. In 2023 20th International Bhurban Conference on Applied Sciences and Technology (IBCAST) (pp. 385-392). IEEE.
[Google Scholar]
Idrees, M., Mushtaq, B., Rehman, M. A., Khalid, S., & Iqbal, J. (2023, August). Design of a Planar Four-Port Microstrip Triplexer using Stub-Loaded Coupled Line Resonator for Advanced Wireless Applications. In 2023 20th International Bhurban Conference on Applied Sciences and Technology (IBCAST) (pp. 398-403). IEEE.
[Google Scholar]
Rehman, M. A., Mushtaq, B., Khalid, S., & Rehman, M. U. (2024). Design of a miniaturized multi resonance resonator based highly selective dual wideband bandpass filter. Microelectronics Journal, 153, 106411.
[Google Scholar]
Idrees, M., Khalid, S., Abdulrehman, M., Mushtaq, B., Najam, A. I., & Alhaisoni, M. (2023). Design of a stub-loaded coupled line diplexer for IoT-based applications. IEEE Embedded Systems Letters, 16(2), 186-189.
[Google Scholar]
Abdul Rehman, M., Khalid, S., Mushtaq, B., Uddin, M., Iqbal, J., Abdelhaq, M., & Alsaqour, R. (2023). A Novel Synthesis of Quasi-Chebyshev Ultra-Wideband Bandpass Filter Using N th Order Stub Loaded Coupled-Line Resonator. Micromachines, 14(10), 1874.
[Google Scholar]
Rehman, A., Rahman, M. A., Aziz, N., Mushtaq, B., Abbass, M. J., Khalid, S., & Jan, A. Z. (2022). Design of Highly Selective Dual Band Band Stop Filter using Dual-Path Step Impedance Resonator. Pakistan Journal of Engineering and Technology, 5(2), 146-151.
[Google Scholar]

Cite This Article

APA Style

Fatima, B., Mushtaq, B., Iqbal, M. A., & Ahmed, A. (2024). IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges. ICCK Transactions on Internet of Things, 2(4), 74–82. https://doi.org/10.62762/TIOT.2024.723193

Export Citation

RIS Format

Compatible with EndNote, Zotero, Mendeley, and other reference managers

RIS format data for reference managers

TY  - JOUR
AU  - Fatima, Baleegha
AU  - Mushtaq, Bilal
AU  - Iqbal, Muhammad Ammar
AU  - Ahmed, Abbas
PY  - 2024
DA  - 2024/11/19
TI  - IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges
JO  - ICCK Transactions on Internet of Things
T2  - ICCK Transactions on Internet of Things
JF  - ICCK Transactions on Internet of Things
VL  - 2
IS  - 4
SP  - 74
EP  - 82
DO  - 10.62762/TIOT.2024.723193
UR  - https://www.icck.org/article/abs/TIOT.2024.723193
KW  - convolutional neural networks
KW  - sign language
KW  - artificial intelligence
KW  - machine learning
KW  - American sign language
KW  - image processing
KW  - internet of things
KW  - home automation
AB  - This paper reviews advancements in assistive technology for deaf and hard of hearing individuals, highlighting the Internet of Things' (IoTs) pivotal role in enhancing their daily lives. Despite progress in sign language technologies, communication barriers persist. To address these gaps, we developed a video-based American Sign Language (ASL) identification system. Our approach utilizes MediaPipe for hand tracking, OpenCV for image normalization, and Gesture Control Convolutional Neural Network (CNN) for gesture localization. Implemented in Python, the system records video streams, filters hand regions, and recognizes ASL letter gestures with high accuracy. Leveraging computer vision and machine learning, our system enhances user experience, breaks communication barriers, promotes inclusivity, and supports accessible technologies. This innovative solution empowers deaf and hard of hearing individuals to fully participate in their communities, contributing to a more inclusive and accessible environment.
SN  - pending
PB  - Institute of Central Computation and Knowledge
LA  - English
ER  -

BibTeX Format

Compatible with LaTeX, BibTeX, and other reference managers

BibTeX format data for LaTeX and reference managers

@article{Fatima2024IoTbased,
  author = {Baleegha Fatima and Bilal Mushtaq and Muhammad Ammar Iqbal and Abbas Ahmed},
  title = {IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges},
  journal = {ICCK Transactions on Internet of Things},
  year = {2024},
  volume = {2},
  number = {4},
  pages = {74-82},
  doi = {10.62762/TIOT.2024.723193},
  url = {https://www.icck.org/article/abs/TIOT.2024.723193},
  abstract = {This paper reviews advancements in assistive technology for deaf and hard of hearing individuals, highlighting the Internet of Things' (IoTs) pivotal role in enhancing their daily lives. Despite progress in sign language technologies, communication barriers persist. To address these gaps, we developed a video-based American Sign Language (ASL) identification system. Our approach utilizes MediaPipe for hand tracking, OpenCV for image normalization, and Gesture Control Convolutional Neural Network (CNN) for gesture localization. Implemented in Python, the system records video streams, filters hand regions, and recognizes ASL letter gestures with high accuracy. Leveraging computer vision and machine learning, our system enhances user experience, breaks communication barriers, promotes inclusivity, and supports accessible technologies. This innovative solution empowers deaf and hard of hearing individuals to fully participate in their communities, contributing to a more inclusive and accessible environment.},
  keywords = {convolutional neural networks, sign language, artificial intelligence, machine learning, American sign language, image processing, internet of things, home automation},
  issn = {pending},
  publisher = {Institute of Central Computation and Knowledge}
}

Article Metrics

Citations:

Google Scholar

Crossref

Scopus

Web of Science

Article Access Statistics:

PDF Downloads: 597

Publisher's Note

ICCK stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and Permissions

Institute of Central Computation and Knowledge (ICCK) or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

ICCK Transactions on Internet of Things

ISSN: pending (Online)

Email: [email protected]

Portico

All published articles are preserved here permanently:
https://www.portico.org/publishers/icck/

User

Unlimited Downloads

Complete Library Access

Membership Eligibility

Community Leadership Opportunities

Google Scholar

Crossref

Scopus

Web of Science