Science Journal of Circuits, Systems and Signal Processing

Submit a Manuscript

Publishing with us to make your research visible to the widest possible audience.

Propose a Special Issue

Building a community of authors and readers to discuss the latest research and develop new ideas.

Multiple Sign Language Identification Using Deep Learning Techniques

The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.

SLI, Capsule Neural Network, Deep Learning, VGG16, LeNet

APA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. (2023). Multiple Sign Language Identification Using Deep Learning Techniques. Science Journal of Circuits, Systems and Signal Processing, 11(1), 1-11. https://doi.org/10.11648/j.cssp.20231101.11

ACS Style

Ahmed Mahmoud Sultan; Waleed Makram Mohamed Zaki; Mohammed Kayed; Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci. J. Circuits Syst. Signal Process. 2023, 11(1), 1-11. doi: 10.11648/j.cssp.20231101.11

AMA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci J Circuits Syst Signal Process. 2023;11(1):1-11. doi: 10.11648/j.cssp.20231101.11

Copyright © 2023 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. J. Quer and M. Steinbach, "Handling sign language data: The impact of modality," Frontiers in Psychology, 2019.
2. R. Kushalnagar, "Deafness and hearing loss," Human–Computer Interaction Series, pp. 35-47, 2019.
3. B. I. II, "Deafness in the Arab world: a general investigation, with applications to Lebanon," scholarship.tricolib.brynmawr.edu, 2018.
4. C. D. Monteiro, C. M. Mathew, R. Gutierrez-Osuna and F. Shipman, "Detecting and identifying sign languages through visual features," 2016 IEEE International Symposium on Multimedia (ISM), 2016.
5. A. Sultan, W. Makram, M. Kayed and A. A. ALi, "Sign language identification and recognition: A comparative study," Open Computer Science, pp. 191-210, 2022.
6. M. Mohandes, Junzhao Liu and M. Deriche, "A survey of image-based Arabic sign language recognition," 2014 IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14), 2014.
7. R. Daroya, D. Peralta and P. Naval, "Alphabet sign language image classification using deep learning," TENCON 2018 - 2018 IEEE Region 10 Conference, 2018.
8. E. Goceri, "Analysis of capsule networks for Image Classification," Proceedings of the 15th International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing (CGVCVIP 2021), the 7th International Conference on Connected Smart Cities (CSC 2021) and 6th International Conference on Big Data A, 2021.
9. S. Sabour, N. Frosst and H. Geoffrey E, "Dynamic routing between capsules," Advances in neural information processing systems, 2017.
10. Y. LeCun, B. Yoshua and H. Geoffrey, "Deep learning," nature, vol. 521, no. 7553, pp. 436-444, 2015.
11. A. El Zaar, N. Benaya and A. El Allati, "Sign language recognition: High performance deep learning approach applyied to multiple sign languages," E3S Web of Conferences, 2022.
12. H. T. Nguyen, L. T. Pham, T. T. Mai, T. K. Vo and T. T. Dien, "Letter recognition in hand sign language with VGG-16," Intelligent Systems and Networks, pp. 410-417, 2022.
13. Z. Alsaadi, E. Alshamani, M. Alrehaili, A. A. Alrashdi, S. Albelwi and A. O. Elfaki, "A real time Arabic Sign Language Alphabets (arsla) recognition model using deep learning architecture," Computers, vol. 11, no. 5, p. 78, 2022.
14. A.-J. Tanseem N and A.-J. Abu-Jamie, "Classification of Sign-language Using VGG16," 2022.
15. V. Shreya and Y. Shaik Sohail, "Sign Language Interpreter Using Computer Vision and LeNet-5 Convolutional Neural Network Architecture," International Journal of Innovative Science and Research Technolog, vol. 6, 2021.
16. M. Bilgin and K. Mutludogan, "American sign language character recognition with capsule networks," 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2019.
17. H. Xiao, Y. Yang, K. Yu, J. Tian, X. Cai, U. Muhammad and J. Chen, "Sign language digits and alphabets recognition by Capsule Networks," Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 4, pp. 2131-2141, 2021.
18. A. M. B. S. A. K. M. B. and M., "Different techniques of hand segmentation in the real time," IJCAIT, pp. 45-49, 2013.
19. Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard and L. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, pp. 541-551, 1989.
20. S. Karen and Z. Andrew, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv: 1409.1556, 2014.
21. G. B and S. Natarajan, "Hyperparameter optimisation for Capsule Networks," EAI Endorsed Transactions on Cloud Systems, 2019.
22. J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009.
23. K. Suri and R. Gupta, "Continuous sign language recognition from wearable Imus using deep capsule networks and game theory," Computers & Electrical Engineering, vol. 78, pp. 493-503, 2019.