Pengenalan Alfabet Bahasa Isyarat Tangan Secara Real-Time dengan Menggunakan Metode Convolutional Neural Network dan Recurrent Neural Network
Keywords:
Corporate Social Responsibility, Information Asymmetry, Sustainability Reporting, Bid Ask Spread, Volatility Cash Flow, and Industry Profile.Abstract
Sign language is one of the communication tools commonly used by people with disabilities. The alphabet sign language is a basic tool used by teachers to teach people with hearing impairment and speech impairment to recognize basic alphabet letters. However, many people find it difficult to communicate with these groups because of a lack of community insight into hand sign language. Research on sign language has experienced much progress in processing static images but is still experiencing problems due to difficulties in processing dynamic images / video given that most of the sign language is represented by body, hand, and face movements.
This study uses Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods with video input. The CNN method will be used as a feature extraction in the spatial feature while the RNN is tasked to tolerate between frames extracted by CNN on the temporal feature.
The final result to be displayed is in the form of text alphabet which is the result of the recognition of the sign language alphabet. Based on the test carried out, obtained an average accuracy value of 60.58% for all letters while real-time testing has failed because the technology used cannot sustain the architecture created.References
[1] Adityo, M. 2019, March 25. Sistem Isyarat Bahasa Indonesia (SIBI) atau Bahasa Isyarat Indonesia (BISINDO)?. Retrieved from Kolom YOTers: https://www.youngontop.com/read/20433/sistem-isyarat-bahasa-indonesia-sibi-atau-bahasa-isyarat-indonesia-bisindo/
[2] Aishwarya. n.d. Introduction to Recurrent Neural Network. Retrieved from GeeksforGeeks: https://www.geeksforgeeks.org/introduction-to-recurrent-neural-network/
[3] Benafanne, Y. 2019, August 13. Transformer vs RNN and CNN for Translation Task. Retrieved from Medium: https://medium.com/analytics-vidhya/transformer-vs-rnn-and-cnn-18eeefa3602b
[4] Ciaburro, G., & Venkateswaran, B. 2017. Neural Networks with R. Birmingham: Packt Publishing Ltd.
[5] Dinesh, S., Sivaprakash, S., Keshav, M., & Ramya, K. 2018, March. Real-Time American Sign Language Recognition with Faster Regional Convolutional Neuralnetworks. International Journal of Innovative Reaserch in Science Engineering and Technology (IJIRSET) (Vol. 7).
[6] Ferlet, P. 2019, July 23. How to work with Time Distributed data in a neural network. Retrieved from Medium: https://medium.com/smileinnovation/how-to-work-with-time-distributed-data-in-a-neural-network-b8b39aa4ce00
[7] Isma, S. T. n.d. MENELITI BAHASA ISYARAT DALAM PERSPEKTIF VARIASI BAHASA.
[8] Lokhande, P., Prajapati, R., & Pansare, S. 2015. Data gloves for sign language recognition system. International Journal of Computer Applications, 975, 8887.
[9] Masood, S., Srivastava, A., Thuwal, H. C., & Ahmad, M. 2018. Real-time sign language gesture (word) recognition from video sequences using CNN and RNN. In Intelligent Engineering Informatics (pp. 623-632). Springer, Singapore.
[10] Sofia, N. 2018. CONVOLUTIONAL NEURAL NETWORK. Retrieved from Medium: https://medium.com/@nadhifasofia/1-convolutional-neural-network-convolutional-neural-network-merupakan-salah-satu-metode-machine-28189e17335b
[11] Sommerville, I. 2004. Real-Time Software Design (7th ed., Chapter 15). Software Engineering. Retrieved from https://www.coursehero.com/file/17891300/ch15/
[12] Tao, W., Leu, M. C., & Yin, Z. 2018. American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion. Engineering Applications of Artificial Intelligence, 76, 202-213
[13] Zaytar, M. A., & El Amrani, C. 2016. Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. International Journal of Computer Applications, 143(11), 7-1
