Penerapan metode hand gesture recognition dalam melakukan kontrol terhadap aplikasi powerpoint dan media player untuk kebutuhan online conference

William Sean Wiyogo(1*), Liliana Liliana(2),

(1) Program Studi Teknik Informatika, Universitas Kristen Petra Surabaya
(2) Program Studi Teknik Informatika, Universitas Kristen Petra Surabaya
(*) Corresponding Author


Since the prolonged COVID-19 pandemic, most human activities are seen with the concept of virtual meetings. This concept is helped by the use of an online conferencing platform. As a result, needs arise among the new society. The learning model of hybrid learning, or blended learning is a combination of face-to-face learning with e-learning. This learning method reduces teaching performance due to limited range of motion. Thus, hand tracking gesture recognition can be used as a solution to overcome this problem. This study aims to model a gesture recognition system with statistical and dynamic recognition. The method used in this research is CNN-based RT3D_16F which is used as dynamic motion prediction and Mediapipe hand pipeline which is used as static motion prediction. The data set used consists of 27 movement labels (includes 2 movement labels that shouldn't be recognized as specific moves).


hand tracking gesture recognition; RT3D_16F; CNN; mediapipe; online conference

Full Text:



Abraham, L., Urru, A., Normani, N., Wilk, M. P., Walsh, M.,

& O’Flynn, B. (2018). Hand tracking and gesture recognition

using lensless smart sensors. Sensors, 18(9), 2834. DOI:


Agusti, R., Alcaraz, C., Au, R., Barin, K., Bernardo, F.,

Bertino, E., ... & Dakopoulos, D. (2010). 2010 Index IEEE

Transactions on Systems, Man, and Cybernetics, Part C

(Applications and Reviews) Vol. 40. IEEE TRANSACTIONS




Bretzner, L., Laptev, I., & Lindeberg, T. (2002, May). Hand

gesture recognition using multi-scale colour features,

hierarchical models and particle filtering. In Proceedings of

fifth IEEE international conference on automatic face gesture

recognition (pp. 423-428). IEEE.

Dardas, N. H., & Georganas, N. D. (2011). Real-time hand

gesture detection and recognition using bag-of-features and

support vector machine techniques. IEEE Transactions on

Instrumentation and measurement, 60(11), 3592-3607. DOI:


Freeman, W. T., & Roth, M. (1995, June). Orientation

histograms for hand gesture recognition. In International

workshop on automatic face and gesture recognition (Vol. 12,

pp. 296-301).

Ganjar, M., Mardiko, S. (2022). Hybrid Learning: Suatu

Solusi Di Tengah Ancaman Dan Tantangan Pendidikan Di

Masa Pandemi. aman-dan-tantangan-pendidikan-dimasa-pandemi/.

Graham, C. R. (2005). Blended learning systems: Definition,

current trends, and future directions. In C. J. Bonk & C. R.

Graham (Eds.). Handbook of blended learning: Global

perspectives, local designs (pp. 3–21). San Francisco: Pfeiffer


Makahaube, S. S. et al. (2021). Implementation of Gesture

Recognition Technology for Automated Education Service

Kiosk. Jurnal Teknik Informatika, 14(4), 465-472.

Starner, T., & Pentland, A. (1997). Real-time american sign

language recognition from video using hidden markov

models. In Motion-based recognition (pp. 227-243). Springer,

Dordrecht. DOI: 10.1007/978-94-015-8935-2_10.

Wu, Y., & Huang, T. S. (1999, March). Vision-based gesture

recognition: A review. In International gesture workshop (pp.

-115). Springer, Berlin, Heidelberg. DOI: 10.1007/3-540-


Xiang, Y. T., Li, W., Zhang, Q., Jin, Y., Rao, W. W., Zeng, L.

N., et al. (2020). Timely research papers about COVID-19 in

China. The Lancet. DOI: 10.1016/S0140-6736(20)30375-5.

Xu, P. (2017). A real-time hand gesture recognition and

human-computer interaction system. arXiv preprint


Yunita, H., Setyati, E. (2019). Hand Gesture Recognition

Sebagai Pengganti Mouse Komputer Menggunakan Kamera.

Pusat Penelitian dan Pengabdian kepada Masyarakat (P3M)

Politeknik Negeri Banjarmasin. DOI:



  • There are currently no refbacks.

Jurnal telah terindeks oleh :