Deteksi Aktivitas Manusia Berdasarkan Data Skeleton dengan Menggunakan Modifikasi VGG16
Abstract
In general, detection of human activity is carried out to detect activity in daily life. With further development, the detection of human activity is utilized to detect suspected activity (not routine) as an early warning application. Detection of human activity in early warning applications will then be implemented in other systems. However, there are several problems in detecting human activity, among others, the presence of variations in performing an activity, the movement of transitions between activities, and the similarity of movements in different activities. Detection of human activities with existing variations can be done if utilizing a deep learning approach to conduct the training process. The deep learning method used is VGG16. VGG16 will receive input in the form of skeleton data images. The skeleton data used is obtained from the NTU RGB+D dataset. Skeleton data will be represented as 2D images by going through a process of covering the crop, converted into grayscale, resizing, and connecting 10 images into 10 channels for each sequence of activity sequences. To detect human activity is applied transfer learning on VGG16 that is changing the fully connected layer. VGG16 modification test results with skeleton data representation resulting in the highest accuracy rate of 54.59%. This level of accuracy is obtained from model testing using the same dataset as the training dataset. The VGG16 modification is still the best model based on testing with other Convolutional Neural Network models. Modification of VGG16 can classify indoor activities.References
[1] Almaadeed, N., Elharrouss, O., Al-Maadeed, S.,
Bouridane, A. and Beghdadi, A. 2019. A Novel
Approach for Robust Multi Human Action Detection and
Recognition based on 3-Dimentional Convolutional
Neural Networks. (2019), 1–7.
[2] Bevilacqua, A., MacDonald, K., Rangarej, A., Widjaya,
V., Caulfield, B. and Kechadi, T. 2019. Human Activity
Recognition with Convolutional Neural Networks.
Springer International Publishing.
[3] Browne, D., Michael, G. and Steven, P. 2019. Deep
learning human activity recognition. CEUR Workshop
Proceedings. 2563, (2019), 76–87. DOI:
10.1016/j.neucom.2020.11.020.
[4] Cippitelli, E., Gambi, E., Spinsante, S. and FlórezRevuelta, F. 2016. Evaluation of a skeleton-based
method for human activity recognition on a large-scale
RGB-D dataset. IET Conference Publications. 2016, 4
(2016). DOI: 10.1049/ic.2016.0063.
[5] Dhiman, C., Saxena, M. and Vishwakarma, D.K. 2019.
Skeleton-based view invariant deep features for human
activity recognition. Proceedings - 2019 IEEE 5th
International Conference on Multimedia Big Data,
BigMM 2019. (2019), 225–230.
DOI:10.1109/BigMM.2019.00-21.
[6] Du, Y., Fu, Y. and Wang, L. 2016. Skeleton based action
recognition with convolutional neural network.
Proceedings - 3rd IAPR Asian Conference on Pattern
Recognition, ACPR 2015. (2016), 579–583. DOI:
10.1109/ACPR.2015.7486569.
[7] Feng, S. and Duarte, M.F. 2019. Few-shot learning-based
human activity recognition. Expert Systems with
Applications. 138, (2019), 112782. DOI:
10.1016/j.eswa.2019.06.070.
[8] Franco, A., Magnani, A. and Maio, D. 2020. A
multimodal approach for human activity recognition
based on skeleton and RGB data. Pattern Recognition
Letters. 131, (2020), 293–299. DOI:
10.1016/j.patrec.2020.01.010.
[9] Hussain, Z., Sheng, Q.Z. and Zhang, W.E. 2020. A
review and categorization of techniques on device-free
human activity recognition. Journal of Network and
Computer Applications. 167, June (2020), 102738. DOI:
10.1016/j.jnca.2020.102738.
[10] Ignatov, A. 2018. Real-time human activity recognition
from accelerometer data using Convolutional Neural
Networks. Applied Soft Computing Journal. 62, (2018),
915–922. DOI:10.1016/j.asoc.2017.09.027.
[11] Janani, M., Nataraj, M. and Ganesh, C.R.S. 2020. Mining
and monitoring human activity patterns in smart
environment-based healthcare systems. Elsevier Inc.
[12] Jobanputra, C., Bavishi, J. and Doshi, N. 2019. Human
activity recognition: A survey. Procedia Computer
Science. 155, 2018 (2019), 698–703. DOI:
10.1016/j.procs.2019.08.100.
[13] Lee, I., Kim, D., Kang, S. and Lee, S. 2017. Ensemble
Deep Learning for Skeleton-Based Action Recognition
Using Temporal Sliding LSTM Networks. Proceedings
of the IEEE International Conference on Computer
Vision. 2017-Octob, (2017), 1012–1020. DOI:
10.1109/ICCV.2017.115.
[14] Li, Y., Lan, C., Xing, J., Zeng, W., Yuan, C. and Liu, J.
2016. Online human action detection using joint
classification-regression recurrent neural networks.
Lecture Notes in Computer Science (including subseries
Lecture Notes in Artificial Intelligence and Lecture Notes
in Bioinformatics). 9911 LNCS, (2016), 203–220. DOI:
10.1007/978-3-319-46478-7_13.
[15] Liliana, Chae, J.H., Lee, J.J. and Lee, B.G. 2020. A
robust method for VR-based hand gesture recognition
using density-based CNN. Telkomnika
(Telecommunication Computing Electronics and
Control). (2020).
DOI:10.12928/TELKOMNIKA.v18i2.14747.
[16] Liu, J., Shahroudy, A., Wang, G., Duan, L.Y. and Kot,
A.C. 2020. Skeleton-Based Online Action Prediction
Using Scale Selection Network. IEEE Transactions on
Pattern Analysis and Machine Intelligence. 42, 6 (2020),
1453–1467. DOI: 10.1109/TPAMI.2019.2898954.
[17] Núñez, J.C., Cabido, R., Pantrigo, J.J., Montemayor, A.S.
and Vélez, J.F. 2018. Convolutional Neural Networks
and Long Short-Term Memory for skeleton-based human
activity and hand gesture recognition. Pattern
Recognition. 76, (2018), 80–94. DOI:
10.1016/j.patcog.2017.10.033.
[18] Seemanthini, K. and Manjunath, S.S. 2018. Human
Detection and Tracking using HOG for Action
Recognition. Procedia Computer Science. 132, Iccids
(2018), 1317–1326. DOI: 10.1016/j.procs.2018.05.048.
[19] Si, C., Chen, W., Wang, W., Wang, L. and Tan, T. 2019.
An attention enhanced graph convolutional lstm network
for skeleton-based action recognition. Proceedings of the
IEEE Computer Society Conference on Computer Vision
and Pattern Recognition. 2019-June, (2019), 1227–1236.
DOI: 10.1109/CVPR.2019.00132.
[20] Si, C., Jing, Y., Wang, W., Wang, L. and Tan, T. 2020.
Skeleton-based action recognition with hierarchical
spatial reasoning and temporal stack learning network.
Pattern Recognition. 107, xxxx (2020). DOI:
10.1016/j.patcog.2020.107511.
[21] Si, C., Jing, Y., Wang, W., Wang, L. and Tan, T. 2018.
Skeleton-Based Action Recognition with Spatial
Reasoning and Temporal Stack Learning. Lecture Notes
in Computer Science (including subseries Lecture Notes
in Artificial Intelligence and Lecture Notes in
Bioinformatics). 11205 LNCS, (2018), 106–121. DOI:
10.1007/978-3-030-01246-5_7.
[22] Simonyan, K. and Zisserman, A. 2015. Very deep
convolutional networks for large-scale image
recognition. 3rd International Conference on Learning
Representations, ICLR 2015 - Conference Track
Proceedings. (2015), 1–14.
[23] Subasi, A., Khateeb, K., Brahimi, T. and Sarirete, A.
2020. Human activity recognition using machine
learning methods in a smart healthcare environment.
Elsevier Inc.
[24] Vrigkas, M., Nikou, C. and Kakadiaris, I.A. 2015. A
review of human activity recognition methods. Frontiers
Robotics AI. 2, NOV (2015), 1–28. DOI:
10.3389/frobt.2015.00028.
[25] Yan, S., Li, Z., Xiong, Y., Yan, H. and Lin, D. 2019.
Convolutional sequence generation for skeleton-based
action synthesis. Proceedings of the IEEE International
Conference on Computer Vision. 2019-Octob, (2019),
4393–4401. DOI: 10.1109/ICCV.2019.00449.
[26] Zhou, K., Wu, T., Wang, C., Wang, J. and Li, C. 2020.
Skeleton Based Abnormal Behavior Recognition Using
Spatio-Temporal Convolution and Attention-Based
LSTM. Procedia Computer Science. 174, 2019 (2020),
424–432. DOI: 10.1016/j.procs.2020.06.110.
[27] Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L. and
Xie, X. 2016. Co-Occurrence feature learning for
skeleton based action recognition using regularized deep
LSTM networks. 30th AAAI Conference on Artificial
Intelligence, AAAI 2016. i (2016), 3697–3703.
