Deteksi Aktivitas Manusia Berdasarkan Data Skeleton dengan Menggunakan Modifikasi VGG16

Daniel Subroto(1*), Liliana Liliana(2),


(1) Program Studi Informatika
(2) Program Studi Informatika
(*) Corresponding Author

Abstract


In general, detection of human activity is carried out to detect activity in daily life. With further development, the detection of human activity is utilized to detect suspected activity (not routine) as an early warning application. Detection of human activity in early warning applications will then be implemented in other systems. However, there are several problems in detecting human activity, among others, the presence of variations in performing an activity, the movement of transitions between activities, and the similarity of movements in different activities. Detection of human activities with existing variations can be done if utilizing a deep learning approach to conduct the training process. The deep learning method used is VGG16. VGG16 will receive input in the form of skeleton data images. The skeleton data used is obtained from the NTU RGB+D dataset. Skeleton data will be represented as 2D images by going through a process of covering the crop, converted into grayscale, resizing, and connecting 10 images into 10 channels for each sequence of activity sequences. To detect human activity is applied transfer learning on VGG16 that is changing the fully connected layer. VGG16 modification test results with skeleton data representation resulting in the highest accuracy rate of 54.59%. This level of accuracy is obtained from model testing using the same dataset as the training dataset. The VGG16 modification is still the best model based on testing with other Convolutional Neural Network models. Modification of VGG16 can classify indoor activities.

Keywords


Human Activity Recognition; NTU RGB+D; Convolutional Neural Network; VGG16; Skeleton data

Full Text:

PDF

References


Almaadeed, N., Elharrouss, O., Al-Maadeed, S.,

Bouridane, A. and Beghdadi, A. 2019. A Novel

Approach for Robust Multi Human Action Detection and

Recognition based on 3-Dimentional Convolutional

Neural Networks. (2019), 1–7.

Bevilacqua, A., MacDonald, K., Rangarej, A., Widjaya,

V., Caulfield, B. and Kechadi, T. 2019. Human Activity

Recognition with Convolutional Neural Networks.

Springer International Publishing.

Browne, D., Michael, G. and Steven, P. 2019. Deep

learning human activity recognition. CEUR Workshop

Proceedings. 2563, (2019), 76–87. DOI:

1016/j.neucom.2020.11.020.

Cippitelli, E., Gambi, E., Spinsante, S. and FlórezRevuelta, F. 2016. Evaluation of a skeleton-based

method for human activity recognition on a large-scale

RGB-D dataset. IET Conference Publications. 2016, 4

(2016). DOI: 10.1049/ic.2016.0063.

Dhiman, C., Saxena, M. and Vishwakarma, D.K. 2019.

Skeleton-based view invariant deep features for human

activity recognition. Proceedings - 2019 IEEE 5th

International Conference on Multimedia Big Data,

BigMM 2019. (2019), 225–230.

DOI:10.1109/BigMM.2019.00-21.

Du, Y., Fu, Y. and Wang, L. 2016. Skeleton based action

recognition with convolutional neural network.

Proceedings - 3rd IAPR Asian Conference on Pattern

Recognition, ACPR 2015. (2016), 579–583. DOI:

1109/ACPR.2015.7486569.

Feng, S. and Duarte, M.F. 2019. Few-shot learning-based

human activity recognition. Expert Systems with

Applications. 138, (2019), 112782. DOI:

1016/j.eswa.2019.06.070.

Franco, A., Magnani, A. and Maio, D. 2020. A

multimodal approach for human activity recognition

based on skeleton and RGB data. Pattern Recognition

Letters. 131, (2020), 293–299. DOI:

1016/j.patrec.2020.01.010.

Hussain, Z., Sheng, Q.Z. and Zhang, W.E. 2020. A

review and categorization of techniques on device-free

human activity recognition. Journal of Network and

Computer Applications. 167, June (2020), 102738. DOI:

1016/j.jnca.2020.102738.

Ignatov, A. 2018. Real-time human activity recognition

from accelerometer data using Convolutional Neural

Networks. Applied Soft Computing Journal. 62, (2018),

–922. DOI:10.1016/j.asoc.2017.09.027.

Janani, M., Nataraj, M. and Ganesh, C.R.S. 2020. Mining

and monitoring human activity patterns in smart

environment-based healthcare systems. Elsevier Inc.

Jobanputra, C., Bavishi, J. and Doshi, N. 2019. Human

activity recognition: A survey. Procedia Computer

Science. 155, 2018 (2019), 698–703. DOI:

1016/j.procs.2019.08.100.

Lee, I., Kim, D., Kang, S. and Lee, S. 2017. Ensemble

Deep Learning for Skeleton-Based Action Recognition

Using Temporal Sliding LSTM Networks. Proceedings

of the IEEE International Conference on Computer

Vision. 2017-Octob, (2017), 1012–1020. DOI:

1109/ICCV.2017.115.

Li, Y., Lan, C., Xing, J., Zeng, W., Yuan, C. and Liu, J.

Online human action detection using joint

classification-regression recurrent neural networks.

Lecture Notes in Computer Science (including subseries

Lecture Notes in Artificial Intelligence and Lecture Notes

in Bioinformatics). 9911 LNCS, (2016), 203–220. DOI:

1007/978-3-319-46478-7_13.

Liliana, Chae, J.H., Lee, J.J. and Lee, B.G. 2020. A

robust method for VR-based hand gesture recognition

using density-based CNN. Telkomnika

(Telecommunication Computing Electronics and

Control). (2020).

DOI:10.12928/TELKOMNIKA.v18i2.14747.

Liu, J., Shahroudy, A., Wang, G., Duan, L.Y. and Kot,

A.C. 2020. Skeleton-Based Online Action Prediction

Using Scale Selection Network. IEEE Transactions on

Pattern Analysis and Machine Intelligence. 42, 6 (2020),

–1467. DOI: 10.1109/TPAMI.2019.2898954.

Núñez, J.C., Cabido, R., Pantrigo, J.J., Montemayor, A.S.

and Vélez, J.F. 2018. Convolutional Neural Networks

and Long Short-Term Memory for skeleton-based human

activity and hand gesture recognition. Pattern

Recognition. 76, (2018), 80–94. DOI:

1016/j.patcog.2017.10.033.

Seemanthini, K. and Manjunath, S.S. 2018. Human

Detection and Tracking using HOG for Action

Recognition. Procedia Computer Science. 132, Iccids

(2018), 1317–1326. DOI: 10.1016/j.procs.2018.05.048.

Si, C., Chen, W., Wang, W., Wang, L. and Tan, T. 2019.

An attention enhanced graph convolutional lstm network

for skeleton-based action recognition. Proceedings of the

IEEE Computer Society Conference on Computer Vision

and Pattern Recognition. 2019-June, (2019), 1227–1236.

DOI: 10.1109/CVPR.2019.00132.

Si, C., Jing, Y., Wang, W., Wang, L. and Tan, T. 2020.

Skeleton-based action recognition with hierarchical

spatial reasoning and temporal stack learning network.

Pattern Recognition. 107, xxxx (2020). DOI:

1016/j.patcog.2020.107511.

Si, C., Jing, Y., Wang, W., Wang, L. and Tan, T. 2018.

Skeleton-Based Action Recognition with Spatial

Reasoning and Temporal Stack Learning. Lecture Notes

in Computer Science (including subseries Lecture Notes

in Artificial Intelligence and Lecture Notes in

Bioinformatics). 11205 LNCS, (2018), 106–121. DOI:

1007/978-3-030-01246-5_7.

Simonyan, K. and Zisserman, A. 2015. Very deep

convolutional networks for large-scale image

recognition. 3rd International Conference on Learning

Representations, ICLR 2015 - Conference Track

Proceedings. (2015), 1–14.

Subasi, A., Khateeb, K., Brahimi, T. and Sarirete, A.

Human activity recognition using machine

learning methods in a smart healthcare environment.

Elsevier Inc.

Vrigkas, M., Nikou, C. and Kakadiaris, I.A. 2015. A

review of human activity recognition methods. Frontiers

Robotics AI. 2, NOV (2015), 1–28. DOI:

3389/frobt.2015.00028.

Yan, S., Li, Z., Xiong, Y., Yan, H. and Lin, D. 2019.

Convolutional sequence generation for skeleton-based

action synthesis. Proceedings of the IEEE International

Conference on Computer Vision. 2019-Octob, (2019),

–4401. DOI: 10.1109/ICCV.2019.00449.

Zhou, K., Wu, T., Wang, C., Wang, J. and Li, C. 2020.

Skeleton Based Abnormal Behavior Recognition Using

Spatio-Temporal Convolution and Attention-Based

LSTM. Procedia Computer Science. 174, 2019 (2020),

–432. DOI: 10.1016/j.procs.2020.06.110.

Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L. and

Xie, X. 2016. Co-Occurrence feature learning for

skeleton based action recognition using regularized deep

LSTM networks. 30th AAAI Conference on Artificial

Intelligence, AAAI 2016. i (2016), 3697–3703.


Refbacks

  • There are currently no refbacks.


Jurnal telah terindeks oleh :