TY - GEN
T1 - A Multi-channel Episodic Memory Model for Human Action Learning and Recognition
AU - Kato, Kunpei
AU - Chin, Wei Hong
AU - Toda, Yuichiro
AU - Kubota, Naoyuki
N1 - Funding Information:
The authors would like to thank Tokyo Metropolitan Government for supporting the research by providing the Human Resources Fund.
Publisher Copyright:
© 2018 IEEE.
Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.
PY - 2019/1/16
Y1 - 2019/1/16
N2 - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.
AB - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.
KW - Action Recognition
KW - Adaptive Resonance Theory
KW - Episodic Memory
KW - Unsupervised Learning
UR - http://www.scopus.com/inward/record.url?scp=85062222845&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062222845&partnerID=8YFLogxK
U2 - 10.1109/SMC.2018.00151
DO - 10.1109/SMC.2018.00151
M3 - Conference contribution
AN - SCOPUS:85062222845
T3 - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
SP - 843
EP - 849
BT - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
Y2 - 7 October 2018 through 10 October 2018
ER -