Abstract
Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 843-849 |
| Number of pages | 7 |
| ISBN (Electronic) | 9781538666500 |
| DOIs | |
| Publication status | Published - Jan 16 2019 |
| Event | 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 - Miyazaki, Japan Duration: Oct 7 2018 → Oct 10 2018 |
Publication series
| Name | Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 |
|---|
Conference
| Conference | 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 |
|---|---|
| Country | Japan |
| City | Miyazaki |
| Period | 10/7/18 → 10/10/18 |
Fingerprint
Keywords
- Action Recognition
- Adaptive Resonance Theory
- Episodic Memory
- Unsupervised Learning
ASJC Scopus subject areas
- Information Systems
- Information Systems and Management
- Health Informatics
- Artificial Intelligence
- Computer Networks and Communications
- Human-Computer Interaction
Cite this
A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. / Kato, Kunpei; Chin, Wei Hong; Toda, Yuichiro; Kubota, Naoyuki.
Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 843-849 8616147 (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - A Multi-channel Episodic Memory Model for Human Action Learning and Recognition
AU - Kato, Kunpei
AU - Chin, Wei Hong
AU - Toda, Yuichiro
AU - Kubota, Naoyuki
PY - 2019/1/16
Y1 - 2019/1/16
N2 - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.
AB - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.
KW - Action Recognition
KW - Adaptive Resonance Theory
KW - Episodic Memory
KW - Unsupervised Learning
UR - http://www.scopus.com/inward/record.url?scp=85062222845&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062222845&partnerID=8YFLogxK
U2 - 10.1109/SMC.2018.00151
DO - 10.1109/SMC.2018.00151
M3 - Conference contribution
AN - SCOPUS:85062222845
T3 - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
SP - 843
EP - 849
BT - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
PB - Institute of Electrical and Electronics Engineers Inc.
ER -