A Multi-channel Episodic Memory Model for Human Action Learning and Recognition

Kunpei Kato, Wei Hong Chin, Yuichiro Toda, Naoyuki Kubota

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages843-849
Number of pages7
ISBN (Electronic)9781538666500
DOIs
Publication statusPublished - Jan 16 2019
Event2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 - Miyazaki, Japan
Duration: Oct 7 2018Oct 10 2018

Publication series

NameProceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

Conference

Conference2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
CountryJapan
CityMiyazaki
Period10/7/1810/10/18

Fingerprint

Episodic Memory
Learning
Short-Term Memory
Skeleton
Data storage equipment
Neurons
Human Body
Recognition (Psychology)
Action learning
Cues
Cluster Analysis
Joints

Keywords

  • Action Recognition
  • Adaptive Resonance Theory
  • Episodic Memory
  • Unsupervised Learning

ASJC Scopus subject areas

  • Information Systems
  • Information Systems and Management
  • Health Informatics
  • Artificial Intelligence
  • Computer Networks and Communications
  • Human-Computer Interaction

Cite this

Kato, K., Chin, W. H., Toda, Y., & Kubota, N. (2019). A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. In Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 (pp. 843-849). [8616147] (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2018.00151

A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. / Kato, Kunpei; Chin, Wei Hong; Toda, Yuichiro; Kubota, Naoyuki.

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 843-849 8616147 (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kato, K, Chin, WH, Toda, Y & Kubota, N 2019, A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. in Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018., 8616147, Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018, Institute of Electrical and Electronics Engineers Inc., pp. 843-849, 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018, Miyazaki, Japan, 10/7/18. https://doi.org/10.1109/SMC.2018.00151
Kato K, Chin WH, Toda Y, Kubota N. A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. In Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 843-849. 8616147. (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018). https://doi.org/10.1109/SMC.2018.00151
Kato, Kunpei ; Chin, Wei Hong ; Toda, Yuichiro ; Kubota, Naoyuki. / A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 843-849 (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018).
@inproceedings{4a33bce531f04f3f89c9bb598eb16ad2,
title = "A Multi-channel Episodic Memory Model for Human Action Learning and Recognition",
abstract = "Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.",
keywords = "Action Recognition, Adaptive Resonance Theory, Episodic Memory, Unsupervised Learning",
author = "Kunpei Kato and Chin, {Wei Hong} and Yuichiro Toda and Naoyuki Kubota",
year = "2019",
month = "1",
day = "16",
doi = "10.1109/SMC.2018.00151",
language = "English",
series = "Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "843--849",
booktitle = "Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018",

}

TY - GEN

T1 - A Multi-channel Episodic Memory Model for Human Action Learning and Recognition

AU - Kato, Kunpei

AU - Chin, Wei Hong

AU - Toda, Yuichiro

AU - Kubota, Naoyuki

PY - 2019/1/16

Y1 - 2019/1/16

N2 - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.

AB - Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.

KW - Action Recognition

KW - Adaptive Resonance Theory

KW - Episodic Memory

KW - Unsupervised Learning

UR - http://www.scopus.com/inward/record.url?scp=85062222845&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062222845&partnerID=8YFLogxK

U2 - 10.1109/SMC.2018.00151

DO - 10.1109/SMC.2018.00151

M3 - Conference contribution

AN - SCOPUS:85062222845

T3 - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

SP - 843

EP - 849

BT - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -