A Multi-channel Episodic Memory Model for Human Action Learning and Recognition

Kunpei Kato, Wei Hong Chin, Yuichiro Toda, Naoyuki Kubota

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and Recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action Recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and Recognition.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages843-849
Number of pages7
ISBN (Electronic)9781538666500
DOIs
Publication statusPublished - Jan 16 2019
Event2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 - Miyazaki, Japan
Duration: Oct 7 2018Oct 10 2018

Publication series

NameProceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

Conference

Conference2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
CountryJapan
CityMiyazaki
Period10/7/1810/10/18

Keywords

  • Action Recognition
  • Adaptive Resonance Theory
  • Episodic Memory
  • Unsupervised Learning

ASJC Scopus subject areas

  • Information Systems
  • Information Systems and Management
  • Health Informatics
  • Artificial Intelligence
  • Computer Networks and Communications
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'A Multi-channel Episodic Memory Model for Human Action Learning and Recognition'. Together they form a unique fingerprint.

  • Cite this

    Kato, K., Chin, W. H., Toda, Y., & Kubota, N. (2019). A Multi-channel Episodic Memory Model for Human Action Learning and Recognition. In Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 (pp. 843-849). [8616147] (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2018.00151