A study of reinforcement learning with knowledge sharing for distributed autonomous system

Kazuyuki Ito, Akio Gofuku, Yoshiaki Imoto, Mitsuo Takeshita

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Reinforcement learning is one of effective controller for autonomous robots. Because it does not need priori knowledge and behaviors to complete given tasks are obtained automatically be repeating trial and error. However a large number of trials are required to realize complex tasks. So the task that can be obtained using the real robot is restricted to simple ones. Considering these points, various methods that prove the learning cost of reinforcement learning have been proposed. In the method that uses priori knowledge, the methods lose the autonomy that is most important feature of reinforcement learning in applying it to the robots. In the Dyna-Q, that is one of simple and effective reinforcement learning architecture integrating online planning, a model of environment is learned from real experience and by utilizing the model to learn, the learning time is decreased. In this architecture, the autonomy is held, however the model depends on the task, so acquired knowledge of environment cannot be reused to other tasks. In the real world, human beings can learn various behaviors to complete complex tasks without priori knowledge of the tasks. We can try to realize the task in our image without moving our body. After the training in the image, by trying to the real environment, we save time to learn. It means that we have model of environment and we utilize the model to learn. We consider that the key ability that makes the learning process faster is construction of environment model and utilization of it. In this paper, we have proposed a method to obtain an environment model that is independent of the task. And by utilizing the model we have decreased learning time. We consider distributed autonomous agents, and we show that the environment model is constructed quickly by sharing the experience of each agent, even when each agent has own independent task. To demonstrate the effectiveness of the proposed method, we have applied the method to the Q-learning and simulations of a puddle world are carried out. As a result effective behaviors have been obtained quickly.

Original languageEnglish
Title of host publicationProceedings - 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation
Subtitle of host publicationComputational Intelligence in Robotics and Automation for the New Millennium
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1120-1125
Number of pages6
ISBN (Electronic)0780378660
DOIs
Publication statusPublished - 2003
Event2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation, CIRA 2003 - Kobe, Japan
Duration: Jul 16 2003Jul 20 2003

Publication series

NameProceedings of IEEE International Symposium on Computational Intelligence in Robotics and Automation, CIRA
Volume3

Other

Other2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation, CIRA 2003
CountryJapan
CityKobe
Period7/16/037/20/03

ASJC Scopus subject areas

  • Computational Mathematics

Fingerprint Dive into the research topics of 'A study of reinforcement learning with knowledge sharing for distributed autonomous system'. Together they form a unique fingerprint.

  • Cite this

    Ito, K., Gofuku, A., Imoto, Y., & Takeshita, M. (2003). A study of reinforcement learning with knowledge sharing for distributed autonomous system. In Proceedings - 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation: Computational Intelligence in Robotics and Automation for the New Millennium (pp. 1120-1125). [1222154] (Proceedings of IEEE International Symposium on Computational Intelligence in Robotics and Automation, CIRA; Vol. 3). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CIRA.2003.1222154