Adaptive actor-critic learning for the control of mobile robots by applying predictive models

Rafiuddin Syam, Keigo Watanabe, Kiyotaka Izumi

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.

Original languageEnglish
Pages (from-to)835-845
Number of pages11
JournalSoft Computing
Volume9
Issue number11
DOIs
Publication statusPublished - Nov 2005
Externally publishedYes

Fingerprint

Predictive Model
Mobile Robot
Mobile robots
Control Problem
Update
Model-based
Learning algorithms
Nonlinear systems
Wheels
Radial Basis Function Neural Network
Nonholonomic
Trajectory Tracking
Trajectories
Dynamical Model
Tracking Control
Neural networks
Feedback
Value Function
Wheel
Actors

Keywords

  • Actor-critic algorithms
  • Nonholonomic mobile robot
  • Predictive model
  • Temporal difference learning
  • Tracking control problem

ASJC Scopus subject areas

  • Computational Mechanics

Cite this

Adaptive actor-critic learning for the control of mobile robots by applying predictive models. / Syam, Rafiuddin; Watanabe, Keigo; Izumi, Kiyotaka.

In: Soft Computing, Vol. 9, No. 11, 11.2005, p. 835-845.

Research output: Contribution to journalArticle

@article{2660a017787b4416a4c2b46801e3ed70,
title = "Adaptive actor-critic learning for the control of mobile robots by applying predictive models",
abstract = "In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.",
keywords = "Actor-critic algorithms, Nonholonomic mobile robot, Predictive model, Temporal difference learning, Tracking control problem",
author = "Rafiuddin Syam and Keigo Watanabe and Kiyotaka Izumi",
year = "2005",
month = "11",
doi = "10.1007/s00500-004-0424-1",
language = "English",
volume = "9",
pages = "835--845",
journal = "Soft Computing",
issn = "1432-7643",
publisher = "Springer Verlag",
number = "11",

}

TY - JOUR

T1 - Adaptive actor-critic learning for the control of mobile robots by applying predictive models

AU - Syam, Rafiuddin

AU - Watanabe, Keigo

AU - Izumi, Kiyotaka

PY - 2005/11

Y1 - 2005/11

N2 - In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.

AB - In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.

KW - Actor-critic algorithms

KW - Nonholonomic mobile robot

KW - Predictive model

KW - Temporal difference learning

KW - Tracking control problem

UR - http://www.scopus.com/inward/record.url?scp=29544438065&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=29544438065&partnerID=8YFLogxK

U2 - 10.1007/s00500-004-0424-1

DO - 10.1007/s00500-004-0424-1

M3 - Article

VL - 9

SP - 835

EP - 845

JO - Soft Computing

JF - Soft Computing

SN - 1432-7643

IS - 11

ER -