An adaptive actor-critic algorithm with multi-step simulated experiences for controlling nonholonomic mobile robots

Rafiuddin Syam, Keigo Watanabe, Kiyotaka Izumi

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

In this paper, we propose a new algorithm of an adaptive actor-critic method with multi-step simulated experiences, as a kind of temporal difference (TD) method. In our approach, the TD-error is composed of two value- functions and m utility functions, where m denotes the number of multi-steps in which the experience should be simulated. The value-function is constructed from the critic formulated by a radial basis function neural network (RBFNN), which has a simulated experience as an input, generated from a predictive model based on a kinematic model. Thus, since our approach assumes that the model is available to simulate the m-step experiences and to design a controller, such a kinematic model is also applied to construct the actor and the resultant model based actor (MBA) is also regarded as a network, i.e., it is just viewed as a resolved velocity control network. We implement this approach to control nonholonomic mobile robot, especially in a trajectory tracking control problem for the position coordinates and azimuth. Some simulations show the effectiveness of the proposed method for controlling a mobile robot with two-independent driving wheels.

Original languageEnglish
Pages (from-to)81-89
Number of pages9
JournalSoft Computing
Volume11
Issue number1
DOIs
Publication statusPublished - Jan 2007
Externally publishedYes

Fingerprint

Nonholonomic
Mobile Robot
Mobile robots
Kinematic Model
Value Function
Model-based
Kinematics
Radial Basis Function Neural Network
Trajectory Tracking
Azimuth
Predictive Model
Tracking Control
Velocity control
Utility Function
Wheel
Difference Method
Control Problem
Denote
Wheels
Controller

Keywords

  • Actor-critic algorithms
  • Kinematic model
  • Multi-step prediction
  • Nonholonomic mobile robot
  • Nonlinear predictive model
  • Simulated experience

ASJC Scopus subject areas

  • Computational Mechanics

Cite this

An adaptive actor-critic algorithm with multi-step simulated experiences for controlling nonholonomic mobile robots. / Syam, Rafiuddin; Watanabe, Keigo; Izumi, Kiyotaka.

In: Soft Computing, Vol. 11, No. 1, 01.2007, p. 81-89.

Research output: Contribution to journalArticle

@article{ced467b7e2214ff89d03d461d2c4f146,
title = "An adaptive actor-critic algorithm with multi-step simulated experiences for controlling nonholonomic mobile robots",
abstract = "In this paper, we propose a new algorithm of an adaptive actor-critic method with multi-step simulated experiences, as a kind of temporal difference (TD) method. In our approach, the TD-error is composed of two value- functions and m utility functions, where m denotes the number of multi-steps in which the experience should be simulated. The value-function is constructed from the critic formulated by a radial basis function neural network (RBFNN), which has a simulated experience as an input, generated from a predictive model based on a kinematic model. Thus, since our approach assumes that the model is available to simulate the m-step experiences and to design a controller, such a kinematic model is also applied to construct the actor and the resultant model based actor (MBA) is also regarded as a network, i.e., it is just viewed as a resolved velocity control network. We implement this approach to control nonholonomic mobile robot, especially in a trajectory tracking control problem for the position coordinates and azimuth. Some simulations show the effectiveness of the proposed method for controlling a mobile robot with two-independent driving wheels.",
keywords = "Actor-critic algorithms, Kinematic model, Multi-step prediction, Nonholonomic mobile robot, Nonlinear predictive model, Simulated experience",
author = "Rafiuddin Syam and Keigo Watanabe and Kiyotaka Izumi",
year = "2007",
month = "1",
doi = "10.1007/s00500-006-0054-x",
language = "English",
volume = "11",
pages = "81--89",
journal = "Soft Computing",
issn = "1432-7643",
publisher = "Springer Verlag",
number = "1",

}

TY - JOUR

T1 - An adaptive actor-critic algorithm with multi-step simulated experiences for controlling nonholonomic mobile robots

AU - Syam, Rafiuddin

AU - Watanabe, Keigo

AU - Izumi, Kiyotaka

PY - 2007/1

Y1 - 2007/1

N2 - In this paper, we propose a new algorithm of an adaptive actor-critic method with multi-step simulated experiences, as a kind of temporal difference (TD) method. In our approach, the TD-error is composed of two value- functions and m utility functions, where m denotes the number of multi-steps in which the experience should be simulated. The value-function is constructed from the critic formulated by a radial basis function neural network (RBFNN), which has a simulated experience as an input, generated from a predictive model based on a kinematic model. Thus, since our approach assumes that the model is available to simulate the m-step experiences and to design a controller, such a kinematic model is also applied to construct the actor and the resultant model based actor (MBA) is also regarded as a network, i.e., it is just viewed as a resolved velocity control network. We implement this approach to control nonholonomic mobile robot, especially in a trajectory tracking control problem for the position coordinates and azimuth. Some simulations show the effectiveness of the proposed method for controlling a mobile robot with two-independent driving wheels.

AB - In this paper, we propose a new algorithm of an adaptive actor-critic method with multi-step simulated experiences, as a kind of temporal difference (TD) method. In our approach, the TD-error is composed of two value- functions and m utility functions, where m denotes the number of multi-steps in which the experience should be simulated. The value-function is constructed from the critic formulated by a radial basis function neural network (RBFNN), which has a simulated experience as an input, generated from a predictive model based on a kinematic model. Thus, since our approach assumes that the model is available to simulate the m-step experiences and to design a controller, such a kinematic model is also applied to construct the actor and the resultant model based actor (MBA) is also regarded as a network, i.e., it is just viewed as a resolved velocity control network. We implement this approach to control nonholonomic mobile robot, especially in a trajectory tracking control problem for the position coordinates and azimuth. Some simulations show the effectiveness of the proposed method for controlling a mobile robot with two-independent driving wheels.

KW - Actor-critic algorithms

KW - Kinematic model

KW - Multi-step prediction

KW - Nonholonomic mobile robot

KW - Nonlinear predictive model

KW - Simulated experience

UR - http://www.scopus.com/inward/record.url?scp=33747593885&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33747593885&partnerID=8YFLogxK

U2 - 10.1007/s00500-006-0054-x

DO - 10.1007/s00500-006-0054-x

M3 - Article

VL - 11

SP - 81

EP - 89

JO - Soft Computing

JF - Soft Computing

SN - 1432-7643

IS - 1

ER -