System identification based on online variational bayes method and its application to reinforcement learning

Junichiro Yoshimoto; Shin Ishii; Masa Aki Sato

doi:10.1007/3-540-44989-2_16

System identification based on online variational bayes method and its application to reinforcement learning

Junichiro Yoshimoto, Shin Ishii, Masa Aki Sato

Research output: Chapter in Book/Report/Conference proceeding › Chapter

8 Citations (Scopus)

Abstract

In this article, we present an on-line variational Bayes (VB) method for the identification of linear state space models. The learning algorithm is implemented as alternate maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.

Original language	English
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Editors	Okyay Kaynak, Ethem Alpaydin, Erkki Oja, Lei Xu
Publisher	Springer Verlag
Pages	123-131
Number of pages	9
ISBN (Print)	3540404082, 9783540404088
DOIs	https://doi.org/10.1007/3-540-44989-2_16
Publication status	Published - 2003
Externally published	Yes

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	2714
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/3-540-44989-2_16

Cite this

Yoshimoto, J., Ishii, S., & Sato, M. A. (2003). System identification based on online variational bayes method and its application to reinforcement learning. In O. Kaynak, E. Alpaydin, E. Oja, & L. Xu (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 123-131). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2714). Springer Verlag. https://doi.org/10.1007/3-540-44989-2_16

Yoshimoto, Junichiro ; Ishii, Shin ; Sato, Masa Aki. / System identification based on online variational bayes method and its application to reinforcement learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). editor / Okyay Kaynak ; Ethem Alpaydin ; Erkki Oja ; Lei Xu. Springer Verlag, 2003. pp. 123-131 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{c001dcd5066949639a5cbcc5c43e93a3,

title = "System identification based on online variational bayes method and its application to reinforcement learning",

abstract = "In this article, we present an on-line variational Bayes (VB) method for the identification of linear state space models. The learning algorithm is implemented as alternate maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.",

author = "Junichiro Yoshimoto and Shin Ishii and Sato, {Masa Aki}",

year = "2003",

doi = "10.1007/3-540-44989-2_16",

language = "English",

isbn = "3540404082",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "123--131",

editor = "Okyay Kaynak and Ethem Alpaydin and Erkki Oja and Lei Xu",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

address = "Germany",

}

Yoshimoto, J, Ishii, S & Sato, MA 2003, System identification based on online variational bayes method and its application to reinforcement learning. in O Kaynak, E Alpaydin, E Oja & L Xu (eds), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2714, Springer Verlag, pp. 123-131. https://doi.org/10.1007/3-540-44989-2_16

System identification based on online variational bayes method and its application to reinforcement learning. / Yoshimoto, Junichiro; Ishii, Shin; Sato, Masa Aki.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). ed. / Okyay Kaynak; Ethem Alpaydin; Erkki Oja; Lei Xu. Springer Verlag, 2003. p. 123-131 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2714).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

TY - CHAP

T1 - System identification based on online variational bayes method and its application to reinforcement learning

AU - Yoshimoto, Junichiro

AU - Ishii, Shin

AU - Sato, Masa Aki

PY - 2003

Y1 - 2003

N2 - In this article, we present an on-line variational Bayes (VB) method for the identification of linear state space models. The learning algorithm is implemented as alternate maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.

AB - In this article, we present an on-line variational Bayes (VB) method for the identification of linear state space models. The learning algorithm is implemented as alternate maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.

UR - http://www.scopus.com/inward/record.url?scp=33646231363&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33646231363&partnerID=8YFLogxK

U2 - 10.1007/3-540-44989-2_16

DO - 10.1007/3-540-44989-2_16

M3 - Chapter

AN - SCOPUS:33646231363

SN - 3540404082

SN - 9783540404088

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 123

EP - 131

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

A2 - Kaynak, Okyay

A2 - Alpaydin, Ethem

A2 - Oja, Erkki

A2 - Xu, Lei

PB - Springer Verlag

ER -

Yoshimoto J, Ishii S, Sato MA. System identification based on online variational bayes method and its application to reinforcement learning. In Kaynak O, Alpaydin E, Oja E, Xu L, editors, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag. 2003. p. 123-131. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/3-540-44989-2_16

System identification based on online variational bayes method and its application to reinforcement learning

Abstract

Publication series

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this