A solving method for MDPs by minimizing variational free energy

Junichiro Yoshimoto; Shin Ishii

doi:10.1109/IJCNN.2004.1380884

A solving method for MDPs by minimizing variational free energy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this article, we propose a novel approach to acquiring the optimal policy for a continuous Markov decision process. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. A good policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kaiman filtering and smoothing techniques. Even in non-linear problems, a semioptimal policy can be obtained by Monte Carlo technique with a Gaussian process method.

Original language	English
Title of host publication	2004 IEEE International Joint Conference on Neural Networks - Proceedings
Pages	1817-1822
Number of pages	6
DOIs	https://doi.org/10.1109/IJCNN.2004.1380884
Publication status	Published - 2004
Externally published	Yes
Event	2004 IEEE International Joint Conference on Neural Networks - Proceedings - Budapest, Hungary Duration: 25-07-2004 → 29-07-2004

Publication series

Name	IEEE International Conference on Neural Networks - Conference Proceedings
Volume	3
ISSN (Print)	1098-7576

Conference

Conference	2004 IEEE International Joint Conference on Neural Networks - Proceedings
Country/Territory	Hungary
City	Budapest
Period	25-07-04 → 29-07-04

All Science Journal Classification (ASJC) codes

Software

Access to Document

10.1109/IJCNN.2004.1380884

Cite this

@inproceedings{fc9c5a05a2974190addc20dca2e011a4,

title = "A solving method for MDPs by minimizing variational free energy",

abstract = "In this article, we propose a novel approach to acquiring the optimal policy for a continuous Markov decision process. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. A good policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kaiman filtering and smoothing techniques. Even in non-linear problems, a semioptimal policy can be obtained by Monte Carlo technique with a Gaussian process method.",

author = "Junichiro Yoshimoto and Shin Ishii",

year = "2004",

doi = "10.1109/IJCNN.2004.1380884",

language = "English",

isbn = "0780383591",

series = "IEEE International Conference on Neural Networks - Conference Proceedings",

pages = "1817--1822",

booktitle = "2004 IEEE International Joint Conference on Neural Networks - Proceedings",

note = "2004 IEEE International Joint Conference on Neural Networks - Proceedings ; Conference date: 25-07-2004 Through 29-07-2004",

}

Yoshimoto, J & Ishii, S 2004, A solving method for MDPs by minimizing variational free energy. in 2004 IEEE International Joint Conference on Neural Networks - Proceedings. IEEE International Conference on Neural Networks - Conference Proceedings, vol. 3, pp. 1817-1822, 2004 IEEE International Joint Conference on Neural Networks - Proceedings, Budapest, Hungary, 25-07-04. https://doi.org/10.1109/IJCNN.2004.1380884

TY - GEN

T1 - A solving method for MDPs by minimizing variational free energy

AU - Yoshimoto, Junichiro

AU - Ishii, Shin

PY - 2004

Y1 - 2004

N2 - In this article, we propose a novel approach to acquiring the optimal policy for a continuous Markov decision process. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. A good policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kaiman filtering and smoothing techniques. Even in non-linear problems, a semioptimal policy can be obtained by Monte Carlo technique with a Gaussian process method.

AB - In this article, we propose a novel approach to acquiring the optimal policy for a continuous Markov decision process. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. A good policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kaiman filtering and smoothing techniques. Even in non-linear problems, a semioptimal policy can be obtained by Monte Carlo technique with a Gaussian process method.

UR - http://www.scopus.com/inward/record.url?scp=10844269629&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=10844269629&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2004.1380884

DO - 10.1109/IJCNN.2004.1380884

M3 - Conference contribution

AN - SCOPUS:10844269629

SN - 0780383591

T3 - IEEE International Conference on Neural Networks - Conference Proceedings

SP - 1817

EP - 1822

BT - 2004 IEEE International Joint Conference on Neural Networks - Proceedings

T2 - 2004 IEEE International Joint Conference on Neural Networks - Proceedings

Y2 - 25 July 2004 through 29 July 2004

ER -

A solving method for MDPs by minimizing variational free energy

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this