Publication detail
Continuous Q-learning application
VĚCHET, S. KREJSA, J.
Czech title
Praktická aplikace Q-učení
English title
Continuous Q-learning application
Type
conference paper
Language
en
Original abstract
Standard algorithm of Q-Learning is limited by discrete states and actions and Q-function is usually represented as discrete table. To avoid this obstacle and extend the use of Q-learning for continuous states and actions the algorithm must be modified and such modification is presented in the paper. Straightforward way is to replace discrete table with suitable approximator. A number of approximators can be used, with respect to memory and computational requirements the local approximator is particularly favorable. We have used Locally Weighted Regression (LWR) algorithm. The paper discusses advantages and disadvantages of modified algorithm demonstrated on simple control task.
Czech abstract
Standardní algoritmus metody Q-učení je omezen používáním diskrétních stavů a akcí. V tomto případě je Q-funkce representována jako diskrétní tabulka. Metoda popisovaná v tomto příspěvku se snaží obejít problém s diskretizací tím, že je od počátku navržena jako spojitá. Diskrétní tabulka Q-hodnot je nahrazena vhodným aproximátorem. V tomto příspěvku hodnotíme výhody a nevýhody spojitého Q-učení oproti jeho diskrétní variantě.
English abstract
Standard algorithm of Q-Learning is limited by discrete states and actions and Q-function is usually represented as discrete table. To avoid this obstacle and extend the use of Q-learning for continuous states and actions the algorithm must be modified and such modification is presented in the paper. Straightforward way is to replace discrete table with suitable approximator. A number of approximators can be used, with respect to memory and computational requirements the local approximator is particularly favorable. We have used Locally Weighted Regression (LWR) algorithm. The paper discusses advantages and disadvantages of modified algorithm demonstrated on simple control task.
Keywords in English
Q-learning, Machine learning, Locall approximators
RIV year
2004
Released
10.05.2004
Publisher
Institute of Thermonechanics Academy of Sciences of the Czech Republic, Prague 2004
Location
Prague
ISBN
80-85918-88-9
Book
Engineering Mechanics 2004
Edition number
1
Pages count
2
BIBTEX
@inproceedings{BUT14018,
author="Stanislav {Věchet} and Jiří {Krejsa},
title="Continuous Q-learning application",
booktitle="Engineering Mechanics 2004",
year="2004",
month="May",
publisher="Institute of Thermonechanics Academy of Sciences of the Czech Republic, Prague 2004",
address="Prague",
isbn="80-85918-88-9"
}