Dynamic Action Space Handling Method for Reinforcement Learning Models


Sangchul Woo, Yunsick Sung, Journal of Information Processing Systems Vol. 16, No. 5, pp. 1223-1230, Oct. 2020  

10.3745/JIPS.02.0146
Keywords: Dance Tutorial System, Q-learning, Reinforcement Learning, Virtual Tutor
Fulltext:

Abstract

Recently, extensive studies have been conducted to apply deep learning to reinforcement learning to solve the state-space problem. If the state-space problem was solved, reinforcement learning would become applicable in various fields. For example, users can utilize dance-tutorial systems to learn how to dance by watching and imitating a virtual instructor. The instructor can perform the optimal dance to the music, to which reinforcement learning is applied. In this study, we propose a method of reinforcement learning in which the action space is dynamically adjusted. Because actions that are not performed or are unlikely to be optimal are not learned, and the state space is not allocated, the learning time can be shortened, and the state space can be reduced. In an experiment, the proposed method shows results similar to those of traditional Q-learning even when the state space of the proposed method is reduced to approximately 0.33% of that of Q-learning. Consequently, the proposed method reduces the cost and time required for learning. Traditional Q-learning requires 6 million state spaces for learning 100,000 times. In contrast, the proposed method requires only 20,000 state spaces. A higher winning rate can be achieved in a shorter period of time by retrieving 20,000 state spaces instead of 6 million.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from November 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.




Cite this article
[APA Style]
Woo, S. & Sung, Y. (2020). Dynamic Action Space Handling Method for Reinforcement Learning Models. Journal of Information Processing Systems, 16(5), 1223-1230. DOI: 10.3745/JIPS.02.0146.

[IEEE Style]
S. Woo and Y. Sung, "Dynamic Action Space Handling Method for Reinforcement Learning Models," Journal of Information Processing Systems, vol. 16, no. 5, pp. 1223-1230, 2020. DOI: 10.3745/JIPS.02.0146.

[ACM Style]
Sangchul Woo and Yunsick Sung. 2020. Dynamic Action Space Handling Method for Reinforcement Learning Models. Journal of Information Processing Systems, 16, 5, (2020), 1223-1230. DOI: 10.3745/JIPS.02.0146.