文章基本信息

标题：A Flexible Behavioral Learning System with Modular Neural Networks
本地全文：下载
作者：Johane Takeuchi ; Osamu Shouno ; Hiroshi Tsujino 等
期刊名称：人工知能学会論文誌
印刷版ISSN：1346-0714
电子版ISSN：1346-8030
出版年度：2012
卷号：27
期号：2
页码：92-102
DOI：10.1527/tjsai.27.92
出版社：The Japanese Society for Artificial Intelligence
摘要：Future robots/agents will perform situated behaviors for each user. Flexible behavioral learning is required for coping with diverse and unexpected users' situations. Unexpected situations are usually not tractable for machine learning systems that are designed for pre-defined problems. In order to realize such a flexible learning system, we were trying to create a learning model that can function in several different kinds of state transitions without specific adjustments for each transition as a first step. We constructed a modular neural network model based on reinforcement learning. We expected that combining a modular architecture with neural networks could accelerate the learning speed of neural networks. The inputs of our neural network model always include not only observed states but also memory information for any transition. In pure Markov decision processes, memory information is not necessary, rather it can lead to lower performance. On the other hand, partially observable conditions require memory information to select proper actions. We demonstrated that the new learning model could actually learn those multiple kinds of state transitions with the same architectures and parameters, and without pre-designed models of environments. This paper describes the performances of constructed models using probabilistically fluctuated Markov decision processes including partially observable conditions. In the test transitions, the observed state probabilistically fluctuated. The new learning model could function in those complex transitions. In addition, the learning speeds of our model are comparable to a reinforcement learning algorithm implemented with a pre-defined and optimized table-representation of states.
关键词：reinforcement learning ; neural network ; modular structure