文章基本信息

标题：An Actor-Critic Method for Simulation-Based Optimization
本地全文：下载
作者：Kuo Li ; Qing-Shan Jia ; Jiaqi Yan 等
期刊名称：IFAC PapersOnLine
印刷版ISSN：2405-8963
出版年度：2022
卷号：55
期号：11
页码：7-12
DOI：10.1016/j.ifacol.2022.08.040
语种：English
出版社：Elsevier
摘要：AbstractIn this work, we study simulation-based optimization, where the agent aims to select the best configuration from the design space with as few as possible iterations. Inspired by the success of deep reinforcement learning (DRL), we formulate the sampling process as policy searching and give a solving method from the perspective of policy iteration. Concretely, a surrogate model for predicting the performance of each configuration and a parameterized sampling policy are applied, which correspond to the critic and actor in actor-critic (AC) method, respectively. We further derive the updating rule and propose two algorithms for configuration selection in continuous and discrete design spaces, respectively. Finally, the algorithms are validated experimentally on 1) two toy examples to intuitively explain the principle and 2) two high-dimensional tasks to reveal the effectiveness in large-scale problems. The results show that the proposed algorithms can efficiently deal with large-scale problems and effectively eliminate sub-optimal configurations.
关键词：Keywordssimulation-based optimizationreinforcement learningpolicy searchingactor-criticpolicy iteration