文章基本信息

标题：Asymptotic Prediction Error Variance for Feedforward Neural Networks
本地全文：下载
作者：Magnus Malmström ; Isaac Skog ; Daniel Axehill 等
期刊名称：IFAC PapersOnLine
印刷版ISSN：2405-8963
出版年度：2020
卷号：53
期号：2
页码：1108-1113
DOI：10.1016/j.ifacol.2020.12.1310
语种：English
出版社：Elsevier
摘要：AbstractThe prediction uncertainty of a neural network is considered from a classical system identification point of view. To know this uncertainty is extremely important when using a network in decision and feedback applications. The asymptotic covariance of the internal parameters in the network due to noise in the observed dependent variables (output) and model class mismatch, i.e., the true system cannot be exactly described by the model class, is first surveyed. This is then applied to the prediction step of the network to get a closed form expression for the asymptotic, in training data information, prediction variance. Another interpretation of this expression is as the non-asymptotic Cramér-Rao Lower Bound. To approximate this expression, only the gradients and residuals, already computed in the gradient descent algorithms commonly used to train neural networks, are needed. Using a toy example, it is illustrated how the uncertainty in the output of a neural network can be estimated.
关键词：KeywordsNeural NetworksFeedforward NetworksUncertaintySystem IdentificationEstimation TheoryCramér-Rao BoundIdentification for ControlMachine Learning