文章基本信息

标题：Automatic Classification of Cichlid Behaviors Using 3D Convolutional Residual Networks
本地全文：下载
作者：Lijiang Long ; Zachary V. Johnson ; Junyu Li 等
期刊名称：iScience
印刷版ISSN：2589-0042
出版年度：2020
卷号：23
期号：10
页码：1-28
DOI：10.1016/j.isci.2020.101591
语种：English
出版社：Elsevier
摘要：SummaryMany behaviors that are critical for survival and reproduction are expressed over extended time periods. The ability to inexpensively record and store large volumes of video data creates new opportunities to understand the biological basis of these behaviors and simultaneously creates a need for tools that can automatically quantify behaviors from large video datasets. Here, we demonstrate that 3D Residual Networks can be used to classify an array of complex behaviors in Lake Malawi cichlid fishes. We first apply pixel-based hidden Markov modeling combined with density-based spatiotemporal clustering to identify sand disturbance events. After this, a 3D ResNet, trained on 11,000 manually annotated video clips, accurately (>76%) classifies the sand disturbance events into 10 fish behavior categories, distinguishing between spitting, scooping, fin swipes, and spawning. Furthermore, animal intent can be determined from these clips, as spits and scoops performed during bower construction are classified independently from those during feeding.Graphical AbstractDisplay OmittedHighlights•A dataset of more than 14,000 annotated animal behavior videos was created•3D residual networks can be used to classify animal behavior•Different intents of similar behavioral actions can be distinguished•A working solution to study long-term behaviors was establishedPiscine Behavior; Zoology; Computer Science; Artificial Intelligence