Abstract:With the rapid growth of public security demand, the number of surveillance cameras is increasing, and video surveillance data is growing explosively. The traditional video monitoring system is difficult to understand and analyze such a large amount of data, so the intelligent video monitoring system came into being. As an interdisciplinary research field, intelligent video surveillance system abnormal behavior detection technology is facing great opportunities and challenges. In order to study the abnormal behavior detection algorithm of intelligent video surveillance system better, this paper combed the related research and divides different algorithms into energy based, clustering based, reconstruction based, inference based and depth learning based, and made comparative analysis and summarizes the branch research directions of various algorithms. Then, the common public data sets of abnormal behavior detection are introduced. Finally, this paper discussed the challenges of the current abnormal behavior detection algorithm and put forward the feasible research direction of the future intelligent video monitoring system abnormal behavior detection algorithm.