Abstract: In recent years, video surveillance systems have been commonly adopted around the world because of security concerns and their low hardware cost. Anomaly detection is one of the research areas in the field of video surveillance. In this study, different existing cluster based, EM clustering and classification based anomaly detection techniques in video surveillance are discussed. The video surveillance system includes background modeling, object detection, object tracking, activity recognition and classification. Recently, the machine learning based anomaly detection techniques plays a major role in the classification of the events into normal and abnormal events. The new approaches like the combination of Convolution Neural Network (CNN) and Recurrent Neural Network (RNN) and cascade deep learning are the robust algorithms for large datasets.