Abstract: To develop a new automatic moving object segmentation and classification system from the level-1 and level-2 sub bands, the Local Shape (LoS) and the Histogram oriented Gradients (HoG) features are extracted. These extracted features are then fused at the feature level Fusion using Salp Swarm optimization (FFSSO) algorithm. For convenience, the fused features are now called w-LoSHoG descriptor hereafter. Moreover, the feature extraction technique is applied on Least Enclosing Rectangle (LER) of the segmented object to increase the processing speed. The main intuition of this salp swarm algorithm relays on reducing the computational load of the proposed classifier by removing the repetitive and unrelated features from the feature vector. Also, increased training samples of similar shaped classes when applied on the classifier can generate the mis classification results. Thus, a new layered kernel based Support Vector Machine (k-SVM) classifier is developed by means of integrating the k-neural network classifier and layered SVM classifier. Because of the high dimensional features there occurs a difficulty in the application of single classifier. In order to ease the computational load, this multi classifier is integrated with shadow elimination technique to classify the object categories of intelligent transportation system such as motorcycle, bicycle, car and pedestrians.