Abstract: Skin diseases are more common than other diseases. These diseases may be caused by fungal infection, bacteria, allergy or viruses, etc. Despite being common its diagnosis is extremely difficult because of its complexities of skin tone, color, presence of hair. Treatment options for each type of disease are varying depending on the prognosis of a disease. Traditional method of initial clinical screening requires a visual diagnosing by specialized expertise but the cost of dermatologist to monitor is very high. The advancement of lasers and Photonics based medical technology has made it possible to diagnose the skin diseases much more quickly and accurately. But the cost of such diagnosis is also still limited and very expensive. There are often infections in skin due to viscus damages, therefore it’s necessary to spot these diseases as soon as possible. Thus, there is a need to develop an automated system of classification for the early diagnosis of severity of the disease and to prevent its spread. The proposed method is built on well-known convolutional neural network VGG16. The study focuses on improving the classification accuracy of skin disease diagnosing. The CNN Model is used to extract feature from the images and feature set is given as an input to machine learning algorithms like random forest, kNN and support vector machine. The simulation result for the classification of skin disease show the flexibility and effectiveness of the proposed system. However, SVM has achieved a higher classification accuracy of 98%.