Abstract: Medical image analysis is an emerging technique for diagnosis, surgical and treatment planning. These image computation methods lead the physician to isolate the unwanted tissues of the human organs. Every one of the patients are expecting without cutting, without bleeding and day care operations. So these scanned images are analyzed before preceding the treatment process. In this study, the image segmentation and classification are done for various medical images and Gradient Vector Flow based Multi-classifier (GVFM) has proposed and implemented by Matlab coding. The medical image analysis parameters sensitivity, specificity and accuracy are calculated for multimodality medical images.