Neutrosophic logic is a relatively new logic that is a generalization of fuzzy logic. In this paper, for the first time, neutrosophic logic is applied to the field of classifiers where a support vector machine (SVM) is adopted as the example to validate its feasibility and effectiveness. The proposed neutrosophic set is integrated into a reformulated SVM, and the performance of the obtained classifier N-SVM is evaluated under a region-based image categorization system. Images are first segmented by a hierarchical two-stage self-organizing map (HSOM), using color and texture features. A novel approach is proposed to select the training samples of HSOM based on homogeneity properties. A diverse density support vector machine (DD-SVM) framework is then applied to viewing an image as a bag of instances corresponding to the regions obtained from image segmentation. Each bag is mapped to a point in the new bag space, and the categorization is transformed to a classification problem. Then, the proposed N-SVM is used as the classifier in the new bag space. N-SVM treats samples differently according to the weighting function, and it helps to reduce the effects of outliers. Experimental results have demonstrated the validity and effectiveness of the proposed method which may find wide applications in the related areas.