Objective According to the acne tongue features of traditional Chinese medicine,a scheme based on support vector machine (SVM) for recognition accuracy and recognition speed of tongue color and moss color were studied,in order to improve the accuracy and speed in objectification of tongue diagnosis in traditional Chinese medicine research. Methods In the first stage,the paper proposed tongue color,moss color and highlight area peer recognition avoiding highlight area being mistakenly identified as white moss. Cross validation kernel function,one against one (OAO) and voting method were used to train multi-class SVM classifier. Respectively,GridSearch,GA and PSO algorithm combined with cross validation for parameter optimization of SVM method,the best classification results were obtained. Finally,the proposed method with threshold method,clustering method and the DAG method was compared the recognition accuracy and recognition speed. Results This method in recognition accuracy and recognition speed of was higher than other methods. In this paper,the identification accuracy was 84.73%,higher than the DAG method which was 81.04%. Recognition speed was 0.5599s,better than that of DAG method as 1.6394s. Conclusions The recognition accuracy and recognition speed for the tongue color and moss color of this method increased greatly,which was important to assist clinical diagnosis,treatment and research.