Performance Comparison of Facial Skin Type Classification Using the Segment Anything Model

Abstrak

The facial skin is the first area to often experience various problems. Knowing one’s skin type is an important step in choosing the right skincare routine, but it can be difficult to determine accurately without a specialist's help, which can be costly. To address this, a deep learning approach can be applied to help automatically classify skin types. In this study, several combinations of CNN, MobileNetV3, and SAM models were applied and compared for facial skin type classification. The dataset used, sourced from the figshare platform, consists of 2,250 facial images representing 5 skin types: normal, dry, oily, sensitive, and combination. The dataset was divided into three parts: training (80%), validation (10%), and testing (10%). Each model was evaluated using a confusion matrix, with accuracy, precision, recall, and F1-score metrics used to determine and compare model performance. The results show that the CNN performed worst, while the MobileNetV3-based CNN was the best-performing model, achieving an accuracy of 97%. Meanwhile, adding SAM did not improve performance and actually decreased accuracy. This study demonstrates that using MobileNetV3 without segmentation is more effective than adding SAM segmentation for facial skin type classification.