Vol. 29 No. 1 (2026) Cover Image
Vol. 29 No. 1 (2026)

Published: March 20, 2026

Pages: 141-151

Original Article

YOLOv11 with spatial attention and preprocessing enhancements for accurate skin cancer classification

Aythem Khairi Kareem 1 Ahmed Adil Nafea 2 Afrig Aminuddin 3
1 Department of Heet Education, General Directorate of Education in Anbar, Ministry of Education, Iraq.
2 Department of Artificial Intelligence, College of Computer Science and Information Technology, University of Anbar, Ramadi, Anbar, Iraq.
3 Faculty of Computer Science, Universitas Amikom Yogyakarta Sleman, Indonesia.
History
  • Received: October 31, 2025
  • Revised: December 29, 2025
  • Accepted: January 18, 2026
  • Available Online: March 20, 2026
DOI

https://doi.org/10.29194/NJES.29010141

Abstract

This work suggests a Deep Learning (DL) architecture based on You Only Look Once YOLOv11 for Skin Cancer (SC) detection. The similarity between malignant and benign lesions makes visual inspection a failure to distinguish between them. To solve this problem, the proposed approach uses a 3-step pre-processing stage, namely hair removal, color normalization, and Contrast Limited Adaptive Histogram Equalization (CLAHE) contrasts, has been conducted to eliminate artifacts and improve image quality. Balanced data augmentation on the training set of the PROVe-AI dataset.  In this process, YOLOv11 with C3k2 module and C2PSA module showed significant results in optimized multi-scale feature collection and spatial interest. The experimental outcome demonstrates that the proposed model has a classification accuracy of 93.09% and led the baseline models, such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and Artificial Neural Network (ANN). The proposed optimized YOLOv11 architecture allows for skin cancer detection in a computationally efficient framework with promising preliminary results so that the proposed approach can be a beneficial Artificial intelligence (AI) tool for early diagnosis, particularly in a lack of high-tech medical facilities.

Keywords

References

  1. B. Ozdemir and I. Pacal, "An innovative deep learning framework for skin cancer detection employing ConvNeXtV2 and focal self‑attention mechanisms," Results Eng., vol. 25, p. 103692, 2025. https://doi.org/10.1016/j.rineng.2024.103692
  2. H. J. Mohammed, A. A. Nafea, H. K. Almulla, S. A. S. Aliesawi, and M. M. Al‑Ani, "An effective hybrid model for skin cancer detection using transfer learning," in Proc. 2023 16th Int. Conf. Developments eSystems Eng. (DeSE), IEEE, 2023, pp. 840-845. https://doi.org/10.1109/DeSE60595.2023.10468994
  3. R. Karthik, R. Menaka, S. Atre, J. Cho, and S. V. Easwaramoorthy, "A hybrid deep learning approach for skin cancer classification using Swin Transformer and dense group shuffle non‑local attention network," IEEE Access, 2024. https://doi.org/10.1109/ACCESS.2024.3485507
  4. A. Kumar, A. Vishwakarma, V. Bajaj, and S. Mishra, "Novel mixed domain hand‑crafted features for skin disease recognition using multiheaded CNN," IEEE Trans. Instrum. Meas., vol. 73, pp. 1-13, 2024. https://doi.org/10.1109/TIM.2024.3370772
  5. H. L. Gururaj, N. Manju, A. Nagarjun, V. N. M. Aradhya, and F. Flammini, "DeepSkin: a deep learning approach for skin cancer classification," IEEE Access, vol. 11, pp. 50205-50214, 2023. https://doi.org/10.1109/ACCESS.2023.3274848
  6. V. S. S. B. T. Sathvika et al., "Pipelined structure in the classification of skin lesions based on AlexNet CNN and SVM model with bi‑sectional texture features," IEEE Access, vol. 12, pp. 57366-57380, 2024. https://doi.org/10.1109/ACCESS.2024.3387533
  7. I. F. Jassam, A. A. Mukhlif, A. A. Nafea, M. A. Tharthar, and A. I. Khudhair, "A review of breast cancer histological image classification: challenges and limitations," Iraqi J. Comput. Sci. Math., vol. 6, no. 1, p. 1, 2025. https://doi.org/10.52866/2788-7421.1232
  8. A. A. Nafea, A.‑M. Manar, K. M. A. Alheeti, M. S. I. Alsumaidaie, and M. M. Al‑Ani, "A hybrid method of 1D‑CNN and machine learning algorithms for breast cancer detection," Baghdad Sci. J., vol. 21, no. 10, pp. 3333-3343, 2024. https://doi.org/10.21123/bsj.2024.9443
  9. Q. Yao, D. Zhuang, Y. Feng, Y. Wang, and J. Liu, "Accurate detection of brain tumor lesions from medical images based on improved YOLOv8 algorithm," IEEE Access, 2024. https://doi.org/10.1109/ACCESS.2024.3472039
  10. Y. Lin, C. Dong, H. Long, J. Liang, M. Liu, and Y. Liu, "Algorithm research for melanoma detection based on improved YOLOv11," in Proc. 2025 10th Int. Conf. Intell. Comput. Signal Process. (ICSP), IEEE, 2025, pp. 557-560. https://doi.org/10.1109/ICSP65755.2025.11086968
  11. A. T. Ibrahim, M. Abdullahi, A. F. D. Kana, M. T. Mohammed, and I. H. Hassan, "Categorical classification of skin cancer using a weighted ensemble of transfer learning with test time augmentation," Data Sci. Manag., vol. 8, no. 2, pp. 174-184, 2025. https://doi.org/10.1016/j.dsm.2024.10.002
  12. A. Pandey, M. S. Teja, P. Sahare, V. Kamble, M. Parate, and M. F. Hashmi, "Skin cancer classification using non‑local means denoising and sparse dictionary learning based CNN," J. Electr. Syst. Inf. Technol., vol. 11, no. 1, p. 36, 2024. https://doi.org/10.1186/s43067-024-00162-0
  13. R. A. Diptho and S. Basak, "Enhancing dermatological diagnosis through medical image analysis: how effective is YOLO11 compared to leading CNN models?," NDT, vol. 3, no. 2, p. 11, 2025. https://doi.org/10.3390/ndt3020011
  14. N.‑C. Huang, A. Mukundan, R. Karmakar, S. Syna, W.‑Y. Chang, and H.‑C. Wang, "Novel snapshot‑based hyperspectral conversion for dermatological lesion detection via YOLO object detection models," Bioengineering, vol. 12, no. 7, p. 714, 2025. https://doi.org/10.3390/bioengineering12070714
  15. H. Erbay, Y. M. Abulgasim, D. E. Özer, and F. Ertürk, "Enhancing multi‑class skin lesion diagnosis through ensemble learning of CNN and transformer architectures," Eng. Sci. Technol. Int. J., vol. 70, p. 102145, 2025. https://doi.org/10.1016/j.jestch.2025.102145
  16. A. Shaik, S. S. Dutta, I. M. Sawant, S. Kumar, A. Balasundaram, and K. De, "An attention‑based hybrid approach using CNN and BiLSTM for improved skin lesion classification," Sci. Rep., vol. 15, no. 1, p. 15680, 2025. https://doi.org/10.1038/s41598-025-00025-2
  17. P. Nanda, D. Rout, and S. Kumari, "Multi‑class skin cancer detection using CNN‑architecture based deep learning models," Procedia Comput. Sci., vol. 260, pp. 226-235, 2025. https://doi.org/10.1016/j.procs.2025.03.197
  18. A. Asriani, N. T. Lapatta, D. W. Nugraha, A. Amriana, and W. Wirdayanti, "Implementation of ResNet‑50‑based convolutional neural network for mobile skin cancer classification," J. Appl. Informatics Comput., vol. 9, no. 4, pp. 1577-1969, 2025. https://doi.org/10.30871/jaic.v9i4.9696
  19. S. Mondal, R. Bonthagorla, H. Kotapati, H. Tummala, and D. Khatua, "Deep learning for skin cancer: hybrid feature extraction with GoogleNet and MobileNet," in Proc. 2025 Int. Conf. Adv. Modern Age Technol. Health Eng. Sci. (AMATHE), IEEE, 2025, pp. 1-6. https://doi.org/10.1109/AMATHE65477.2025.11080891
  20. U. Ilyas, M. Fuzail, M. K. Abid, T. F. Khan, A. Naeem, and N. Aslam, "Multiclass skin cancer classification using ResNet and dermoscopic imaging," Spectr. Eng. Sci., vol. 3, no. 6, pp. 671-685, 2025.
  21. M. A. Marchetti et al., "Prospective validation of dermoscopy‑based open‑source artificial intelligence for melanoma diagnosis (PROVE‑AI study)," NPJ Digit. Med., vol. 6, no. 1, p. 127, 2023. https://doi.org/10.1038/s41746-023-00872-1
  22. R. Kasmi et al., "SharpRazor: automatic removal of hair and ruler marks from dermoscopy images," Skin Res. Technol., vol. 29, no. 4, p. e13203, 2023. https://doi.org/10.1111/srt.13203
  23. X. Shen, L. Wei, and S. Tang, "Dermoscopic image classification method using an ensemble of fine‑tuned convolutional neural networks," Sensors, vol. 22, no. 11, p. 4147, 2022. https://doi.org/10.3390/s22114147
  24. S. Barın and G. E. Güraksın, "An automatic skin lesion segmentation system with hybrid FCN‑ResAlexNet," Eng. Sci. Technol. Int. J., vol. 34, p. 101174, 2022. https://doi.org/10.1016/j.jestch.2022.101174
  25. S. Joseph and O. O. Olugbara, "Preprocessing effects on performance of skin lesion saliency segmentation," Diagnostics, vol. 12, no. 2, p. 344, 2022. https://doi.org/10.3390/diagnostics12020344
  26. F. Perez, C. Vasconcelos, S. Avila, and E. Valle, "Data augmentation for skin lesion analysis," in Proc. Int. Workshop Comput.‑Assisted Robot. Endoscopy, Springer, 2018, pp. 303-311. https://doi.org/10.1007/978-3-030-01201-4_33
  27. M. Alsaidi, M. T. Jan, A. Altaher, H. Zhuang, and X. Zhu, "Tackling the class imbalanced dermoscopic image classification using data augmentation and GAN," Multimed. Tools Appl., vol. 83, no. 16, pp. 49121-49147, 2024. https://doi.org/10.1007/s11042-023-17067-1
  28. A. Shah et al., "A comprehensive study on skin cancer detection using artificial neural network (ANN) and convolutional neural network (CNN)," Clin. eHealth, vol. 6, pp. 76-84, 2023. https://doi.org/10.1016/j.ceh.2023.08.002
  29. A. Ashtikar and S. Shastri, "A CNN model for skin cancer detection and classification by using image processing techniques," J. Sci. Res. Technol., pp. 251-263, 2025.
  30. S. S. Zareen, G. Sun, M. Kundi, S. F. Qadri, and S. Qadri, "Enhancing skin cancer diagnosis with deep learning: a hybrid CNN‑RNN approach," Comput. Mater. Contin., vol. 79, no. 1, 2024. https://doi.org/10.32604/cmc.2024.047418
  31. I. R. Putri, A. Amalia, and T. F. Abidin, "Hybrid algorithm using CNN and LSTM to improve performance in skin lesion image classification," in Proc. 2024 Ninth Int. Conf. Informatics Comput. (ICIC), IEEE, 2024, pp. 1-6. https://doi.org/10.1109/ICIC64337.2024.10956620
  32. H. K. Verma, N. D. Londhe, A. Kumar, and R. S. Sonawane, "YOLOv11 for psoriasis classification: enhancing feature extraction for accurate severity differentiation," in Proc. 2025 Int. Conf. Electron., AI Comput. (EAIC), IEEE, 2025, pp. 1-6. https://doi.org/10.1109/EAIC66483.2025.11101292
  33. R. Khanam and M. Hussain, "YOLOv11: an overview of the key architectural enhancements," arXiv Prepr. arXiv2410.17725, 2024.
  34. Q. Zhao and J. Zhu, "An improved YOLOv11 architecture with multi‑scale attention and spatial fusion for fine‑grained residual detection," Results Eng., p. 107061, 2025. https://doi.org/10.1016/j.rineng.2025.107061
  35. A. K. Kareem and K. M. A. Alheeti, "Hybrid approach for fall detection based on machine learning," in Proc. Int. Conf. New Trends Inf. Commun. Technol. Appl., Springer, 2021, pp. 111-130. https://doi.org/10.1007/978-3-030-93417-0_8
  36. K. M. A. Alheeti, A. Alzahrani, M. Alamri, A. K. Kareem, and D. Al_Dosary, "A comparative study for SDN security based on machine learning," Int. J. Interact. Mob. Technol., vol. 17, no. 11, 2023. https://doi.org/10.3991/ijim.v17i11.39065