Hybrid CNN-transformer architecture for multi-class crop disease detection and severity assessment: CropHybrid-Net with benchmark evaluation on CropDisease-12

Abstract

Grain diseases lead to losses of 20-40% of the harvests each year, representing a threat to the food security of the world. Accurate and automated diagnosis of disease from remote picture taking would be key to prompt and directed interventions. Most current deep-learning approaches, however, are based on controlled lab images, on a single crop and ignore the assessment of disease severity. CropHybrid-Net is a three-branch ensemble architecture with ResNet-50, EfficientNet-B4 and Swin Transformer (Swin-T) that combines them using Efficient Channel Attention (ECA) fusion layer for simultaneous disease detection and severity estimation. CropDisease-12 is a benchmark dataset of 43200 images belonging to 12 classes representing four major crops (tomato, wheat, rice, and cotton) from PlantVillage and its own disease dataset collected in Yavatmal, Maharashtra, India. When evaluated on the CropDisease-12 test split, CropHybrid-Net outperforms all baselines tested such as standalone Swin-T (94.5%), ViT-B/16 (93.9%) and EfficientNet-B4 (93.7%), with the highest accuracy of 97.8%, and macro f1 score of 97.3%. The average value of AUC for the 12 classes is 0.995. In addition, a comprehensive literature review has been conducted, comprising of 62 papers (2015-2024), and grouped into five research streams: conventional machine learning, CNN-based methods, transfer learning, transformer-based methods, and multi-task severity approaches. The Grad-CAM visualizations are in line with the locations of biologically meaningful lesions. The framework proposed is deployed on common precision agriculture-edge of-use devices and achieves the goal of 39.3 ms per image, being relevant to smart precision agriculture applications.