INTRODUCTION

Placer Mosquito and Vector Control District (District) is dedicated to assessing and advancing cutting-edge technologies to optimize mosquito surveillance strategies. This commitment is driven by the need to address the complex challenges presented by rapidly changing environments, including urban development (Duval et al. 2023, Fletcher et al. 2023) and agricultural practices (Wang et al. 2024). As urbanization and agricultural intensification reshape landscapes and ecosystems, traditional surveillance methods must evolve to accurately track mosquito populations and inform control strategies (Wheeler et al. 2022). Therefore, it is essential to explore and integrate innovative monitoring tools that can adapt to these shifting conditions, improve data collection, and enhance our ability to respond to mosquito population trends.

A central component of this effort is the accurate identification of mosquito species. Different species vary in their ecological niches, behaviors, and capacity to transmit pathogens (Ferraguti 2024). Members of the genus Culex are among the most important vectors of arboviruses worldwide, including West Nile virus (WNV), St. Louis encephalitis virus (SLE), and western equine encephalitis virus (WEE). Within this genus, Culex tarsalis Coquillett, is widely distributed across Western North America (Gorris et al. 2021), and serves as a key vector, particularly in agricultural and riparian habitats where transmission risk is elevated (Eisen et al. 2010). Precise identification of Culex species, including Cx. tarsalis, enables accurate pathogen testing, informs targeted insecticide applications, and is critical for mitigating the spread of vector-borne diseases (Sim et al. 2009).

Meeting these identification needs has been increasingly investigated by artificial intelligence (AI) that use convolutional neural networks (CNNs). They have the potential to transform mosquito identification tools from traditional trained staff and dichotomous keys, to image-based tech with minimal training and easy user interfaces. Early work investigated classic CNN architectures such as LeNet, AlexNet, and GoogleNet in mosquito identification (Motta et al. 2019), whereas other studies investigated deeper networks such as VGG-16, ResNet-50, and SqueezeNet for multispecies classification (Park et al. 2020). CNNs trained on photographs of adult mosquitoes now achieve human-level performance for key vectors like Aedes aegypti (Linnaeus) and Ae. albopictus (Skuse) under controlled conditions (Ong et al. 2021) and have further progressed into wing-based classifications that improve discrimination among morphologically similar species (Sauer et al. 2024).

Despite these advances, further studies on CNN-based mosquito identification are needed to address several challenges. First, most existing datasets are relatively limited in size, geographic scope, and species diversity, which limits model applications across different global contexts and populations. Second, although lab-reared mosquitoes and their associated images yield high accuracies, field collected mosquitoes introduces variability in specimen conditions that can degrade performance. Vector control surveillance requires accurate recognition across dozens of morphologically similar species and varying arrays of degradation.

Vectech (Baltimore, MD) is a public benefit corporation that provides AI technology-based solutions. They have been developing the Identification-X machine (IDX) (Fig. 1) with software that utilizes supervised machine learning to identify adult mosquitoes from images using CNNs (Brey et al. 2022, Gupta et al. 2024). Although developing accurate AI models requires significant training, the abundance of disease-carrying mosquito species highlights the practical value of this technology. In 2021, the District began a collaboration with Vectech to provide training material and testing for the IDX machine. Since that time, over 5,100 adult mosquitoes were identified and imaged, representing 24 distinct species from 5 genera.

View Figure

• Download as Powerpoint
• Download Figure

Our study evaluated the current species identification accuracy of the IDX system for a group of local mosquito species. We also compared potential discrepancies in identification between the first and second image perspectives. Two additional investigations were conducted: the first assessed the impact of progressive specimen degradation on the system’s ability to identify Cx. tarsalis females, whereas the second compared the identification accuracy and efficiency of individual vector control technicians to that of the IDX system. This work aimed to demonstrate that advanced AI-driven software has the potential to have practical applications for improving vector surveillance within mosquito control programs.

MATERIALS AND METHODS

To train the IDX machine, mosquito specimens were first identified by a trained vector control technician. The specimens were then carefully placed into wells within a specially designed IDX imaging tray, allowing for 12 specimens to be imaged at a time. After loading, the tray was placed into the machine, where images of the entire tray were captured. The machine analyzed these images to separate the specimens by insect type, genus, species, and sex. At this stage, the user enhances the dataset by assigning manual identifications (genus, species, and sex) to each specimen. The tray was then removed, inverted, and reinserted to acquire additional images from a different orientation. As more specimens are imaged and annotated, the overlap between user and machine identifications improves (following retraining as part of an active learning loop), thereby building a robust dataset to enhance the IDX machine’s identification accuracy.

The primary objective of this study was to evaluate the current accuracy of the IDX machine. This was assessed by imaging 10 wild-caught mosquito species, with at least 100 specimens per species, focusing predominantly on female mosquitoes. The species included Anopheles freeborni Aitken, An. franciscanus McCracken, Cx. tarsalis, Cx. pipiens Linnaeus, Cx. stigmatosoma Dyar, Cx. erythrothorax Dyar, Aedes vexans (Meigen), Ae. melanimon Dyar, Ae. aegypti (Linnaeus), and Ae. sierrensis (Ludlow). In the larger 2021–2024 dataset, images were excluded from the analysis if paired first and second images were not available. The analysis included both yearly species accuracy and the agreement between the first and second images. For each species and year, the percentage of specimens correctly identified in both images was calculated.

To determine at what point specimen degradation impacted the machines’ ability to correctly identify Cx. tarsalis, 100 laboratory-reared, Kern National Wildlife Refuge (KNWR) strain adult females were frozen at −80°C for several days. Specimens were then thawed and imaged with the IDX machine before being placed in a 3D-printed (Bambu Lab, X1C. Austin, Texas) tumbler (Fig. 2), with 5, 4 mm borosilica beads (Avantor Inc., Radnor, Pennsylvania). The tumbler was then placed on a 3D printed bottle roller and rotated at 370 rotations per min. The tumbler was removed from the bottle roller and specimen images were taken after 2, 4, 8, 12, 20, and 28 min. Percent of specimens identified accurately was calculated at each time point. The trial was discontinued when the IDX machine failed to correctly image >20% of the specimens. The latest IDX software update, V5.0.4, assesses both images to make a final identification determination referred to as “fused.” Fused identifications were used in this and the following analyses.

View Figure

• Download as Powerpoint
• Download Figure

To compare the identification accuracy and speed of VCTs with the imaging accuracy and speed of the IDX machine, 4 VCTs identified mosquitoes from 3 mixed batches of species each containing 50 specimens (Table 1). Mosquitoes used in the mixed batches were part of a pre-identified reference collection, and identifications were confirmed prior to the experiment. The identification time and the percent accuracy were recorded for each VCT. Each batch utilized different species mixtures to evaluate consistency across varying species compositions. Specimens were replaced with the same species, if damaged during the identification process. All data were exported from the IDX machine and assessed in Tableau (Salesforce, Inc., San Francisco, CA), an analytics platform geared towards data organization and visualization.

Table 1.Mosquito species diversity of batches 1-3 for vector control technician (VCT) identification accuracy trials vs. Vectech’s (Baltimore, MD) Identification-X (IDX) machine.

View Fullscreen

RESULTS

Of the 10 species analyzed, 6 had well-established image training datasets from previous years, with at least 200 existing images per species. These species had an accuracy of 96% or higher, with identification of Cx. tarsalis and Ae. vexans reaching 100% accuracy (Table 2). In contrast, 4 other species had fewer than 50 historical training images from the District, but 2 (Ae. aegypti and Cx. erythrothorax) likely contained enough training images from other districts and institutions to have a moderate accuracy of 95.4% and 60.3%, respectively. At the time of testing both An. franciscanus and Cx. stigmatosoma were not yet included in the deployed identification algorithm for our region and were thus not appropriately scored for accuracy by IDX.

Table 2.Sample size and identification accuracy for 10 local mosquito species identified by Vectech’s (Baltimore, MD) Identification-X (IDX) machine.

View Fullscreen

Identification accuracy for most mosquito species improved between 2021 and 2024 (Fig. 3). The District provided over 900 Cx. tarsalis images for IDX training from 2021 to 2024, with identification accuracy showing steady but substantial increases over time. In 2022 and 2023, the District provided images for Ae. sierrensis, but the overall accuracy remained at 0% until after a software update resulted in increased identification accuracy for this species.

View Figure

• Download as Powerpoint
• Download Figure

When comparing the identification accuracy of the first and second images for 2024, there was a 100% accuracy across all species. In contrast, specimens imaged in 2021 had many mismatched identifications, resulting in an agreement between 56.3% and 77.5% (Fig. 4). This indicated that software updates and increasing images resulted in improved agreement between images of the same specimens.

View Figure

• Download as Powerpoint
• Download Figure

Results indicate that as specimens experienced progressive loss of scales, legs, and other morphological features within the tumbler (Fig. 5), the accuracy of the IDX machine in identifying specimens declined after 4 min (Fig. 6). By 8 min, the identification accuracy for Cx. tarsalis was 95%, and by 28 min, it was reduced to 12% (Fig. 6). As correct species identification decreased, the IDX system increasingly assigned alternative identifications, often misclassifying specimens as morphologically similar species such as Cx. pipiens or Cx. tritaeniorhynchus Giles. In many cases, specimens were categorized as “unknown.”

View Figure

• Download as Powerpoint
• Download Figure

View Figure

• Download as Powerpoint
• Download Figure

Results of the comparisons between 4 VCTs and the IDX machine regarding identification speed and accuracy indicated that the IDX machine may be as accurate and as efficient as a vector control technician with 22 months or less of experience (VCT I) but was only half as fast as a senior experienced technician (VCTII) with 144 months of experience (Table 3).

Table 3.Identification time and accuracy for 4 vector control technicians (VCT) with between 7 and 144 months of experience vs. Vectech’s (Baltimore, MD) Identification-X (IDX) machine when identifying batches of 50 mosquitoes including 6-8 species.

View Fullscreen

DISCUSSION

The data presented here indicate that increasing the number of training specimens significantly improved the identification accuracy of the IDX machine between 2021 and 2024. This improvement is likely attributed to the model’s exposure to a broader range of morphological traits and natural variation within species. Convolutional neural networks perform best when trained on large and diverse datasets, as greater data availability allows the model to learn a stronger sense of appropriate features (Barbedo 2018). In our study, increases in accuracy were further supported by the inclusion of additional mosquito images from other vector control agencies and research institutions (Gupta et al. 2024). Because of the subtlety of certain morphological traits, the learning model may require larger sets of training images to accurately differentiate between certain species, such as Cx. erythrothorax and Cx. pipiens. Previous research has reported similar challenges: Goodwin et al. (2021) noted reduced identification accuracy within the Culex genus, particularly for Cx. salinarius Coquillett, whereas a study by Nolte et al. (2024) focused exclusively on morphologically similar species within the Aedes genus and reported comparable misclassification errors, despite differences in photography equipment.

Results also indicated that updates to the system’s algorithm were a crucial factor in improving overall identification accuracy, as was seen with Ae. sierrensis, which was initially imaged with an accuracy of 0% in 2023 with update v1.5.8 and 98.7% in 2024 with update v5.0.4. Additionally, An. franciscanus and Cx. stigmatosoma were 2 species in this examination that had fewer than the recommended number of training images (<100) and required further algorithmic updates to improve performance, leading to low identification percentages. During the initial phase of data capture, algorithm accuracy percentages are not immediately established. Instead, captured images undergo a preliminary evaluation to assess both image quality and identification accuracy prior to the deployment in the IDX software, and the collection of additional specimen images. At this early stage of species-specific data acquisition, laboratory-reared specimens are often utilized, as they are more likely to exhibit complete scale patterns and well-defined identifying characteristics, which facilitate more reliable model training.

To increase current accuracy and to build larger species lists, additional specimens are required, not only locally, but nationwide to capture the diversity within and among species. Although it is important to provide quality specimens for training purposes, all wild-caught specimens will experience some level of degradation because of aging processes in the field, or through mechanical degradation in collection traps (Cansado-Utrilla et al., 2020; Sauer et al. 2024). Rarely do vector management staff deal with pristine specimens. The amount of specimen degradation can substantially reduce mosquito identification accuracy for both vector control technicians and AI-based systems such as the IDX machine. Mosquito identification on a species level often involves using scale patterns, which rub off easily through natural and mechanical processes. Although prior studies have noted that degradation and morphological variability introduce noise and hinder classification accuracy (Goodwin et al. 2021), our findings provide direct evidence that varying states of degradation, particularly morphological damage and the loss of key features, strongly decrease identification performance. For this reason, it is valuable to provide the IDX with damaged mosquitos so that it may train on these more difficult specimens. Our study provided preliminary data to demonstrate what the identification outcome was for degraded specimens, but further work is needed to clarify the level of accuracy sufficient for operational purposes. Reliable identification of vector species is a key component of effective surveillance and control programs. The IDX system’s ability to classify uncertain specimens as “unknown species” provides an added layer of quality control, which may enhance user confidence by minimizing the risk of false identifications and encouraging further expert review.

Additionally, we compared the identification speed and accuracy of the IDX machine with that of trained vector control technicians to evaluate its practicality in day-to-day surveillance. Our findings indicated that the IDX machine performed comparably to human technicians with approximately 22 months or less of field experience. Although VCTs undergo less intensive training than taxonomic experts, they still require substantial experience to accurately and consistently identify the diverse range of material encountered in the field. Similar patterns have been reported in another study evaluating CNNs for insect identification. In a study by Ärje et al. (2020) human experts working with physical specimens and microscopes achieved lower error rates than their CNN counterparts, although the authors noted that CNN performance remained within the variability observed among experts. Although that study did not directly assess time required for identification, the study highlighted the trade-offs between accuracy and scalability that also apply to the deployment of automated systems such as IDX.

Our research has shown that as the system currently stands, the IDX system offers several practical applications for mosquito surveillance, including accurate species identifications and reasonable processing times. Looking ahead, its integration into surveillance programs has the potential for enhancing scalability and responsiveness, particularly during busy periods where trained staff may be strained by workload demands. For districts that rely on seasonal staff with limited experience in mosquito identification, the IDX could provide consistent, accurate results, reducing training time and maintaining data quality. Continued software development and refinement will further improve the system’s accuracy and efficiency while expanding datasets to include wider geographic areas and more species diversity.