Accurate species identification forms the foundation for our knowledge of the natural world. It is a prerequisite to citizen science, conservation, and public engagement in the natural world, but most people can name only a tiny fraction of the species around them. For a novice, classifying species to the correct family or genus can be daunting and some species are difficult for experts to identify. Even in a popular taxon such as birds, the availability of experts does not scale across broad geographic regions to engage broader communities of users. To overcome these limitations, we are developing Visipedia, which engages citizen scientists to gather media that are then identified by experts. These expert-identified and scientifically curated media are used to build computer vision and machine learning models to identify species in images. Methods. Our goal is to engage citizen scientists in building image datasets of various taxonomic groups beginning with birds. We are developing a system to collect data from citizen scientists in many formats, including images, audio recordings, videos, and observations for all taxa along with critical metadata. This material is archived in the Macaulay Library at the Cornell Lab of Ornithology. Citizen scientists are providing annotations to assist the computer learning. Experts create verified testing and training datasets. Results. On-demand computing infrastructure currently being tested provides real-time detection and classification algorithms across taxa. Classification services are delivered to the community, meeting the identification needs of end-users. These services support front-end interfaces such as Merlin Bird Photo ID, which helps the community identify 400 species of birds in images. In the process, we are growing and modernizing the Macaulay Library, an archive of more than 1.2 million scientifically curated images, audio recordings and videos of wildlife that have been collected since 1929. The user community for Merlin is also huge – to date more than 1 million people have used Merlin on their smartphones. Conclusions. Visipedia classification services will become a way to verify the identification of photos submitted to citizen science projects or images taken by remote sensors. By employing machine learning and computer vision we are developing a novel monitoring tool, capable of working in locations in which expert reviewers are unavailable.
