A deep learning model for pollinator plant surveys

Buff-tailed bumblebee (Bombus terrestris) feeding on the nectar of Creeping thistle (Cirsium arvense) flowers © Damien Hicks

Authors Damien Hicks and Christoph Kratz introduce their team’s latest research demonstrating the use of machine learning for quadrat surveys to improve accessibility and resource efficiency of current methods for floral vegetation monitoring.

The nectar sugar contained in flowers is a key driver of pollinator abundance and diversity. Reduction of this food resource, primarily through habitat loss and changes in land-use, is indicated as a major cause of insect pollinator decline yet its measurement in the field and laboratory is laborious, and requires specialist skills.

To address this, an international partnership of Natural England, the University of Edinburgh, University of Orléans and Microsoft Corporation have trained a Convolutional Neural Network model to identify and count the nectar-producing flowers in surveyors’ photographs of flower-rich grasslands – which can then be used to calculate the weight of sugar available to pollinators.

qudrat + wildflowers
Left: Plant surveyor Pierre Ouvrard at a pollinator transect © Damien Hicks. Right: A flower-rich wildflower meadow, a suitable target for the species identification and flower-counting abilities of a Convolutional Neural Network. This is one of eight meadows planted near Edinburgh with funding from NatureScot, and one of many projects around the UK seeking to improve floral resources for pollinators. © Graham Stone

Building a deep learning model

One of the nice things about this research was that all of the equipment used was inexpensive. We took over 2,000 top-down images, each of around 1m2 of vegetation with no extraneous objects (e.g. quadrat frame, litter, feet!), then used open-source software (Python, R and VoTT) to create a dataset of 25,000 labelled ‘tags’ (i.e. bounding boxes) around the nectar-producing flower species in these images.

With the support of the Microsoft AI for Earth program we accessed a powerful virtual machine to run several Convolutional Neural Network (CNN) models. This method is designed to learn feature hierarchies and local patterns in a stochastic way, typically learning these in small two-dimensional frames of input images.

Assessing model performance

machine learning
An example of the visual output of the object detection model, given a new quadrat image containing Spear thistle (Cirsium vulgare), Ox-eye daisy (Leucanthemum vulgare) and Red campion (Silene dioica) © Damien Hicks

Our final model can be given new, model-unseen quadrat images, from which it outputs a visual display of its proposal regions and taxon classifications together with a spreadsheet of counts of detected taxa converted to nectar sugar mass per m2.  On a standard outdated laptop this takes approximately 6 seconds, and for a standard quadrat survey this tool could cut pollinator-plant survey time per stand of vegetation from hours to minutes.

To gauge the performance of the model we gathered another 50 new images from a range of countries, and compared the nectar sugar mass estimates made by the CNN to those of three human surveyors. Happily, these returned similar means and standard deviations and over half of the nectar sugar mass estimates made by the model fell within the absolute range of those of the human surveyors.

Applications of deep learning to ecological management

The project was originally conceived from discussions with the Results-Based Agricultural Payments team of Natural England.

As agri-environment schemes in the UK move from prescriptive to results-based, this approach provides an independent barometer for grassland management which is usable by both landowner and scheme administrator. This work is paralleled by increased management of greenspaces in cities, such as a new set of eight urban pollinator meadows planted this March along Edinburgh’s Firth of Forth shoreline.

machine learning_2+3
Examples of the visual outputs of the object detection model, given new quadrat images containing Ox-eye daisy (Leucanthemum vulgare, left) and Common hogweed (Heracleum sphondylium, right) © Damien Hicks

The CNN model can be adapted to visual estimations of other ecological resources such as winter bird food, floral pollen volume, insect infestation and tree flowering/fruiting, and by adjustment of classification threshold may show acceptable taxonomic differentiation for presence-absence surveys. The team are investigating further applications of this deep learning approach, including the monitoring of agricultural pests, bioacoustics of bumblebees and detection of artificial drains on peatland.

Read the full article: “Deep learning object detection to estimate the nectar sugar mass of flowering vegetation” in Issue 2:3 of Ecological Solutions and Evidence.

Publication supported by Department for Environment, Food & Rural Affairs (Defra) through their AER membership.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s