Alexandra Tyukavina

University of Maryland / Committee on Earth Observation Satellites Working Group on Calibration and Validation LPV subgroup Land Cover Co-Chair

Varada Shevade 

International Center for Innovation in Geospatial Analytics and Earth Observation (Int. Geo Center) at the University of Maryland.

The Joint Workshop to develop community good practices for validating cropland and crop type maps in Beltsville, MD from Sep 12-14, 2023 was co-organized by the Committee on Earth Observation Satellites Working Group on Calibration and Validation Land Product Validation subgroup Land Cover focus area (CEOS LPV Land Cover) and the Group on Earth Observations Global Agricultural Monitoring Initiative Essential Agricultural Variables Working Group (GEOGLAM EAV) and supported by the International Center for Innovation in Geospatial Analytics and Earth Observation (Int. Geo Center) at the University of Maryland. It was hosted by the USDA/ARS at the National Agricultural Library, Beltsville, Maryland, USA.

The goals of the workshop were to draft community guidelines and other resources for accuracy assessment specific to cropland and crop type mapping, and to move towards consistent comparable, multi-scalar EAVs and publish these guidelines as a peer-reviewed paper.  These guidelines are expected to supplement the ongoing effort of the CEOS LPV Land Cover focus area to update the validation best practice protocol for global land cover maps and the GEOGLAM Essential Agricultural Variables documentation. The main discussion topics of the workshop included, Typology of mapping EAV’s for cropland extent and crop type; Sampling design; Response design - including reference data quality; and Performance metrics / map quality. 

Workshop Attendees at the National Agricultural Library, Beltsville, MD

After the welcome by Mike Cosh (CEOS LPV Co-Chair, USDA ARS) and Chris Justice (GEOGLAM Co-Chair, Int. Geo Center, UMD), the workshop began with opening remarks from Alexandra Tyukavina (UMD) and Sophie Bontemps (UCLouvain) providing the CEOS LPV perspective and Sven Gilliams (VITO) providing the GEOGLAM EAV perspective. This was followed by presentations from key cropland and crop type map producers focused on their map validation approaches. The presentations ranged from validation approaches used for global and regional maps to national and sub-national cropland / crop type maps. After an overview of the current approaches and state-of-the-art of cropland / crop type validation, the workshop focused on keynote presentations on the main topics before going into breakout group discussions on each of the topics. Sven Gilliams and Alyssa Whitcraft (UMD) provided an overview of the EAV mapping typology and current hierarchy, Xiaopeng Song (UMD) provided an overview presentation on Sampling Design, while Pierre Defourny (UCLouvain) and Sophie Bontemps (UCLouvain) gave keynote presentations on Response Design and Performance Metrics respectively. These were followed by breakout group discussions on each of the topics. 

Sophie Bontemps delivering her keynote presentation on Performance Metrics

The deliberations during the breakout sessions and the following report back sessions provided an overview of the main take-home messages from the community. The take-home messages focused on choosing probability sampling designs based on the objectives and the users or the audience that the maps are intended for, having unified definitions / typology, understanding the uniqueness of croplands when considering validation approaches, reiterating the obvious and essential basics for map validation and some minimum requirements for the various topics were agreed upon. Guidelines for sharing the data or developing standardized metadata were considered to be extremely important. The group discussed the importance of reviewing the uniqueness of croplands when suggesting guidelines for validation. Areas of interest that need further research were also discussed. Finally, the group agreed on a general structure for reporting the community good practices for validation which is planned as a high-impact but open source journal publication to reach the largest audience. The guidelines developed through this exercise will also supplement the current CEOS LPV efforts to develop land cover accuracy assessment guidelines and will be reported / linked to on the CEOS website.