Several studies in the past thirty years have shown that the technical quality and usability of soil maps, especially the ones produced through national soil surveys, has often been overestimated or neglected. Here we present a step-by-step guide to evaluate the adequacy and usability of (traditional) polygon-based soil maps following the 10 methodological steps. We have combined three methodological frameworks: (1) methodology to asses the adequacy of soil maps as described by Forbes et al. (1982) and Rossiter (2003); (2) methodology to assess the spatial data quality as described by Guptill and Morrison (1995); and (3) methodology to evaluate the usability of spatial databases as described by Hunter et al. (2003). For complete reference see:

Hengl T., Husnjak S. 2006. Assessing adequacy and usability of soil resource inventories: The National soil inventory in Croatia. Soil. Sci. Soc. Am. J., 70: 920-929.

The following guidelines will instruct you how to evaluate adequacy and usability for a set of soil maps accompanied with reports. This can help you produce a final report for a given soil resource inventory.

1. Lineage
Description: Lineage is usually contained in the metadata and survey reports. It should be rather technical to ensure that further data transformation, updating or merging with other GIS will be successful.
Things to consider: How exactly was the map produced?
Measures: Completeness and accuracy of the metadata

2. Consistency
Description: Consistency describes the heterogeneity of methods used to produce the data. For the applicability of a SIS, it is necessary that all area of interest is inventoried using the same soil survey methodology. This can be cross-checked, for example, by inspecting the density of profile observations in different parts of the area.
Things to consider: How consistent is the mapping methodology?
Measures: Density of profile observations per sub-area (map sheet), calibration error between different soil laboratories

3. Completeness
Description: Completeness is a measure of how many elements of the product are either not available or are available but do not comply with the prescribed standards. Incomplete products, even if only a small part of the area is missing, can eventually finish being highly unusable.
Things to consider: Does the map provide information for the whole area of interest?
Measures: Proportion of completed maps and reports

4. Effective scale
Description: The effective scale is the scale that is determined by the actual cartographic properties of the maps (size and detail of delineations, spatial accuracy of boundaries etc). Ideally, effective scale should match the nominal scale and should be the same over the whole area of interest.
Things to consider: What is the average decision support size? How accurate are soil boundaries?
Measures: Average size delineation, Shape complexity index, Spatial accuracy of soil boundaries

Spatial accuracy of soil boundaries and soil profiles: The procedure can be summarized as follows: (1) make control survey map using stereoscopic photo-interpretation and validate the boundaries on the field; (2) input the control survey map and the original soil map in the GIS; (3) identify the most contrasting adjacent units based on topographic or geomorphic features, e.g. slope breaks, change of general landscape; and (4) calculate area of disagreement between the two lines.

Figure: Example showing how to assess the spatial accuracy of soil boundaries: the original (left) and control survey (right) and area of disagreement between the most contrasting mapping units.

5. Attribute accuracy
Description: Attribute accuracy shows how well does the semantic information (soil attributes) showed in the map match the reality. It can be only inspected by doing field work, i.e. profile observations (ground truth). Due to high local variability of many soil properties, attribute accuracy of soil maps will never be as good as the accuracy of vegetation or land cover maps.
Things to consider: How well does the attribute data correspond to the reality?
Measures: Normalized error at randomly selected control points for a set of diagnostic properties

6. Thematic contrast
Description: A soil surveyor should try to delineate soil bodies in such a way that the contrast between adjacent SMUs is maximized, which reflects the idea of the maximum amount of information in a system. In the case of traditional soil maps, the key measure of the thematic map quality is the separability of attribute values between mapping units.
Things to consider: How distinct are adjacent polygons in the feature space?
Measures: Normalized variation within the soil mapping units, thematic overlap (probability) between adjacent polygons

Figure: Schematic example of the low (left) and high (right) thematic overlap between the adjacent (in this case sorted) soil mapping units (SMUs) for a given property (x).

7. Accuracy of legends
Description: Soil map legends usually show composition of soil mapping units following the designation of soil types done in the field. By doing independent profile observations, one can re-estimate the composition of legends and then run statistical tests to see what is the probability that this difference is by accident.
Things to consider: How well do map legends correspond to the reality?
Measures: Kappa statistics, Confusion index based on randomised control surveys

Fig: Example showing how to evaluate accuracy of soil map legend (left) using control survey with profile observation of soil types (right).

8. Integrity
Description: Integrity describes how flexible are the elements of the SIS for integration with other thematic GIS. For example, soil boundaries should match the water bodies, geomorphological units etc. If the target GIS is for example in raster format, then the SIS should also be available in the same format (equal level of detail, equal thematic accuracy).
Things to consider: How to integrate the data within an existing GIS? Is the data optimised for distribution and import?
Measures: Digital data format and the required amount of preprocessing

9. Popularity
Description: Sometimes even the highest quality products can be completely unusable. This usually happens because the soil data producers do not consider what is the background knowledge of users and how will they interpret the information contained in a SIS. Popularity can be increased by providing didactic, brief materials that explain the essence of the most uncommon terms, principles and similar.
Things to consider: How popular are the concepts and classification systems used? How to interpret the product?
Measures: Number of users compared to the potential number of users, degree of their satisfaction (based on the questionnaire)

10. Accessibility
Description: Even if the data if of high quality and well described it can still be only used by few. Especially for governmental organizations it might be important to facilitate the use of data by helping potential users to locate and purchase it quickly. The worst situation is if the data access and sharing policies are unclear and non transparent.
Things to consider: How to obtain data? How much does it cost?
Measures: Speed of access – time needed to obtain the data, price per km2 of the map

CASE STUDY: The National Soil Inventory in Croatia

In the case of Croatia, there is an extensive amount of data collected during different Soil survey projects. Unfortunately, this data is not fully utilised for Land use planning and resource conservation projects, as it is in more developed Western European countries. This applies especially at management and regional levels. Why is this data not fully used and should it be replaced with a new survey? In order to answer this, we needed to evaluate spatial and thematic quality of the surveys and investigate problems related to the overall usability. Following the above concepts, we have listed several specific research questions as our research objectives:
1. What is the overall quality/adequacy of Soil map of Croatia?
2. Can the existing data be used for Land Evaluation and Land Use Planning (agricultural extensions) at farm level?
3. How accurate are the attribute maps derived from the Soil map of Croatia?
4. How reliable is the data (soil profile observations, soil boundaries, SMUs)?
5. Why is soil data not used; what needs to be changed?
6. Should we aim at starting a new national survey project, and if yes, which methodology should we use?
We try to give answers on these questions supported by the data from control surveys. To evaluate and analyse the problems of the overall usability, we have also made a small survey to investigate: who are the current users of soil maps and digital databases in Croatia; how accurate and detailed information do these users need; and what type and format of the data do they prefer?

Husnjak, S; Rossiter, DG; Hengl, T; Milos, C. In press Soil inventory and soil classification in Croatia: historical review, current activities, and future directions. ISRIC Country Report.

DATASETS (coordinate system "Croatia"):

• Soil map of Croatia at scale 1:300K: - Polygon map (5320 polygons) with 65 SMUs and CYS names. *.shp (4 MB), © Soil Science Department in Zagreb.
• Legend of the 1:300K: Soil map Spreadsheet with the composition of 65 SMUs from the Soil map of Croatia at scale 1:300K. *.xls (49 KB), © Soil Science Department in Zagreb.
• Ten detailed soil profile descriptions used to validate the thematic accuracy of soil profiles.
• Soil map of Croatia at scale 1:1M: Polygon map (980 polygons) with 32 SMUs and 1986 FAO soil names. *.shp (1.7 MB), © Soil Science Department in Zagreb.
• 10 860 Soil profiles: Point map showing profiles from the 1:50K map sheets digitised at the Department of Photogrammetry in Zagreb. *.shp (222 KB),© Soil Science Department in Zagreb.
• Soil map sheets 1:50K: Map sheet quandrants covering Croatia with names. *.shp (559 KB),© Soil Science Department in Zagreb
• Profiles from the control survey: 120 profiles from the control survey. *.xls (656 KB)
• Profile statistics per SMUs Profile statistics for Clay %, pH and OM in top soil. Neighbouring polygons pairs derived in ILWIS using the highest length of boundary operation. *.xls (33 KB)
• Soil geographic database with app. 2350 profiles in whole Croatia: (source Martinović and Vranković, 1997). *.shp (4 MB), © Ministry of Civil Engineering and Environmental Protection, Croatia.

spatial-analyst.net/soilmaps:  1827   times