How to calculate

a confusion matrix

To assess the accuracy of an image classification, it is common practice to create a confusion matrix. In a confusion matrix, your classification results are compared to additional ground truth information. The strength of a confusion matrix is that it identifies the nature of classification errors, as well as their quantities.

To obtain a confusion matrix:

Have the output raster map of your image classification available; check the Properties of the classified image to know which domain and georeference are used by the classified image.
Create a raster map which contains additional ground truth information, such a map is also known as the test set. There are several ways to create a test set, these are shortly described here; at the end of this topic you will find more details of creating a test set.

Create a new raster map with the same domain and georeference as the classified image, and add some test pixels to this map yourself with the pixel editor while using an existing map as a background (for instance a land use map),
Use an existing, recent and reliable raster map with the same domain and georeference as the classified image directly as a test set.
Use half of your ground truth data for the sample set before classification, and use the other half of your ground truth data for creating a test set.

Make sure that the ground truth/test set raster map does not contain the same pixels as the sample set raster map from the training phase. Your accuracy assessment will show too optimistic figures when the pixels in the sample set (on which the classification is based) are also used in the test set (with which the classification results are checked).

Perform a Cross operation with your ground truth map and the classified image to obtain a cross table.

To start the Cross operation:

from the Operations menu in the Main window, select Raster Operations and then the Cross command, or
double-click the Cross item in the Operation-list, or
use the right mouse button on your test set raster map in the Catalog and from the context-sensitive menu select Raster Operations and Cross.

In the Cross dialog box:

for First Map, select the test set map,
accept the default selection of the Ignore Undefs check box for the first map,
for Second Map, select the classified image,
clear the Ignore Undefs check box for the second map,
type a name for the output cross table, and
select the Show check box to directly display the output cross table in a table window.

In the table window displaying the cross table, open the View menu and choose Confusion matrix.

In the Confusion matrix dialog box:

for First Column, select the test set column in the table,
for Second Column, select the classified image column in the table,
for Frequency, select the NPix column in the table.

When you click OK, the confusion matrix is displayed in a matrix window.

For more information on the interpretation of a confusion matrix, refer to Table window : Confusion matrix.

Additional information to create a test set:

Creating a test set yourself with the pixel editor using a background map:

Check the domain and georeference of the classified image in the Properties dialog box of the image. Check the georeference of the background map which can be for instance an existing land use map. If the background map does not have the same georeference as your classified image, you can use the Resample operation to resample the existing map to the georeference of the classified image.
Display the background raster map in a map window.
From the File menu in the map window, choose the Create Raster Map command to create an empty ground truth map. In the Create Raster Map dialog box: type a new name for the ground truth map; accept the georeference of the classified image; and select the same domain as used by the classified image,
The pixel editor is opened automatically; the background map is displayed. When you zoom in, you can start selecting and giving names to pixels in your ground truth map.

Using an existing raster map or polygon map of the area:
When you have a recent and reliable polygon or raster map of the area, you can directly use this map as the ground truth map.

If the map does not have the same domain as the classified image, you can first add a column (with the domain of the classified image) to the attribute table of the map, assign a class to each class in the map and create an attribute map.
If the map does not have the same georeference as the classified image, you can first either rasterize the polygon map on the georeference of the image, or resample the raster map to the georeference of the image.
Then, cross the raster map with the classified image.

When you have collected ground truth data in the field, a calculation-wise correct method is to use half of these data for the sample set, and the other half for the test set.