ILWIS objects

Georeference Direct Linear (non-coplanarity of tiepoints)

It is recommended to create a georeference direct linear when:

you have small format photographs, i.e. photographs taken with normal camera and photographs without fiducial marks,
the terrain covered by the photograph has clear height differences, i.e. you need to correct for tilt and relief displacement,
a Digital Terrain Model (DTM) of the area is available.

By creating a georef direct linear and displaying the photograph, you can for instance directly digitize on the displayed non-rectified photograph on your screen. For more information, see How to screen digitize.

Tips:

For photographs that cover an almost flat terrain, you can use a georeference tiepoints with a projective transformation.
For photographs that have fiducial marks, you should use a georeference orthophoto.

General information on a georeference direct linear:

A georef direct linear is calculated by a Direct Linear Transformation (DLT):

Row = (aX + bY + cZ + d) / (eX + fY + gZ +1)

Col = (hX + iY + jZ + k) / (eX + fY + gZ +1)

A georef direct linear requires at least 6 tiepoints (also called control points). For each tiepoint, RowCol numbers from the photograph and real world XY-coordinates are stored. Height (Z) values can be supplied by the user, otherwise these are obtained through the XY-coordinates from the DTM.
The flying height, the camera projection center (X0, Y0, Z0), the camera axis angles (a, b, g) with the X, Y, Z axes are calculated from the tiepoints.

Avoiding coplanarity:

In a georeference orthophoto, the inner orientation calculates the principal point from the fiducial marks and the principal distance. Then, camera position (X₀, Y₀, Z₀) and angles (k, f, w) are calculated from the tiepoints (outer orientation).

In a georeference direct linear, there is no inner orientation; the principal point thus remains unknown. The location (X₀, Y₀, Z₀) and the tilt of the camera are calculated from the tiepoints only.

When all tiepoints in your georef direct linear fit in one (tilted) plane in XYZ-direction, a projection center cannot be calculated and you will get an error message that you have a singular matrix. Your tiepoints are coplanar. The matrix will remain singular when only one tiepoint is outside the common plane.
When the tiepoints almost fit in one plane, i.e. when tiepoints are almost coplanar, then the Direct Linear Transformation seems to work but is not reliable.

A georeference direct linear has the highest accuracy within the 3D envelop bounded by the tiepoints; thus the better the XYZ spread of tiepoints, the better the transformation will work. To obtain a reliable georeference direct linear, it is necessary that (at least) 2 tiepoints clearly deviate in Z-direction from a (tilted) plane that can be fit by means of a least squares approximation through the active tiepoints.

Examples:

When your photograph covers a valley with hills or mountains on either side of the valley, then position your control points both in the valley and on both sides of this valley in the hills or mountains;
When your photograph covers many valleys and hills, then position your tiepoints in a number of valleys and on a number of hills.

Tips:

It is always a good idea to find many reliable tiepoints. The more tiepoints you have, the better a georeference will be.
Make sure that tiepoints are well spread over the photograph (XY-direction), that the tiepoints use different height values in your DEM (Z-direction). The Z-values of at least 2 tiepoints need to be outside a tilted plane that can be fit through all active tiepoints.
The results of a Direct Linear Transformation are most reliable inside the 3D envelop bounded by the tiepoints. For positions far away from any tiepoint (e.g. borders of photograph), and for areas where the difference between the DEM-value and the calculated plane is significantly greater than the Z-confidence value (see below), results will not be reliable.

Detecting coplanarity:

You may have a problem with coplanar tiepoints when you use a georeference direct linear and during screen digitizing you notice a shift between the location of the segments or points which you were digitizing and the location where these digitized segments and points are appearing in the photo on the screen.

By opening the Properties dialog box of a georeference direct linear and by clicking the Additional Info button in the Properties dialog box, you can find for instance:

the number of active tiepoints;
the Root Mean Square (RMS) of the vertical Z-differences of all active control points towards a tilted plane that is the result of a least squares fit through all control points;
the 2 tiepoints that are found to be the outliers (O1 and O2); these are the tiepoints with the largest vertical Z-differences towards the tilted plane;
the geometric mean of the vertical distances d1 and d2 of these outliers towards the tilted plane is called the Z-confidence range; this is an estimate for the reliable Z-range above and below the tilted plane; this is calculated as Ö(d1*d2).

For your information, Additional Info also shows the Direct Linear Transformation equations, the estimated camera projection center, the camera axis angles with respect to the X, Y, Z axes, and the calculated approximate pixel size in the photograph.

In a georeference direct linear without coplanar tiepoints, both the Root Mean Square value and the Z-confidence range are large. The theoretical maximum RMS is about half the range of height values in your DTM.

In a georeference direct linear with almost coplanar tiepoints, the Z-differences of all control points towards the calculated XYZ plane are very small, also the RMS and Z-confidence range are small.

In case of almost coplanar tiepoints, the transformation may even give an estimated camera projection center below the ground (or below the average Z-coordinate of the active control points). Obviously, the transformation will then give completely wrong results.

Technical information

Introduction

Direct Linear Transformation (DLT) as used in an ILWIS 'GeoRefDirectLinear' is based on solving 2 equations between Row,Col pairs in a given (scanned) photo and X,Y,Z coordinates of corresponding ground control points (in meters):

	Row_i = (aX_i + bY_i + cZ_i + d) / (eX_i + fY_i + gZ_i +1)	(Eq. 1)
	Col_i = (hX_i + iY_i + jZ_i + k) / (eX_i + fY_i + gZ_i +1)	(Eq. 2)

There are 11 parameters to be solved (a, b ... k), the so-called DLT-coefficients.

In ILWIS this is done by reformulating (Eq. 1) and (Eq. 2) as linear equations with these coefficients as unknowns:

	X_ia + Y_ib + Z_ic + d + X_iRow_ie + Y_iRow_if + Z_iRow_i*g + Row_i = 0	(Eq. 1a)
	X_ih + Y_ii + Z_ij + d + X_iCol_ie + Y_iCol_if + Z_iCol_i*g + Col_i = 0	(Eq. 2a)

The unknowns can be solved if we have 6 or more control points, giving 12 or more linear equations, that can be solved, unless the points are co-planar (see below).

After solving the DLT coefficients, the position of the aerial camera can be estimated from the intersection of 3 planes in space given by linear equations in terms of ground coordinates:

	a X + b Y + c Z + d = 0	(Eq. 3)
	h X + i Y + j Z + k = 0	(Eq. 4)
	e X + f Y + g Z + 1 = 0	(Eq. 5)

The plane defined by equation (Eq. 5) is parallel to the photo plane and passes through the projection center (PC) (assuming a perfect central projection). It intersects the ground at points that do not appear in the photograph (projected at 'infinity').

The planes defined by (Eq. 3) and (Eq. 4) pass through the projection center and the lines in the photo where (reduced) values of Row and Col resp. are equal to 0.

Furthermore, the knowledge of the DLT coefficients a, b ... k, enables the forward transformation (from XY to RowCols) using (Eq. 1) and (Eq. 2); this is the photographic central projection from ground (3D) to photo plate (2D).

The inverse transformation (from RowCol to XY) is more complicated (from 2D to 3D) and only possible with the use of a complete DTM of the ground.

It is possible to assume the Z-coordinate (say Z1 ) for a point in the ground system associated to a photo point (given as RowCol), and solve the remaining unknowns X and Y. These are the coordinates of the piercing point of a corresponding light-ray with the horizontal level-plane Z = Z1.

The DTM is then used to improve the initial guess of the Z-coordinate. Improving the X,Y estimation is done iteratively with the help of the position parameters of the light-ray (slope cotangents with X and Y -axes and start X1,Y1 of piercing point). The iterative algorithm to find the piercing point of each light-ray is the main part of the inverse DLT.

This is very similar to what is implemented for the Georeference Orthophoto.

The accuracy of the two georeference transformations (forward and inverse) of the Georeference DirectLinear highly depends on the quality of the DLT coefficients. How well the DLT performs the transformation for the given control points is visible in the value of sigma. Apart from the horizontal XY spread of the control points, it mainly depends on co-planarity properties of the control points.

This is in contrast to the transformation used in a Georeference Orthophoto. In a Georeference Orthophoto, the quality of the transformation mainly depends on the quality of the inner orientation (camera and photo geometry) and the outer orientation (using at least 3 control points).

In both 3D georeferencing methods the quality of the transformation also highly depends on the quality of the underlying DTM.

Recalling the main use of the transformations:

The forward transformation (XY to RowCol) is needed for:

Overlaying the non-rectified photomap with vectors (points, segments, grid, graticule, polygons);
Resampling from the 'photo'-georef to a GeorefCorners (rectification while making orthophotos and mosaicking).

The inverse transformation (RowCol to XY) is needed for:

Getting pixel info from vector maps that are overlaid on the non-rectified photo;
Screen digitizing on the non rectified background photo.

Coplanarity

Ground control points can be co-planar, i.e. lie in one (possibly non-horizontal) plane, even if the terrain is very mountainous. When this is the case, the computation of the DLT- coefficients is impossible because Z is a linear function of X and Y; for each control point: Z = rX + sY + t. This makes the DLT-coefficients 'triple-wise' linearly dependent: (a, b, c), (h, i, j) and (e, f, g) each have one redundant unknown, see Eq. 3, Eq. 4, Eq. 5.

The systems of equations like (Eq. 1a)and (Eq. 2a) only allow to solve 8 independent coefficients out of the 11, no matter how many (coplanar) control points are used. In other words, the rank of the normal equations found from an over-determined collection of equations of types (Eq. 1a) and (Eq. 2a) will be at most 11 - 3 = 8, i.e. the column rank of the matrix of this collection. The loss of rank is caused by vanishing columns related to the unknowns d, k and g because their coefficients contain Z. Hence the system of normal equations (11 by 11) will be singular.

To raise its rank above 11, one needs 2 control points outside a common plane of the others. One 'outsider' together with at least 4 points inside a plane would bring the rank up to 10, making still a singular normal matrix.

Conclusion: to solve the DLT coefficients one needs at least 6 control points and such that 2 are not coplanar with the others.

If the ground control points are nearly co-planar, the system will be ill-conditioned.

This means: small perturbations (errors) in ground control coordinates or RowCol identification in the photo will cause large perturbations in the DLT coefficients. These perturbations will not much influence the sigma value displayed in the editor window of the tiepoint table. This sigma tells how well the transforms of the control points coordinates match with their photo coordinates, but not how good this transformation works in areas lying outside the region enclosed by the ground control points.

The DLT errors in turn will cause errors in the position of the projection center and in the mentioned light-ray parameters which are used in the inverse DLT. Even if control-points were error-free (both in terrain and photo system), in case of near-coplanarity, the DLT would be unreliable for points away from the control points despite of a low sigma value of these control points especially for terrain points far above or below the coplanar 'trend' plane.

Other factors that influence the errors due to near-coplanarity are:

spread of the control points in terms of X,Y coordinates;
the random (measurement) error in the control points in both systems;
the flying height found by the relative positions of the planes of Eq. 3, Eq. 4 and Eq. 5.

Non-coplanarity

As a general rule, one can say that the DLT transformation and its inverse are reliable inside the convex hull of the control points. This hull is defined as the smallest convex polyhedron one can construct around a (3D-) collection of points. A polygon (in 2D) or a polyhedron (in 3D) is called convex if each pair of inside points can be connected by a straight line that lies completely inside the polygon cq. polyhedron.

Practically speaking, if one wants to be sure that the Georeference Direct Linear is reliable for the whole region of interest, one needs to use control points which convex hull englobes the complete region, also with respect to Z coordinates (heights).

For instance, when all control points are located in valleys and are nearly coplanar, the points will not produce a good transformation for points lying far above this common plane.

One way to quantify non-coplanarity is using RMS (root mean square). It is computed after fitting a plane ('trend' plane) through the control points by means of a least squares approximation. The vertical distances between the control points and the plane (Z-differences) are squared, added and divided by n, the number of points. The non-coplanarity is defined as the square root of this result. It is expressed in terrain units (generally meters). Its theoretical maximum (optimum) equals about half the range of the DTM height-values.

For another quantification of non-coplanarity, especially for the 2 necessary outliers, we can find what is called in ILWIS a confidence range of the Z-values. First of all, the outliers, defined as the control points having the largest Z-deviation from the earlier mentioned trend plane need to be found. The 2 vertical distances d1 and d2 of the outliers towards the trend plane are calculated. The geometric mean of d1 and d2 is an estimate for the reliable Z range below and above the trend plane. This is called the Z-confidence Range in Additional Info. It is a measure of the thickness of the convex hull of control points.

If a point to be transformed is at a vertical distance d from the average control point position and if r stands for the earlier defined Z-confidence Range, the ratio d/r gives an idea about the transformation error to be expected.

If d/r < 1 , one might expect errors similar to those in the control points themselves (sigma).

If d/r >> 1 , the point is likely far beyond the convex 'control' hull and the error will increase with the increase of d/r.

The values of r and d can be found c.q. derived from the Additional Info that is delivered after creation or modification of a Direct Linear georeference. In this Additional Info one can find also the DTM height difference of all control points with respect to the trend plane.

Finally one finds information on the computed camera orientation: its distance above the nadir point, its distance from the centroid of the control points and the orientation of the camera axis. If the distances turn out to be negative or too small, the user should conclude that the transformation is not well determined due to errors in the control point coordinates or due to (quasi-)coplanarity.

How the real stochastic error behavior is, inside and outside the convex hull, also strongly depends on the XY spread of the control points with respect to each other and the rest of the terrain, and on variance-covariance models of the observations.

This is beyond the scope of this analysis.

References:

K.G. Grabmaier, Orientation Theory, ITC Lecture Notes.