Data Validation

In general, we trust the validation efforts of Intermap Technologies Corporation. Their products and techniques have undergone extensive review and validation, both internally and independently. You can visit the library to read much of the prior literature on this. Our efforts in this project were largely to ensure that no gross errors exist (e.g., data loss, clipped off streams near acquisition boundaries) and provide another independent check on accuracy. We looked at horizontal accuracy, absolute vertical accuracy, relative vertical (slope) accuracy, systematic noise, data losses, and acquisition extent.

Horizontal accuracy. Our primary means to determine horizontal accuracy was comparing pre-existing outlines of lake boundaries and roads to the corresponding boundaries from Star3i. USGS DLGs were overlaid over the Star3i ORRI and found to consistently visually match the lake boundaries of the ORRI across the entire image. We found that the Star3i data agrees to within 10 m of many DLG and D-GPS boundaries. Click here for several sample images. Differences between a haul road gps transect and had offsets of up to 25 m. Though such differences between the data sets exists, we cannot attribute their cause to any one data set and thus cannot validate the Star3i specification of +/- 2.5 m horizontal accuracy to that level. These differences may be some combination of lake level differences, lake boundary changes, projection errors, and digitizing inaccuracies in the DLGs. Because the interferometry works poorly over water, Intermap manually edited much of the lake data, and this may have introduced some further error. Their procedures are described loosely in their Star3i Product Handbook; we were unable to get any more detailed information than this from them, therefore users should be careful when using Star3i lake data scientifically. Generally speaking, we have no reason to doubt that the data meets their specification, but we cannot verify it down that level ourselves.

Absolute Vertical Accuracy. We compared vertical elevations of several hundred D-GPS spot measurements to the Star3i DEM and found an R.M.S. difference of 2.7 m, within the 3 m specification. Considering that the Star3i is geoid reference and the D-GPS data were mean sea level reference, we considered these close agreement. If the vertical data are shifted to minimize RMS difference, the Star3i data were found to be 1.0 m higher with an RMS of 2.5 m. The errors ranged from -7.7 m to +6.1 m; it should be noted that some of the GPS measurements were made of stream bed (below water), and these points were not culled before the analysis.

Relative Vertical Accuracy. Relative vertical accuracy is a more difficult metric to test. Our approach was to compare a channel network created from the DEM to the USGS DLGs. The theory here is that if relative accuracy (or slope accuracy) is high, then water should flow the right direction. We derived channel networks using the commercial software RiverTools using the 10 m DEM. Generally speaking, there was a very close visual correspondence between the two channel networks (click here for examples). Intermap does not publish a relative vertical accuracy specification, and we have no means to quantify it ourselves. Given that their most accurate absolute vertical accuracy specification is 0.3 m, we understand that this the noise floor of the sensor itself and therefore perhaps a good surrogate value to use for slope accuracy. Noise increases towards the far edge of the imaged swath, and this can introduce a systematic error that adversely affects channel networks, as described next.

Systematic Errors. We found four types of systematic errors: increasing random error across the swath width (noise gradients), systematic humps or ridges related to multi-path in the radome, swath mosaicing errors, and systematic cross-hatching. As the distance between the sensor and the ground increasing towards the far edge of the swath, the amount of random noise increases. This is clearly visible in slope images of the data. In low gradient regions, like the Putuligayuk River watershed (adjacent to the Kuparuk River), this systematic error can erroneously route channel networks and lead to a network where channels are aligned in the direction of data acquisition. Similarly, multipath within the SAR's radome can create small ridges in the data on the order of 10 cm that systematically align in the direction of swath acquisition. These ridges show up clearly in slope images and shaded relief images, and are a well documented feature of Star3i data. Again, they are typically only inconvenient when using the data at full spatial resolution in low gradient areas. In the case of the Kuparuk River, the only location that this caused a problem was near its eastern border with the Putuligayuk river. This error is described more fully in the next section on data gaps. We found swath mosaicking errors to be a minor problem with this data set, but such errors were found. While they meet the 3 m vertical accuracy specification, we encourage future acquisitions of Star3i data to be funded at the best level of vertical accuracy offered, at least in the low-gradient regions of the Arctic. We also found a peculiar noise phenomenon, a regular grid pattern in slope images, particularly in hilly areas. It is clearly a processing artifact, but we do not know the cause. It is a minor artifact, and has not affected any of our analyses thus far.

Data Loss. We were only able to identify one location of true data loss (caused by an instrument malfunction), but numerous areas of shadowing or layover exist that Intermap interpolated across to fill in the gaps. The data loss occurred at the boundary between the Kuparuk and Putuligayuk Rivers, at a very inconvenient location. When the channel network is overlaid onto the ORRI, it becomes apparent that a channel that should flow into the Kuparuk River is routed improperly into the Putuligayuk River, and this stream drains approximately 120 km2 area. Intermap kindly re-acquired data the following season to fill in this gap, however the patch provided (while meeting their 3 m vertical accuracy specification) does not alleviate the problem because the patch itself contains a ridge perpendicular to the gradient that again re-routes flow. Rather than further modify these data manually and potentially introduce further error or subjective interpretations, we have provided the data in the form supplied by Intermap. Shadowing and layover are common problems associated with SAR data acquisition and are described in the Product Handbook found in the library. When such data losses occur, the Intermap processor interpolates across the gaps to provide seamless data. Unfortunately Intermap was unable to provide us with a mask indicating where such errors occurred, but estimated that less than 2% of this area had such losses. They indicated that the data loss in the ORRI roughly corresponds to the DEM data loss. Unfortunately this data loss has the same data value as lake surfaces, and the two can be confused. A careful comparison of the ORRI and some other spatial imagery is therefore recommended in your study location, to determine whether the black areas (DN=0) are indeed lakes or data loss.

Acquisition Extent. We located two regions where the Star3i acquisition likely did not extend far enough to encompass the entire Kuparuk River watershed. We identified these regions as such by overlaying the DLG data onto the ORRI, as seen here. The inside corner on the northern eastern side of the data acquistion missed approximately 2 km2 of data, and approximately 40 km2 in the south-eastern inside corner. To produce the channel networks of the complete watershed, we temporarily merged UGSS National Elevation Data (NED) and re-ran the channel networking software. This worked fine in the south-eastern area as the mosaic was nearly seamless, but only marginally well in the north-western region. Here the USGS formed a 2 m wall that prevented channels to cross the interface between the two data sets. The affected area was quite small, however, the end result is reasonably satisfactory in that all of the headwater area (that would have been 'chopped off' by the break in the stream channel did find its proper outlet; that is, a large meander (outside the aquisition area) was replaced by a straight line connecting the loose ends.

Resampling Errors. We resampled the DEM from 10 m to 25 m, 50 m and 100 m using nearest neighbor and cubic splines. We found that resampling from 10 to 25 m caused significant errors in the resulting channel networks in the low gradient area that separates the Kuparuk watershed from the Putuligayuk watershed. Examples of these errors are provided here. As a result of this resampling, roughly a 125 km2 area switches from the Kuparuk watershed at 10 m and into the Putuligayuk watershed at 25 m. In reality, this area likely flows into the Kuparuk watershed.