Sioux Falls Geometric Test Range Evaluation and Application

Sioux Falls Geometric Test Range: Evaluation and Application Aparajithan Sampath, Don Moe, Jon Christopherson, Greg Strensaas. ASPRS, April 2010. U. S. Department of the Interior U. S. Geological Survey

Outline l l Overview of Geometric test ranges Sioux Falls range u u l Location, dimensions Accuracy assessment of Sioux Falls range Our method for orthoimagery product validation u Using automated image-to-image analysis for rapid and repeatable assessment of accuracy 2

Geometric Test Ranges: Part of Data Provider Certification l USGS to develop validation ranges across US u Aerial orthoimagery validation l l u u l Assess & certify product accuracy over approved range 5 -6 ranges across the US needed to “go operational” Satellite data validation also Li. DAR data validation: In the near future To have quantified wall-to-wall reference imagery over range extents u u Use image-to-image analysis for rapid and repeatable assessment of accuracy May be doing away the wall-to-wall requirement 3

Range Locations Sioux Falls ? ? Pueblo, CO Rolla, MO ? 4

Sioux Falls Range l l Reference Orthoimagery covers Minnehaha and Lincoln counties l 85 km (53 miles) N-S l 54 km (34 mi) E-W 30 cm, 15 cm, and 7. 5 cm (12 in, 6 in, & 3 in) Li. DAR coverage at 1. 4 m posting Orthoimagery and Li. DAR data collected jointly by the USGS, Minnehaha and Lincoln Counties 5

Accuracy Assessment of Ortho Imagery l l Task was to determine the suitability of available imagery to be used as reference imagery GPS-RTK Survey u u l Pick photo identifiable points from orthophotos GPS survey to establish check points Accuracy Assessment u u Compare coordinates from Orthophotos and RTK survey Accuracy Analyst ™ used to measure and compile results into report. 6

Data extents & check points l l l 3320 km² total of 30 cm (1 ft) data over Minnehaha and Lincoln Counties 760 km² of 15 cm (6 in) data over Sioux Falls and surrounding areas 115 km² of 7. 5 cm (3 in) data over Sioux Falls 12” Data over Minnehaha County 3” data over the city of Sioux Falls 6” data over parts of Minnehaha and Lincoln counties 12” Data over Lincoln County RTK Survey points on photo-identifiable features (Red dots) Sanborn survey points (Black dots) 7

Accuracy of Surveying l l l The RTK survey process was tested by surveying known points near the Sioux Falls Airport and EROS Maximum error was 3. 6 cm or 1. 43 inches Perfectly acceptable for assessing 3”, 6” and 12” data products 8

Check Points l l A total of 112 points were surveyed 56 over Sioux Falls covered by 3” GSD data Another 13 in the 6” GSD region Rest in the 12” GSD region over Minnehaha and Lincoln Counties 9

Survey Results Orthoproduct RMSE CE 95 7. 5 cm (3”) 8. 5 cm (3. 4 in) 1. 12 pixels 20. 4 cm (8. 0 in) 2. 68 pixels 15 cm (6”) 11. 6 cm (4. 5 in) 0. 76 pixels 29. 6 cm (11. 4 in) 1. 9 pixels 30 cm (12”) 22. 9 cm (9. 0 in) 0. 75 pixels 56. 1 cm (22. 1 in) 1. 84 pixels 10

RMSE and CE 95 Plots 11

Geometric Assessment of Geo. Eye-1 Over Sioux Falls Test Range l Geo. Eye-1 was launched in September 2008 u u u l Resolution – 0. 41 m at nadir for Panchromatic band Data provided to USGS has been resampled to 0. 5 m 1. 65 m for multispectral (Not Assessed) 15 x 15 km Single point scene 12

Geometric Accuracy Validation: Image-to Image l l Select uniformly random points over reference imagery, and determine corresponding locations in the search image using cross correlation Compare coordinates between search and reference images to obtain accuracy statistics. 13

I 2 I Matching Reference image chip Search image chip Photo-Identifiable point P Error in Search image P’s actual location in study image determined from Image matching P’s estimated location in study image Moving search template that determines cross correlation matching measure 14

Problems l For high resolution data, randomly selected points may prove problematic due to: u u Shadows Look angle may render some points invisible Too many similar features (e. g. parking lot) Too low contrast 15

Geometric Accuracy Validation: Combining Check Points, Open Source Tools l l Combine Geospatial Data Abstraction Library (GDAL) and I 2 I and check points GDAL used to locate and cut image chips around the check points (Reference and search chips) u u l Error corrections for the check points incorporated Reference chips are actually more accurate than the reference image I 2 I measurements carried out between reference chip center and the search chip 16

Advantages l l Reference image chips are accurate to sub-pixel Procedure reduces/eliminates the need to rectify, resample and create large mosaics of reference images (around 200 of them) Technique can potentially validate images in different coordinate systems and resolution Image reference chips can be reused against other images 17

Results 45 check points used to calculate statistics Line Sample Mean 2. 77 0. 11 1. 38 . 05 Standard Deviation 5. 62 4. 28 2. 81 2. 14 RMSE 6. 27 4. 29 3. 13 2. 14 Pixels Meters 18

Going Forward l l Instead of wall-to-wall imagery, collect image chips over surveyed photoidentifiable points Planned use of a UAV with an attached camera (COTS) Plan will generate high accuracy image chips of GSD 3 cm-10 cm resolution Use Image chips from NAIP data to validate lower resolution products 19

Advantages l l No need for Wall-to-Wall coverage Small image (<100 ft * 100 ft) size will make it more manageable Images can be collected from carefully selected locations High accuracy ranges can be built with reduced costs and time 20

Summary l l l Sioux Falls range is ready as Geometric validation site for high resolution satellite and aerial images Visionmap A 3 system was flown over the range, waiting for orthoproduct Geo. Eye-1 data, as well as Rapid. Eye, was analyzed using the range. World. View-2 data will be analyzed in the near future. The combined use open source tools (GDAL, IAS-I 2 I) to handle large datasets is promising and efforts to improve the I 2 I tool will be investigated Two More Ranges in development u Rolla, MO u Pueblo, CO Both have some existing imagery u But not high enough resolution u Developing new ideas for obtaining image chips 21