HYSWEEP Performance Testing PROJECT SAMPLE PERFORMANCE TEST and

® HYSWEEP Performance Testing PROJECT: SAMPLE PERFORMANCE TEST and SINGLE BEAM TEST 11/4/2020 HYPACK® 2006 1

Performance Testing Your Multibeam System We can ask a lot of questions of the data. – How well does multibeam match single beam? – Are depths repeatable? – What beams can you use? – What is the magnitude of error from the system? Two tests in the MULTIBEAM EDITOR can analyze the data and determine how well the system performs. 11/4/2020 HYPACK® 2006 3

Performance Testing a Multibeam System The Performance Test determines the Repeatability of your Multibeam system. 1. Prepare reference surface. 2. Performance Test compares concurrent data. USACE surveyors and contractors may directly compare results with Engineering Manual standards. 11/4/2020 HYPACK® 2006 4

USACE Performance Standards Bottom Type Hard Soft Maximum Outlier Max. Mean Bias 95% confidence depth<15 ft depth 15 – 40 ft depth >40 ft 1 ft <0. 2 ft ± 0. 5 ft ± 1. 0 ft ± 2. 0 ft (from Table 3 -1 Minimum Performance Standards for Corps of Engineers Hydrographic Surveys) 11/4/2020 HYPACK® 2006 5

Reference Surface Small area with a very flat bottom. The uniform bottom removes position errors (DGPS = +/- 6’) from the testing. Represents the best estimation of depths possible with the Multibeam system. With lots of overlap, and ignoring outer beams provides the best confidence for the surface. 11/4/2020 Look for a very unremarkable reference surface. HYPACK® 2006 6

The Reference Survey Two sets of five parallel lines. Line separation = water depth. Run survey at low tide to minimize errors. SV cast immediately before surveying. The reference survey isn’t very large: In 50 feet of water, the area would be 200’ x 200’ 11/4/2020 HYPACK® 2006 7

Matrix file for survey area Create a matrix file bordering the reference surface. – Use HYPACK® Matrix Editor. – Cell size = 1’ x 1’. – Type = HYSWEEP – Save it as Reference. mtx. 11/4/2020 HYPACK® 2006 8

Survey to Surface: Processing in MULTIBEAM EDITOR 1. 2. 3. 4. Load Reference Survey into MB EDITOR. Apply tide and SV corrections, and filter to +/ - 45 degree angle limit. Edit the data for flyers; use the Reference. mtx for Phase 3. Save cell average (minimum of 3 soundings per cell) XYZ format (reference. xyz). Edited reference surface. Flat: All depths 51’ to 52’. 11/4/2020 HYPACK® 2006 9

Check Lines Run in two directions over the reference survey. Process in MB EDITOR with tide and SV corrections. Carefully edit soundings Without Angle Limits. – By not using angle limits, multibeam performance can be checked over the full sonar range. Use Reference. mtx for Phase 3. Multibeam check lines 11/4/2020 HYPACK® 2006 10

Beam Angle Test: Comparison 1. Use MB EDITOR menu: Tools-Beam Angle Test. 2. Click: Open Reference Surface / Start Test. 3. Select Reference. xyz. – Difference Statistics: Reference surface – check line depth. – Graph depth difference vs. beam angle limit. – X’s show bias, ref – check. – O’s show maximum difference at 95% confidence. – Note: Data quality starts to degrade at 45% angle limit. 11/4/2020 HYPACK® 2006 11

Beam Angle Test: Details Statistics at a selected angle limit. 45 degrees in this example. – Histogram = difference (ref check) Distribution. – Max Outlier = Max difference. – Mean Difference = Bias between reference surface and check lines. – 95% Confidence = 2 * Sigma. 95% of differences fall within 2 * sigma. 11/4/2020 HYPACK® 2006 12

Case Study: S/V Dobrin Results for NY District S/V Dobrin fall within USACE accuracy standards for survey data collected in 40 feet on a hard bottom Note: Results give system repeatability. Accuracy must be determined by other means (Bar check, lead line comparison, etc). Max Outlier Mean Difference 95% Confidence Requirement < 1. 0 Ft < 0. 1 Ft < 1. 0 Ft Dobrin Result 0. 5 Ft 0. 0 Ft 0. 3 Ft USACE EM 1110 -2 -1003 manual 11/4/2020 HYPACK® 2006 13

Performance Test in Action 11/4/2020 HYPACK® 2006 14

Single Beam versus Multibeam Statistics 11/4/2020 HYPACK® 2006 15

Statistical Comparison Between Single Beam and Multibeam Soundings Do single beam and multibeam soundings match up? They Better! The Single Beam Test in the MB EDITOR tells you how well. Start with two single beam check lines over the reference surface. (See Performance Test). 11/4/2020 HYPACK® 2006 16

Steps to the Single Beam Test Check Lines: Raw Data Single Beam Editor Edited All Format Export XYZ Format Reference. mtx Multibeam Editor Launch Single Beam Test from Tools menu of Multibeam Editor Reference. xyz 11/4/2020 Single Beam Test HYPACK® 2006 18

Single Beam Processing Edit check lines in SB EDITOR with corrections. Use EXPORT to convert to XYZ format. 11/4/2020 HYPACK® 2006 19

Load Check Lines into MULTIBEAM EDITOR Open exported XYZ file in MB EDITOR. Use reference matrix for MB EDITOR gridding. 11/4/2020 HYPACK® 2006 20

Run Single Beam Test in the MULTIBEAM EDITOR Run Single Beam Test from the MULTIBEAM EDITOR Tools menu. Open reference surface with button and select reference XYZ file. 11/4/2020 HYPACK® 2006 21

Single Beam Test Results As before, all statistics are based on depth difference: Reference – Check. – Max Outlier = Max Difference. – Mean Difference = Bias. THE MAIN POINT OF THE TEST! – Standard Deviation (Sigma) = Spread of difference distribution – 95% Confidence = 2 * Sigma. 95% of differences fall within 2 * sigma. – Histogram shows difference bias and distribution. 11/4/2020 HYPACK® 2006 22

Sample: Single Beam Test 11/4/2020 HYPACK® 2006 23