MOWLES Interim Progress Report Rian Kistner Kathleen Mc

MOWLES Interim Progress Report Rian Kistner, Kathleen Mc. Lane, Joseph Navarro March 29, 2018

Agenda 1. Background & SLAM Algorithms 2. Objectives & Scope 3. Technical Approach 4. Test Plan Development 5. Project Status 6. Progress-to-Date 7. Challenges & Next Steps 2

Background/Introduction MOWLES’ Purpose: Sponsor: Develop an unmanned platform capable of surveying tunnels and buildings without risk to human life. U. S. Army Engineer Research and Development Center (ERDC) Geospatial Research Laboratory (GRL) Topographic Engineering Center (TEC) System : ● Platform ○ GVR-Bot ● Software Package ● Sensor ○ Requirement: A low-cost, low-power, small form factor sensor platform capable of producing 3 D point clouds. Image source: http: //www. nbcnews. com/id/3078710/ns/technology_and_science-science/t/new-robots-well-trained-war/#. Wnv. Krx. OPLy 9 3

Orbbec Astra (w/ or w/o GPU) ZED Photogrammetry (no GPU) Size 160 x 30 x 40 mm 175 x 30 x 33 mm Weight 0. 3 kg 0. 159 kg Range Limit 0. 6 -8 m 0. 5 -20 m Angular Field of View 60 degrees horiz/49. 5 degrees vertical 110 degree field of view Resolution 640 x 480 @ 30 FPS 1080 p @ 30 FPS Computer System Open source/Windows 7/8/10/Linux/Android Windows 7, 8, 10/Linux Issues Hard drive not recognized Need a high performance laptop SLAM Algorithms Highly dense point cloud maps: RTABmap or Elastic fusion Dense point cloud maps: RTABmap or ZEDfu Table 1: Potential Sensors Identified by TEC Team 4

SLAM: Simultaneous Localization and Mapping ● Simultaneously build 3 D maps of the world while tracking the location and orientation of the camera/sensor. ● Performance can be measured by average localization error for each point, after the full dataset has been mapped. ● Output is a point cloud ○ Accuracy significantly influences the quality of derived products such as Digital Terrain Models (DTMs). 5

Types of SLAM RTAB-map ● ● Incremental appearance-based loop closure detector. Loop closure: recognizing an already mapped area. Elastic. Fusion ● Real-time dense visual SLAM system capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments explored using an RGB-D camera. ORBSLAM ● ● Monocular, Stereo and RGB-D cameras Compute real-time camera trajectory and sparse 3 D reconstruction of the scene Close large loops and perform global relocalization in real-time Sparse, indirect DSO ● ● Direct, sparse odometry Able to reconstruct all points, but has the efficiency of sparse methods 6

Significance of SLAM Algorithm Orbbec Astra Sensor ● ● Same sensor, two SLAM algorithms produced different Digital Terrain Model (DTM) results. Elastic. Fusion is higher quality. ○ Point-Cloud quality is more dependent on SLAM algorithm than sensor itself. 7

Sparse Point Clouds Dense Point Clouds Highly Dense Point Clouds Description Points unevenly distributed over surface 0. 5 -1 pts/m^2 Objects must have texture 5 -10 pts/m^2 Objects must have texture 10+ pts/m^2 Pros Quick and simple to utilize Easy to employ Cons Lose detail in sparse areas; hard to use on intricate backgrounds Large amounts of noise Lots of time to process; large amounts of noise Examples ORBSLAM & DSO ZEDfu & RTABMAP Elastic Fusion Table 2: Comparison of Point Clouds from SLAM Algorithms 8

SLAM Modules and Sensor Pairings 9

Objectives & Scope Objectives ● To create a decision analysis (DA) tool to help ERDC GRL select a sensor and SLAM module for the GVR-bot ● To create a test plan to be used by ERDC GRL in evaluation of the GVRbot’s SLAM module, sensor, and comparison of battery capacity Scope ● The sensor and SLAM algorithm is the main focus. This includes sensor selection, sensor testing, and sensor evaluation. 10

Technical Approach: Defining the Value Model Decision Analysis Objectives 1. Increase Accuracy/Decrease Error 2. Increase Range 3. Increase Processing Speed/Decrease Processing Time Analytic Hierarchy Process (AHP) ● Pairwise Comparisons ○ ○ Transitive Consistent *Ask stakeholders to only make two pairwise comparisons to keep matrix consistent. 11

Figure 1: Preliminary DSS interface 12

Technical Approach: Analytic Hierarchy Process ● ● ● Input from stakeholders to assign value to each comparison. Optimal state: Increase accuracy, increase range, and increase speed. Next best: Increase accuracy & speed, even if range is decreased. Figure 2: AHP Weight Figure 3: Additive Value for Each State 13

Technical Approach: Assumptions & Limitations Assumptions ● ● 2 Identified Sensors: only sensors being considered Only SLAM modules and sensor matchings provided by sponsor are being considered One sensor unit per GVR-Bot All equipment necessary is possessed by ERDC GRL Limitations ● ● No SLAM algorithm data Many sensor/algorithm combinations are dependent on acquisition of additional hardware. Defining stakeholder interests Orbbec Astra sensor would rely on saved data sets for SLAM module evaluation (no live feed) 14

Test Plan Development Format: ● Following IEEE 829 Standard for Software and System Test Documentation ○ Test Items, Features tested, Approach, Pass/Fail criteria, Test deliverables, Experiment design, Environmental needs, etc. Setting and Key features: ● CERDEC’s Night Vision tunnel located at Fort Belvoir, VA. ● Calibrated lighting controls provide ideal environment for testing under visual light constraints. 15

Test Plan Development (2) Test Objectives: ● ● ● Collect point cloud datasets for comparative analysis Determine effects of light level on visual odometry Determine effects of GPU on GVR-Bot’s battery capacity Key Metrics and Approach: ● ● δ*i, j = x* j ⊖ x*i : reference positions of the robot, true locations δi, j = x j ⊖ xi : positions of the robot estimated by a SLAM algorithm from node i to j. 16

Project Status Completed ● ● Sensor research Sensor selection design On-going ● ● Decision analysis construction Test plan construction Not Started ● Final Report/Website 17

Progress-to-Date Stakeholder Interaction ● ● Approval of Decision Analysis focus and approach New information on importance of SLAM algorithm on quality of point cloud outputs → shift in Decision Analysis approach Decision Analysis ● ● Established Value Model & objectives to maximize/minimize AHP weights Test Plan Development ● Objectives of next test (4 Q FY 18) determined ○ ○ ○ ● Light-threshold detection Depth-perception analysis GPU effect on GVR-bot’s battery capacity Draft Test Plan prepared ○ Identified key questions to followup with stakeholders 18

Challenges & Next Steps Issues 1. Limited testing data on sensors available 2. Sponsor communication Next Steps 1. Email sponsor with latest questions 2. Construct DA in Microsoft Excel using data collected 3. Create final report and website 19

Backup Slides 20

SLAM Algorithm Usage 21

MOWLES Program-Specific Metrics and Exit Criteria 22

(Mar 8) Sensor Information from Sponsor ● SLAM Modules ○ ○ Elastic. Fusion is better than RTABmap Why? ● ZED Photogrammetry Sensor ○ ○ Improvements with newest SDK (Software Development Kit) Why? How much better? ● ZED vs. Astra ○ ○ ZED has superior sensor specs Astra sensor produces better RTABMAP SLAM modules than the ZED ● Kinect no longer viable option 23

(Mar 8) Test Plan Information from Sponsor Goals of GVR-Bot + Sensor: ● Collect dense and sparse point clouds ● Determine effects of light level ● Determine effects of GPU on battery Mapping ● Data types: Sparse and Dense Environment/conditions ● Robot will be manually driven with joystick ● Data collection controlled manually ● Data collected to hard drive / SD card ● Data processing will take place off-site Tasks Beyond Mapping (Low Priority to TEC team) ● Investigate data thinning options ● Fusing data with existing point clouds ● Georeferencing Point Clouds Timelines for Outputs ● ● ● Dense point cloud with RTABMAP – now Mobile highly dense point cloud – end of 3 Q (awaiting laptop) Photogrammetry – end of 3 Q (awaiting laptop) 24

(Outdated slide) Progress-to-Date Stakeholder Interaction ● ● ● Approval of Decision Analysis focus and approach Setting up test site visit still pending 3/8 interaction ○ Qualitatively measure differences in two SLAM modules on sensors ○ Qualitatively show differences for ZED photogrammetry SDK versions ○ Testing will not involve: ■ Wireless data collection ■ Data fusion with existing point clouds ■ Geo-referencing point clouds ■ Environmental conditions 25

(Outdated slide) Project Plan Deliverables: 1. Decision Support System (DSS) on available sensors 2. Generate Test Plans for testing sensors MOWLES Five-Year Plan to Reach Scheduled Objectives 26

(Outdated Slide) Previous Recommendations If time permits, we will revisit the recommendations for next steps made by the previous GMU team. ● Create a Capabilities Description Document (CDD) for various GVR-Bot configurations relating to the platform and sensors used. ● Refine Tactical & Operational Objectives. Specifically, determine a suitable metric to measure effectiveness for different mission types. 27