Tiger Taxi Design Review Kenny Alexopoulos Zach Carmichael

Tiger Taxi - Design Review Kenny Alexopoulos, Zach Carmichael, Frank Cwitkowitz, Ben Glasstone

Agenda ● ● ● Customer Needs Marketing Requirements Engineering Specifications & Justification Current Project Status System Architecture Design Concepts User Interface Bill of Materials Gantt Chart Risks Test Plan

Customer Needs “The autonomous people mover is currently capable of only obstacle detection and unsafe location avoidance. The cart needs a mechanism for constructing a path between its current location and an arbitrary location on the RIT campus. Users will create the path by choosing the desired location, and the resulting path must comprise walkways of the RIT campus, allowing the cart to reach the destination in a appropriate manner. The solution of the presented path-planning and localization problem must be invariant to weather, time of day, and occlusion. ”

Objective Statement “The objective of this project is to introduce additional functionality to the RIT golf cart to progress toward a completely autonomous solution. An algorithm under the umbrella of simultaneous localization and mapping (SLAM) methods will be implemented to produce a traversable, persistent, and dynamic map of the golf cart’s environment on both a local and global scale. Information from light-detection and ranging (Li. DAR) sensors, odometry, GPS, and other sensors will be utilized as input to make informed decisions of navigable locations and safe routing. ”

Marketing Requirements 1. Should not reduce navigational performance of prior design iterations and should make use of their contributions 2. Should be able to navigate the majority of the RIT campus consistently 3. Should travel autonomously without endangering surroundings 4. Should be able to determine the best feasible path between the current and target locations autonomously 5. Should sustain performance in various weather conditions, times of day or year, campus events, people densities, and initial positions 6. Should need only a single mapping initialization phase per novel environment

Marketing Requirements 7. Should have a navigation user interface to allow the user to create and remove target destinations 8. Should require human interaction only to specify the next path of the cart, but should be able to accept human control when not navigating autonomously 9. Should be designed, developed, and maintained in such a way that future work can be easily incorporated

Engineering Specifications & Justification

Engineering Specifications & Justification

Current Project Status ● Cart upgraded to Ubuntu 16. 04 & ROS kinetic ● Workspace cleanup ○ ○ Eliminated six full packages and unnecessary sub-packages from workspace Still have yet to clean up specific files, although scope has been significantly reduced ● APM Phase 5 functionality restored post-upgrade ○ All members familiar with cart and functionality ● Slight restructuring of individual roles and management plan

System Architecture ● Main components ○ ○ Aerial Map Integration SLAM Navigation Sensor integration

Design Concepts: SLAM ● Why use Simultaneous Localization and Mapping (SLAM)? ● Integrating all available sensor data into SLAM ○ ○ GPS Aerial Maps ● Creating a persistent map

SLAM: Why? ● SLAM is the computational problem of developing or updating a map while maintaining knowledge of the robots location within the environment ● Common uses include: self-driving cars, UAV’s, planetary rovers and autonomous undersea exploration ● Our uses: ○ ○ ○ Create a persistent navigate-able map of the campus Create a global map indicating driveable locations Allow global and local navigation within the environment

SLAM: Some Math ● Pose-Graph SLAM: ○ Find the configuration of the SLAM graph nodes (poses) that minimizes error between a vehicle’s expected pose and its observations of its pose ○ ○ ○ X: The pose state vector Λ: The information matrix (weighting of constraint by amount of information provided)

SLAM: Important ways to compare SLAM Accuracy: How good the localization and map is Scalability: How large of a map the algorithm can compare Availability: How quickly can the system be brought up and tested Recovery: Ability to relocalize within the large-scale map Updatability: Ability to update based on permanent changes while ignoring temporary changes ● Dynamicity: Ability to handle dynamic environments ● ● ●

SLAM: Accuracy on Kitti dataset Method Translation Rotation Main Sensors V-LOAM 0. 60% 0. 0014[deg/m] Li. DAR+Camera LOAM 0. 61% 0. 0014[deg/m] Li. DAR LIMO 2_GP 0. 84% 0. 0022[deg/m] Li. DAR+Camera LIMO 2 0. 86% 0. 0022[deg/m] Li. DAR+Camera LIMO 0. 94% 0. 0026[deg/m] Li. DAR+Camera ORB-SLAM 2 1. 15% 0. 0027[deg/m] Camera (results from stereo setup)

SLAM: Super Sensor ● Main Sensors ○ ○ Li. DAR Monocular Camera ● Secondary Sensors and data ○ ○ ○ Inertial Measurement Unit (IMU) GPS Aerial maps ● All additional sensors are additional constraints within the pose-graph which require optimization ● Additional sensors increase the confidence associated with each point on the graph

SLAM: Map Persistence and the “kidnapped” robot problem ● Generated SLAM maps should be valid throughout the lifespan of the cart with small updates to handle new areas of campus ● Loop closure occurs during initial map creation and when re-aligning with the persistent map ● Methodology: ○ ○ Continuously create a new map while driving Every loop closure-detection iteration, check for loop closure with prior map On loop closure action, align the current map with the prior map After loop closure action: continue creating map aligned with prior map, updating when necessary

Design Concepts: Aerial Map Integration ● ● What data sources? What data types? How to structure data? How to integrate with SLAM?

Aerial Map Integration: Data Sources & Data Types ● RIT Maps Team Data ● WASP Project Data ● Monroe GIS Data ● Open. Street. Map Data

Data Sources & Data Types: RIT Maps Data ● Geo-referenced Building Polygons ● Building Types

Data Sources & Data Types: WASP Data ● Wildfire Airborne Sensor Program (WASP) ○ “Detect and monitor wildfires from an aircraft” ● All Data from 10/25/2012 ● Aerial Image-Stitch Mosaic of Campus ● Aerial Li. DAR Scan

Data Sources & Data Types: Monroe GIS Data ● Monroe County Geographic Information System (GIS) Services ● Land Parcels ● Aerial Image-Stitch of RIT Campus ● Aerial Li. DAR Scan of Rochester Area

Data Sources & Data Types: Open. Street. Map ● ● Geo-referenced Building Polygons Roads, Sidewalks, Misc. Annotation Updated by RIT Maps Team Annotations more up-to-date than Google Maps

Aerial Map Integration: SLAM Integration ● Building Data Used Commonly in SLAM-Based Methods ○ ○ ○ “Improving SLAM by Exploiting Building Information from Publicly Available Maps and Localization Priors” (Vysotska et al. , 2017) “Aerial Image Based Heading Correction for Large Scale SLAM in an Urban Canyon” (Roh et al. , 2017) “Large scale graph-based SLAM using aerial images as prior information” (Kümmerle et al. , 2010) ● Methods Rely ○ ○ ○ Direct building information (e. g. polygons from Open. Street. Map) Extracted building information (from aerial images) Image matching from scene and aerial imagery

SLAM Integration: Selected Method ● Use Direct Building Data ○ ○ More accurate than extracting building edges Updated frequently by RIT maps team ● Desired Characteristics ○ ○ ○ Formulated Using Pose-Graph SLAM Error Term Generic Enough to Additional Constraints (GPS, IMU, etc. ) Low Computational Overhead ● Selected: ○ “Improving SLAM by Exploiting Building Information from Publicly Available Maps and Localization Priors” (Vysotska et al. , 2017)

SLAM Integration: Selected Method ● Line (Building) Extraction: ○ ○ Douglas Peuker Algorithm: Convert Li. DAR Scan into Polyline Segmented into Disconnected Lines (Individual Buildings) ● Alignment: ○ ○ ○ Compute Misalignment Between Extracted Lines and Building Polygons Using Iterative Closest Point (ICP) Nearest Neighbors to Find Correspondences Compute Post-Alignment Pose ● Pose-Graph Error Function Changes to:

Design Concepts: Navigation ● How can we turn all of the data we have into sensible navigation commands for the cart? ● How will the robot know what it can drive on? ● How will we create a reasonable and safe path through the campus to reach a destination? ● What happens if something unexpected occurs? ○ ○ Someone walking in front of the cart A path is closed for construction Emergency Shutdown

Navigation: ROS Navigation Stack ● SLAM-friendly framework for navigating a robot through some environment ● Input ○ ○ Odometry information Sensor stream information Goal pose (w. r. t. map) Map (Optional) Localization ● Output ○ Velocity commands ● Maintains costmaps ○ ○ Store information about obstacles in the world Used to create navigation plans Mapping System

Navigation Stack: Global & Local Costmap ● Costmap ○ ○ Configurable structure Defines an occupancy grid ■ Each cell contains cost (0 -255) ● Local costmap ○ ○ ○ Local planning & obstacle avoidance Responsible for new objects, moving objects, etc. Generated from sensor data ● Global costmap ○ ○ ○ Long-term plans over the entire environment Responsible for buildings, trees, walkways, roads Generated from sensor data and its previous state

Navigation Stack: Global Planner ● Generates a navigation plan ○ ○ ○ Current location Destination Global costmap ● Continually publishes plans as they are computed during navigation ● Configurable

Navigation Stack: Local Planner ● Local plan generated from local costmap ● T. E. B. local planner ○ ○ ○ Plugin to the base_local_planner of the 2 D navigation stack Underlying method called Timed Elastic Band ■ Locally optimizes the robot's trajectory with respect to trajectory execution time, separation from obstacles and compliance with kinodynamic constraints at runtime Appropriate for car-like vehicles (Ackerman drive) ● Configurable

User Interface Idle Confirm Destination Idling Selection ● Simple state machine ● Map on bottom screen of cart ○ ○ Initialized when autonomous mode selected Hides all inner workings from user Confirm Navigating Idle Navigating Idling ● Nice-to-have ○ Diagnostic information on top screen ■ Battery life, date, time ● Will be developed with RQT ○ ROS gui framework Decline Stop

Design Concepts: GPS Integration ● ● Type of GPS Accuracy Reliability Cost

GPS Types ● ● L 1 L 2 GLONASS RTK/DGPS

Extended Kalman Filter ● Takes in IMU+GPS data ● Weighted average of observations ● Higher weight on estimates with higher certainty

GPS compare GPS VN-200 Nov. Atel PWRPAK 7 D-E Trimble LORD 3 DM-GX 5 -45 Type L 1 all L 1/RTK Cost $2, 700 $16, 745 $9, 000+ $2, 700 Accuracy <1 m <10 cm <1 m <2. 5 m IMU yes no yes

VN-200 outputs: ● ● ● ● ● Position Estimates in the following reference frames: ○ Latitude, Longitude, and Altitude ○ X, Y, Z position in Earth Centered Earth Fixed frame ○ X, Y, Z position in North, East, Down frame Velocity Estimates in the following reference frames: ○ X, Y, Z velocities in Earth Centered Earth Fixed frame ○ X, Y, Z velocities in the North, East, Down frame Attitude Estimates: ○ Yaw, Pitch, Roll ○ Quaternions ○ Rotation Matrix INS Filter Uncertainties ○ Position, Velocity, & Attitude GPS Time ○ GPS Time of Week ○ UTC Time Angular Rate Measurements: ○ Bias compensated angular rates ○ Calibrated gyro measurements Acceleration Measurements: ○ Bias compensated acceleration ○ Calibrated acceleration measurements ○ Gravity vector Magnetic Measurements Pressure Measurements / Altitude

Bill of Materials

Gantt Chart Plan Building Polygon Selection Aerial Map Integration Implement Algorithm ▁▁▁▁▁▁▁ 0% Integrate Algorithm w/ SLAM ▁▁▁▁▁▁▁ 0% Compare SLAM Performance (w/ and w/o Aerial Map) ▁▁▁▁▁▁▁ 0% Selection of Final SLAM Algorithm SLAM ███████▁▁▁▁▁▁▁ 50% ███▄▁▁▁▁▁ 25% Sensor Fusion ▁▁▁▁▁▁▁ 0% Map Persistence ▁▁▁▁▁▁▁ 0% Global Cost Map ▁▁▁▁▁▁▁ 0%

Gantt Chart Plan Configure Navigation Stack Navigation GPS Integration █▃▁▁▁▁▁▁ 10% Implement Inner Connectivity ▁▁▁▁▁▁▁ 0% Integrate SLAM ▁▁▁▁▁▁▁ 0% Navigation GUI ▁▁▁▁▁▁▁ 0% GPS Installation ▁▁▁▁▁▁▁ 0% GPS ROS integration █▃▁▁▁▁▁▁ 10% GPS RTK integration ▁▁▁▁▁▁▁ 0% GPS SLAM integration ▁▁▁▁▁▁▁ 0%

Risks ● ● Timeliness of ROS Nav-Stack Integration Efficacy of SLAM Methods GPS Performance in Urban Canyon Precision of CNN (E-Net) - Reduce False Positives

Test Plan Test Case Description SLAM Subsystem Operating Speed Localization and mapping subsystems will operate at a minimum rate of 10 Hz. The output is an allocentric or global position as well as a map update. SLAM on Occluded Path The cart needs to operate in conditions where its sensors are occluded, especially GPS, camera, and Li. DAR. Path Planning Path planning must yield a valid and efficient route between destinations. SLAM in Varying Weather Conditions Full SLAM system needs to operate well in various weather conditions that impact luminance as well as occlude sensor reading, such as via precipitation. Long-Distance Run Autonomously navigate from one side of the campus to the other safely. Employs use of all subsystems and sensors on a non-linear path though occluded and clear pathways.

Final Regards ● Submission to the 2019 EI Conference

Thanks for Watching

References ● ● https: //www 2. monroecounty. gov/files/gis/Base%20 Stations. JPG https: //images. duckgo. com/iu/? u=http%3 A%2 F%2 Fimages. fineartamerica. com%2 Fimagesmedium-large-5%2 Furban-canyon-philadelphia-city-hall-bill-cannon. jpg&f=1 https: //www. vboxautomotive. co. uk/images/products/Base-Station-Setup. jpg http: //www. unmannedsystemstechnology. com/wp-content/uploads/2013/12/VN-200 -Rugged-INS 1024 x 1024. jpg