VISION BASED RELATIVE NAVIGATION FOR AUTONOMOUS PROXIMITY OPERATIONS

VISION BASED RELATIVE NAVIGATION FOR AUTONOMOUS PROXIMITY OPERATIONS Dr. Declan Hughes Star. Vision Technologies College Station, TX Dr. John Valasek Flight Simulation Laboratory Texas A&M University Guidance, Navigation, and Control Technical Committee Lunch ‘n Learn NASA Johnson Space Center 8 January 2004 0

Vis. Nav PROXIMITY OPERATIONS outline of presentation ! PROXIMITY OPERATIONS (Valasek) ! VISION BASED NAVIGATION SYSTEM (Valasek) ! NUMERICAL EXAMPLE (Valasek) ! HARDWARE AND SYSTEMS (Hughes) ! LABORATORY DEMONSTRATION (Hughes) ! SUMMARY AND CONCLUSIONS (Hughes) ! QUESTIONS AND ANSWERS 1

Vis. Nav Personnel research assistants Ju Young Du, Kiran Gunnam, Roberto Alonso, Changwa Cho 2

Vis. Nav Personnel senior researchers and partners ! TEXAS A&M UNIVERSITY ? Dr. John L. Junkins ? Dr. John Valasek ! STAR VISION TECHNOLOGIES INC. ? Mr. Michael Jacox ? Dr. Declan Hughes ? Mr. Brian Wood ! SARGENT FLETCHER INC. ! AIR FORCE RESEARCH LABORATORY ? Munitions Directorate, Eglin AFB ! BOEING ? St. Louis 3

AUTO. PROXIMITY OPERATIONS considerations ! Ideal Scenario ? Quickly arrive at a space object ? Perform maneuvers without: i months of planning i intersatellite communication ? Utilize small mission operations center ! Desired Functionality ? ? Inspection Anomaly assessment Formation flying On-orbit servicing XSS-11 ! Requirements ? Three-axis stable ? Autonomously detect location and maintain attitude 4

AUTO. PROXIMITY OPERATIONS challenges XSS-11 1. Sensing relative position and velocity when in proximity to another object 2. Non-cooperative objects 3. Low power consumption 4. Advanced autonomous event planning 5. Forward thinking resource manager 6. Only two or three people should be required to plan and monitor the S/C 5

RELATIVE NAVIGATION approaches ! ! Radar corridors Ground tracking updates Scanning LIDAR GPS Local Positioning System (LPS) ! Optical-based navigation systems offer promising alternative ? Negligible diffraction, high bandwidth, no droputs ? Multipath reflections minimized by restricted field of view ? Optimal Signal-Noise (S/N) ratio i closed-loop control of beacon intensity ? System selects beacons from redundant set i robustness and flexibility 6

RELATIVE NAVIGATION pattern recognition ! Pattern recognition and visual servoing using a CCD camera to provide the line of sight vector for end game docking maneuver ? Reduced effectiveness in poor optical conditions (sunlight, etc. ) ? Depth of Field i camera must be able to focus over very long and very short distances ? Accuracy i determination of 3 D position coordinates from 2 D images ? Requires very high camera resolution (appx. 106 pixel data on each frame) ? Reliable pattern recognition i greater than 85% reliability is difficult to achieve, even in a perfect laboratory setting ? Requires high computational speed and data bandwidth i current data bandwidth is only 40 Hz with intermediate resolution 7

RELATIVE NAVIGATION Vis. Nav cooperative vision ! Optical sensor with active structured beacon lights that provides an accurate, high speed 6 -DOF navigation solution for the mid to end game docking maneuver. ! Update rate of 100 Hz and high precision under optimum conditions. ! Reduced risk: ? Feasible at current level of optical sensing technology ? Concept validated with hardware in laboratory experiments 8

Vis. Nav SENSOR position sensing diode ! Activated by energy from light sources ! Generates electrical current in 4 directions ! Current imbalances are linearly proportional to location of image centroid 9

Vis. Nav SYSTEM positions and attitudes ! Sensor on Vehicle A ? colinearity equations IU (yo, zo) (yi, zi) IR Pi IL Ideal pin hole camera model Pn z Line-of-sight vector observations ! 4 or more beacons with known locations y Beacons on Vehicle B P 1 ISC ID ! (X 1, Y 1, Z 1) x (Xc, Yc, Zc, f, q, y) PSD sensor output (Xi, Yi, Zi) Z (Xn, Yn, Zn) OSC 10 Y X

DIFFERENTIAL CORRECTION Gaussian least squares ! States Set initial current state ! Measurements (n 4) Compute correction ! Measurement Sensitivity Matrix yes ! no Measurement Residuals Update 11 Stop

EXTENDED KALMAN FILTER zero acceleration ! Linearize about estimated (reference) states Zero acceleration model (or IMU) ! States Compute ! Nonlinear system dynamics Compute gain ! Predicted state & cov. Update state & cov. ! Sensitivity Matrix Propagate state & cov. ! State transition matrix 12

Vis. Nav HARDWARE 6 -DOF algorithm ! 800 Hz beacon switching, 100 Hz 6 -DOF update rate ! Accuracies: ? ~ 1 cm/0. 25 deg at 30 m ? ~ 1 mm/0. 05 deg at 0. 5 m ! Modified Rodriguez Parameters ? Good convergence ! Options: ? Kalman filtering of sensor data ? Combined model/Kalman filter. ! Beacon selection criteria options, computed in real-time ? 6 -DOF data covariance matrix condition number ? Apparent beacon selection width and depth of field 13

Vis. Nav APPLICATION precision landing Beacons Sensor V/STOL Ship Landing Aircraft Landing 14

Vis. Nav APPLICATION autonomous aerial refueling Smart. Lite Beacons 15

Vis. Nav APPLICATION autonomous aerial refueling 16

Vis. Nav APPLICATION data glove 17

Vis. Nav APPLICATION S/C docking 18

NUMERICAL EXAMPLE 19

NUMERICAL EXAMPLE Position and Attitude errors of GLSDC 20

NUMERICAL EXAMPLE Position and Attitude errors of Extended Kalman Filter 21

Vis. Nav HARDWARE psd sensor construction ! Optical filter to block visible light. ! Wide angle lens focuses wide field of view onto PSD ! Approx. 3” x 3” 22

Vis. Nav HARDWARE psd sensor ! Wide angle lens focuses wide field of view onto PSD ! Approximately 3” x 3” 23

Vis. Nav HARDWARE optical filter ! Infrared LED ? close to maximum PSD response ! Thermal noise dominates at low illumination ! Shot noise proportional to sqrt(rms PSD current): sunlight ? large current and shot noise dominates 24

Vis. Nav HARDWARE micro computer ! Small computer ? ? micrporocessor: TI TMS 320 VC 33 DSP @ 60 MHz IMbyte SRAM 0. 5 MByte Flash Eprom 120 MFLOPS, 1 W ! Circuit outline = 2. 3” x 3. 3” ! Analog interface circuit stacks on top 25

Vis. Nav HARDWARE active beacon ! Control signal carrier at ~ 40 KHz ! Largest beacon is 218 LED design ! Light Shaping Defuser (LSD) positioned in front of LED ! Red or IR optical filter ? protects plastic LSD and LEDs from sunlight ! 1 W optical ? ~ 10 W electrical 26

Vis. Nav HARDWARE active beacons ! Three Beacon Sizes ! Largest Beacon = 218 LED design. ! Stacked board design, V->I circuits behind ! A few red LEDs are used for visual check 27

Vis. Nav HARDWARE active beacons ! Aluminum fabricated boxes ! Glass front plate (colored glass can also be used) ! Light Shaping Diffuser (LSD) lens. ! One push-pull connector 28

Vis. Nav HARDWARE new LEDs ! > Watt emitted energy. ! Wide Variety of wavelengths, including 880 nm. ! Tailor radiation pattern by cutting flat surface. 29

Vis. Nav HARDWARE calibration and test rig ! Yaw-Pitch axes actuator 1 for sensor calibration ! X-Y-Z-Yaw-Pitch-Roll actuator 2 to test complete system accuracy ! Beacons placed on optical table 30

Vis. Nav HARDWARE calibration ! Divide by beacon intensity ! Measure at many yaw/pitch data points ! Reduce data set by calculating normalized voltages 31

Vis. Nav HARDWARE calibration ! (L-R)/(L+R) ? intensity effect removed 32 ! Measure at many yaw/pitch data points ! Invert surface and fit

Vis. Nav SYSTEM application l First VISNAV system application. l Beacons placed in Nasa JSC NSTL room on walls and on a movable frame. l Frame may be moved outside; used for docking simulations. l VISNAV will calibrate differential GPS sensor system. 33

Vis. Nav SYSTEM docking simulations ! Beacon frame contains 8 x 60 LED beacons, and 8 x 12 LED beacons. ! Grey box = beacon controller. ! Sensor uses large beacons when in field of view. ! As sensor approaches lower right hand corner it switches to smaller beacons that remain in field of view. 34

Vis. Nav SYSTEM docking simulations VISNAV 3. 2 Schematic 35

Vis. Nav SYSTEM options ! Temperature sensing and compensation. ! Phase lock sensor to incoming signal => better accuracy. ! Daisy chain serial beacon control cable => less wiring. ! Digital transmission => no noise pickup. ! Wireless beacon controller to beacon connections, or add beacon controller to each beacon => no separate beacon controller, no control cables. 36

Vis. Nav SYSTEM wireless IR link ! ! IR wireless sensor to beacon controller link, 115. 2 Kbaud. Crystal locked FSK. Low latency and latency variation. PCB 1. 8” x 1. 4” approx. 37

Vis. Nav SYSTEM options ! Self-test, calibration LEDs in sensor. ! Accelerometers ? Wider bandwidth/less noise. ! Smaller beacons, circular shape. ! Ultrasonic version ? 6 DOF relative navigation underwater. Relate to R/C GPS boat(s) on surface. ! 120 MFLOP TI TMS 320 VC 33 DSP, 150 MFLOP version, or 1 GFLOP TI TMS 320 C 6701 DSP card; still 2. 3” x 3. 3”. ! Addition of LIDAR ? More accurate estimates; useful at longer ranges. ! Designing passive beacon version 38

Vis. Nav SYSTEM smaller DSP Small Computer u. P = TI TMS 320 C 5509 DSP @ 200 MHz 8 Mbyte SRAM, 0. 5 MByte Flash Eprom, 400 MIPS, 0. 5 W Circuit Outline = 30 mm x 36 mm Firewire and many other interfaces. 39

FUTURE DIRECTION stereo vision geometry Sensors on UAV Iup (y. A, z. A) Z Iright z Ileft ISCA Idown OSC y Drogue Beacon PSDs A : (Xc. A, Yc. A, Zc. A, s 1 A, s 2 A, s 3 A ) (Xi, Yi, Zi) y z x Ileft ISCB Iright (y. B, z. B) ! Measurements X x Iup ! Determine position of UAV using LOS vector from two different cameras with known position and attitude States ! Y Idown PSDs B : (Xc. B, Yc. B, Zc. B, s 1 B, s 2 B, s 3 B ) 40

CONCLUSIONS ! Accurate 100 Hz update rate 6 DOF data possible with small low power sensor/beacons to 100+m. ! Beacon signal modulation and optical filtering ? Excellent ambient light rejection. ! Realtime beacon selection/intensity control ? Minimize power requirements. ! Pattern recognition problem effectively eliminated. ! Very wide field of view, no moving parts. ! Distributed beacons ? Very large operating space, redundancy. ! Facing sun operation estimated appears feasible; yet to be demonstrated. 41