Modular Design in Robots for UnderVehicle Inspection Cheng

Modular Design in Robots for Under-Vehicle Inspection Cheng Qian Imaging, Robotics, & Intelligent Systems Laboratory University of Tennessee Spring 2004

Outline 1. Modular concept: “Three primitives motivation ”, Modular architecture 2. Module: Structure, 3. Data storage, 4. Work thread, 5. Synchronization, 6. Priority 7. Implementation 8. A realistic system based on modular design. 9. 9 10. 10 11. 12. DLL Structure Sensing DLL Programming

Modularity Concept: Three primitives 1. A robot is composed of three basic primtives: Sensing, Mobility, Computing Sensing Mobility Mechanical & Power support Information Strategy Computing Robot ( CMU Millibot ) 2. Every primitive is further decomposed into sub-modules

Modularity Concept: Modular Architecture Sensing Sensor ON Sensor 1 Processor Sensor 2 … Power Sensor N Sensor platform Virtual hardware control (Resampling&recalibration ) Lighting QOS Filter (noise filter & feature enhancement ) Motors Modality conversion Communicator Control circuits Wheels Source Compression&Coding Monitoring sensor Mobility Channel Coding Protocol Reactive inspection behaviors Sense 1 Data fusion Data Interpretation Display Operator Sense 1 Strategy planning Deliberative inspection Act 1 Reactive navigation behaviors Deliberative navigation Act 1 Computing

Module: Structure In under-vehicle system, the data is passed, one module after one module, from the start ( sensing ) to the end ( mobility ). So every module is modeled as : Fresh frame of data In. Buffer Process Out. Buffer 1. Read the fresh data from the front modules into the Inbuffer. 2. Process the fresh data. 3. Write the Processed data into the Outbuffer, waiting for back modules to pick up. Module X *The three steps above are executed sequentially, hence, a Module = a Thread. *It is a real-time system, and the sensors are scanning frame by frame, all the time, hence Every round, the module only reads the fresh frame, process the fresh frame, and write the fresh results, and Size of buffer = Size of a frame.

Module: Data Storage The In. Buffer and Out. Buffer only store one frame of fresh data and a frame of fresh processed data. The size of the buffer in the every module is equal to the size of a frame , and every module is only processing the data of the fresh frame. But what if the previous frames are needed in the processing of the module? ――Add buffers to backup the previous frames. The whole data ever read are kept in a temporary file Store: Module X If N Previous frames is frequently needed In. Buffer ( current frame) Backup. Buffer 1 ( Previous Frame 1) Backup. Buffer 2 ( Previous Frame 2) : Backup. Buffer N ( Previous Frame N) * Temp File New Buffer ( N* size of Frame) Move data in the Backup. Buffer. N to Temp file Move data in Backup. Buffer i to Backup. Buffer i+1, i=1~N-1 Move data in the In. Buffer to Back. Buffer 1 Read data into the Inbuffer Retrieval: Search In. Buffer, If failed, Search Back. Buffer i, i=1~N If failed again, Search Temp file

Module: Work Threads Normally, a module only has one work thread. If it is a leaf module without downlinked modules, and without critical real time requirements, the processing in the module is first partially finished, if time permits, other processing is i. e. Display continued. Wait Display data in backup buffer Display data in temp file Display data at low resolution New work (displaying) thread is created while the module is waiting for fresh data to come 1. Initialize the resolution Res = min ( minimum resolution required, ½*Res of last round ) 2. Display data at Res 3. Store 2*Res data into backup buffer 1, ……, 2 N Res data into backup buffer N, and store other data into temp file 4. While Waiting for data of next round, another threading of displaying the data in backup buffer and temp file sequentially is created, and is to be terminated by the termination of waiting

Module: Synchronization Activated by Event, and Terminated by Event Module i 1 …… Module i. M 1. Done-Event from a uplinked module i: → Reading data from that module i into inbuffer i is activated 2. The reading from module i is finished Then ACK is sent to the uplinked module i 3. The readings from all the uplinked modules are finished Then resources are caught and Process is started → Wait … In. Buffer 1 Wait …. . In. Buffer M …. . → Process…. 4. → Out. Buffer Mod ule X Done-Event Wait ACK Data Process is finished Then resources are freed and check wait for ACK from downlinked module. 5. ACK from all downlinked modules Writing the processed data into Outbuffer is activated 6. Writing is finished Then Done-Event is sent to all the downlinked modules and wait for Done-Event from uplinked modules to start a new round. → →

Module: Priority Every module, as a single thread, is executed independently, and to make full use of the computation capability of the computer, the Priority is scheduled as: Fixed, i. e, The priorities of the reactive behavior and data acquisition, are always high. Flexible: Updated when suspending happens Suspended 1. If module Mi causes its back module suspended, MI Ei: Set of module Mk suspended due to Mi, Tk: Suspended Time of Mk Pk : Current priority of Mk , Pi : Current priority of Mi Mi Suspended by MI, MII Mk 2. If Mi is suspended, Suspended due to Mi —— priority of Mi decreases by Ti: Fi: Suspended Time of Mi, Pi : Current priority of Mi Set of module MI suspends Mi, PI : Current priority of MII Too slow —— priority of Mi increases by MI

Module: Implementation Modularity Object-Oriented Programming Definition of Attributes & Behavior CLASS Activation THREAD MFC CThread. App (Module) Void Module X : : Run Module X: Public CThread. App { Loop: { Public: // interface attributes BYTE In. Buffer [ Size. Frame ], Wait for Done-Events from uplinked modules, ( and Create other work thread ) BYTE Out. Buffer [Size. Frame ] If Wait returns, MODULE * Up. Link [ ], * Down. Link [ ] EVENT * Done. Event, * ACK Read data into input buffer Protected: // Internal attributes BYTE Backup. Buffer [ ][Size. Frame] FILE * Temp. File HANDLE * Resource [ ] Private: // Behaviors virtual BOOL Init ( ); virtual VOID Run( ); then (terminate the work thread ) Send ACK back to Uplinked modules Catch Resource …. Process…… Free Resources Wait for ACK from back modules If Wait returns, } then write data into downlinked buffer Set Done-Event Endloop }

A Realistic System Computing = Laptop + Application Sensing = Sensor + Plug-in Driver ( DLL ) Mobility = Mobile Platform + Plug-in Driver (DLL) Application and Drivers are modular-based developed Sensing Computing Mobility

DLL Structure Mobility DLL : Strategy Explanation & Parsing Strategy Vector Creator 1 Creator 2 Strategy Vector Creator. N Communication Port 1 Port 2 Port N … Sensing DLL: Data Acquisition Serial Processing Data Formatting How to build three sensing DLL ? Modality Sensor Range Sick Scanner Texture Line CCD camera Thermal Area thermal temperature) Sensor working mode Data format Continuous (Step up once) continuous points ( 3 D Pos ) Setup before capture sequence of Line ( RGB ) Setup before capture sequence of images (relative

Sensing DLL Diagram Sick. Scanner Adaptive_HLRCLine. CCD Range_Sick. Scanner. LIB Texture_HLRCLine. CCD. LIB Serial communication Sensor Protocol Read FIFO PAK 1 PAK 2 … PAKN Parallel communication Sensor Protocol Serial Proc (Color Calibration) Formatting Range Data Calibration parameters Frame grabber Protocol Sensor Protocol Serial Proc (Noise filtering) PAKN SDK Programming Parallel communication Virtual Setup Write FIFO PAK 2 … Thermal_Raytheon. LIB SDK Programming Frame grabber Protocol PAK 1 Raytheon_Thermal. Camera Setup Application FRM N Virtual Setup Serial Proc (Temperature Normalization) Formatting Write FIFO Line 1 Line 2 … Line N Texture Data Read FIFO FRM 2 … FRM 1 Write FIFO FRM 1 FRM 2 … Calibration parameters Thermal Data FRMN Calibration parameters

“Serial Communication “Programming Scanner Protocol Write Command ( ) { ……. Wait (Data. Continue) Move data to Wbuffer Set. Event( Data. To. IOPort ) } Serial communication Done-Event Write. Thread Loop { Wait ( Data. To. IOPort ); Write. File ( * Wbuffer ) If ( Wbuffer is empty and writing is finished ) Set. Event ( Data. Continue ) } ACK Read Commamd( ) { ……. Wait ( Datafor. Pickup ) Move data from Rbuffer Set. Event( Data. Moved ) ACK } Read. Thread Loop { Wait ( Datafrom. IOPort ) Done-Event If ( Rbuffer is refillable ) Read. File( * Rbuffer) If ( Rbuffer is full or reading is finished ) Set. Event ( Datafor. Pickup ) } Done-Event RS 232 IOPort

Inverse Problem in Thermal Imaging Cheng Qian Imaging, Robotics, & Intelligent Systems Laboratory University of Tennessee Spring 2004

Outline • Inverse problem • PSF building • Inverse Results

Inverse problem g: blurred image PSF: blurring operator fopt: restored image : : unknown Regularization parameter Regularization norm Tikhonov Regularization: Favor smooth f TV Regularization Favor f of large variance Lasso Regularization Favor f of large variance, and ease the computation

PSF building Find a image of a point heat source Small and isotopic Use this PSF to restore thermal images Experiment setting Image of a point heat source Average over 10 continuous capture and over 10 experiment settings Approximated by Gaussian function

Inverse Results Original image Tikhonov Regularization TV Regularization Lasso Regularization

THE END Thanks