Computer Vision Projects 5 Vincenzo Caglioti Giacomo Boracchi

Computer Vision Projects (: , 5) Vincenzo Caglioti Giacomo Boracchi, Simone Gasparini, Alessandro Giusti, Pierluigi Taddei Computer Vision Group

Computer Vision Group Team § § Vincenzo Caglioti Giacomo Boracchi, Simone Gasparini, Alessandro Giusti, Pierluigi Taddei Computer Vision Group

Reconstructing Canal Surfaces, trajectories and spin of moving balls We can reconstruct circular-cross section canal surfaces from a single image (main algorithms are already implemented in well-packaged Java methods). Computer Vision Group

Project proposals §Explore possibilities for a 3 D input device (maybe real-time? ) with a webcam + flexible tube. Develop demo application (Open. GL or Java 3 D). • Example: virtual flexible stick-figure. • Example: 3 D modeling of flexible objects. • Be creative! : ) • Requisites: learn principles of 3 D visualization §Quantitatively evaluate performance of current algorithms, and test possible improvements. Implement camera autocalibration. • Prerequisite: knowledge of camera calibration and projective geometry (Image Analysis and Synthesis classes). Computer Vision Group

Other applications With the same algorithm we can reconstruct the trajectory of a moving ball from a long-exposure photograph (no frame rate issues, can handle very fast games, any lighting condition, inexpensive equipment). § Reconstruct the trajectory of a real moving ball. Evaluate reconstruction accuracy. Can we also measure nonparabolic trajectories (e. g. Pirlo’s penalty kick, spinning table tennis shots, volleyball “floater” serves…)? Mostly implementation work, some interesting possible optimizations. § Implement an automatic refereeing system for table tennis. § Implement a system for detecting the exact bounce position of a fast-moving ball from its blurred trail (“in or out? ”). § Augment low-frame rate videos of table tennis matches (think of the new Shell’s TV advertisement). For example: • Draw ball “shadow” on the table (3 D reconstruction) • Draw ball velocity vector • Predict the remaining trajectory portion • Manipulate the ball trail opacity and/or color (image processing) • Estimate ball speed • Synthethize a 100 fps slow motion replay from a 20 fps video… Computer Vision Group

Ball spin from a single image (ongoing research) If the ball surface is textured: analyze the trail and find the ball spin (axis and rotational speed) – ongoing research • 3 D geometry issues • Image processing issues § Find ball spin axis and speed from traces left by dots on the ball surface (long-exposure). § Find ball spin axis and speed from blur of the ball’s surface features (short-exposure) -- joint work with Giacomo Boracchi. § Find ball spin speed from orthographic images (long-exposure) look for periodic color patterns. Computer Vision Group

Tram transit detection and notification § A webcam will be placed on the DEI building pointing at Via Edoardo Bassini. § The video stream will be analized in order to detect the transit of any ATM tram, identifying its number and registering the transit time § A web framework will be then implemented in order to predict the next tram transit § The system will be exploited by the department employees in order to leave the office at the last usefull moment Next predicted transits: heading to Duomo: 5’ 23’’ heading to Lambrate: 2’ 12’’ Computer Vision Group

Structure from Motion § The aim is to reconstruct the 3 D structure of a scene and the camera motion using as input only the video sequence caputred Computer Vision Group

Project 1: Surface fitting § Given the 3 D points and the initial image frames identify reliable surface patches onto which the initial images can be mapped Coordinators: Caglioti, Taddei Computer Vision Group

Project 2: Feature Tracking for Structure from Motion § § The features tracked may disappear due to occlusion or to wrong matches between images A good tracking algorithm should compensate for these effects and extract as many features as possible Coordinators: Caglioti, Taddei Computer Vision Group

Project 3: Paper Like surfaces § § The object recorded is assumed to be a paper-like surface that is represented by a particular family: developable surfaces The project will be aimed to build a framework to generate sintetic datas to test the alghoritms Coordinators: Caglioti, Taddei Computer Vision Group

Motion Estimation from a Single Blurred Image § Application: 3 D reconstruction from a single image • Local motion extraction from blurred details (Corners/Texture) • Exploit Global Camera Movement Computer Vision Group

Motion Estimation from a Single Blurred Image § Image Restoration: De-Blurring • Build a “Blur Map” • Adapt Existing De-blurring Techniques to real blurred images Computer Vision Group

Robbery detection § Objective: automatically detect holdup situations (“Hands Up!”) from video-surveillance sequences on a dsp-equipped camera. Background subtraction Color-based skin segmentation Computer Vision Group Detection of “hands up” pose

Robbery detection Face and hands detection Computer Vision Group

Robbery detection - Available projects § Pose recognition • Develop an approach based on silhouette extraction and pose recognition § Face detection • Improve performance and robustness of face detector, test on a larger training set § Hands detection • Develop a new detection algorithm (similar to the face detection one) and test performance (hit rate and computational speed) § Skin segmentation • Improve performance of the skin detector using a voting system involving three color spaces RGB, YCb. CR, HUV. Coordinators: Caglioti, Boracchi, Gasparini, Giusti, Taddei Computer Vision Group

Rectification of perspective images § Objective: removing perspective effect from images Perspective Image Computer Vision Group Rectified Image

Natural image recognition through compression level analysis § Natural image recognition • Recognition of natural image (e. g. leaves, flowers) by compressing the contour image and matching the compression levels Coordinator: Caglioti Original image Computer Vision Group Edge image

Rectification of perspective images § Available project: • 3 D reconstruction of urban scene from uncalibrated images for virtual tour Coordinator: Caglioti Computer Vision Group

Calibration of catadioptric camera § Catadioptric camera: a perspective camera placed in front of a curved mirror Catadioptric camera Mirror Camera Computer Vision Group Catadioptric images

Calibration of catadioptric camera § Calibration procedure • Develop a new calibration procedure for catadioptric cameras from single image exploiting the silhouette of the mirror and the alignment constraints deriving from the image of straight lines. Coordinator: Caglioti, Gasparini, Taddei Computer Vision Group

License Plate Recognition § Objective: automatically detect and recognize license plate from video sequences on a dsp-equipped camera. YZH 4025 Computer Vision Group

License Plate Recognition – Available Projects § § § New Starting Project Probably Strict Deadlines ONLY THE BRAVES! § License Plate Detection Module • Develop a module that detect the license plate in image according to color and shape § License Plate Recognition Module • Given the license plate image, develop a module that recognize characters (e. g. using a neural network) and provide the license number § Both projects require good C-programming skills § Coordinators: Caglioti, Gasparini, Taddei Computer Vision Group

Robot mapping § Objective: built a map of the environment collecting laser scans while robot is moving Map built by collecting scans Laser scanner § Scanning while moving algorithm • Implement and test on a real robot (Mo. Ro 2) the mapping algorithm − Coordinators: Caglioti, Gasparini Computer Vision Group

Thank you… Further informations avaialble at www. elet. polimi. it/people/caglioti/ {caglioti | boracchi | gasparini | giusti | taddei}@elet. polimi. it Computer Vision Group