Advanced Video Surveillance archives search Engine for security

Advanced Video Surveillance archives search Engine for security applications Security Research - Capability Project - SEC-2011. 5. 3 -4 Video archive search Advise project presentation Tomas Piatrik (QMUL)

WHAT IS ADVISE? ADVISE is a research project co-funded by the FP 7 -Security Work programme of the European Commission, aimed at designing and developing a unification framework for surveillancefootage archive systems. The ADVISE project results will ease the work of law enforcement authorities in their fight against crime and terrorism, through negotiation of all relevant legal, ethical and privacy constraints, and through location based video archive selection and efficient evidence mining of multiple, heterogeneous video archives. 30/09/2020 Project Presentation 2

ADVISE CONSORTIUM The project is performed by ten partner organizations coordinated by Engineering Ingegneria Informatica. 30/09/2020 Project Presentation 3

MAIN FUNCTIONALITIES ADVISE could offer the functionalities to further enrich the uploaded information to Analysis Work File from Member States competent authorities to EUROPOL, by offering images, video and the sophisticated metadata, accompanied by the Legal, Ethical & Privacy restrictions that apply to the content’s usage. The ADVISE system will collect and analyse information of the suspects’ characteristics which could be identified from the surrounding cameras and a pattern of the culprits will be extracted. This pattern will be used in tracing the suspects before and after incidents, through out any available video archive. 30/09/2020 ADVISE will create specific Geographical Information Systems (GIS) overlays that depict in detail the surveillance architecture providing the first surveillance – specific map. The system will combine location and statistical information and this combination will be visualized into a GIS system. Project Presentation 4

Knowledge Representation Framework Slide 5

Case study sample The red car (object 101201) enters from north-west and exits from south-west of the CCTV view – Geometrical Ontology The white van (object 101221) is parked on the road from 10. 42. 12 AM to 10. 52. 45 AM – Time coded Event ontology The blue car (object 914134) taking an left turn at junction, and hence will be visible in CCTV (1433511) – Geospatial Ontology Slide 6

Privacy-by-Design framework LEP Ontology (ext. SKOS and FOAF) 30/09/2020 Project Presentation Information Processing Individual rights Component Functionality 7

Camera Setup CAM 4 CAM 5 CAM 6

Events Recorded Pickpocketing Luggage theft 7 scenarios Bag/Backpack theft 6 scenarios 11 scenarios Fight 9 scenarios

Adding context awareness SCENE MODELLING

The Concept Goal: Automatically build a context aware scene mapping, using detected trajectories. Identify Points-of-Interest. Entry/Exit points. Meeting spots. Waiting spots. Identify occlusion areas. Predict trajectories. Measure prediction confidence.

Trajectory Prediction: Predict future positions of moving object based on: Trajectory history. Gaussian process models. Evaluate prediction confidence: Cluster nearby trajectories based on direction. Compute confidence of prediction direction.

A CONTEXT AWARE TRACKER OBJECT TRACKING

Object Tracking modules overview Tracking-by-detection approach C 4 pedestrian detector Target-specific adaptive tracking. Online learning Compressive tracker “Real-Time Human Detection Using Contour Cues”, Wu J. , ICRA 2011. “Real-Time Compressive Tracking”, Zhang K. , ECCV 2012. Scene modeling assistance.

C 4 Pedestrian Detector Object detector § Detection map C 4: Human Contour using a Cascade Classifier and the CENTRIST visual descriptor. § Contour is key information for human detection. § CENsus TRansform h. ISTogram descriptor. § Near real-time (5~20 fps) execution per frame (CPU only). § So. A accuracy and performance.

Compressive Tracker (CT) Target-specific online learning tracking Adapts on specific target. Appearance based features. Robust to appearance changes. Online training with positive/negative samples. Provides tracking confidence level. So. A accuracy and performance. Near real-time execution per target. Positive sample: Negative sample:

Identifying pedestrians across different cameras INTER-CAMERA PEDESTRIAN TRACKING

Re-acquiring a pedestrian Challenges Different cameras Lightning Scale Resolution Viewpoint Small (intra-class) differences in appearance. Goals Robust appearance-based modeling. Maximum intra-class discriminative power.

Pre-Processing Segmentation Tri-zone probabilistic mask. White: Pedestrian. Gray: Probably pedestrian. Black: Background. Grab. Cut[1]. Advantages Efficient for automation. Tracker error counterbalance. Center re-alignment. Accurate box cropping. a b c Preprocess unit [1]. C. Rother, V. Kolmogorov, A. Blake. Grab. Cut: Interactive Foreground Extraction using Iterated Graph Cuts. ACM Transactions on Graphics (SIGGRAPH'04), 2004 d

GROUP IDENTIFICATION

Group Identification Group: A collection of individuals performing a combined action. Detection Criteria: Trajectory metrics for: Past trajectory. Future trajectory (Prediction). History of Group membership. Degree of Confidence: Percentage of group confidence.

Trajectory Metrics used for Group Identification: Orientation of trajectory Spatial proximity Velocity Similarity Convex Hull (Ratio of areas)

Event Description Events: Meeting: Loitering People forming a group are stopped for more than a pre-defined timespan. Walking Together People already existing in scene form a group. People forming a group are moving. Split People do not belong in the same group anymore.

Advanced Video Surveillance archives search Engine for security applications THANK YOU FOR YOUR ATTENTION! CONTACT : contact@advise-project. eu www. advise-project. eu ADVISE project coordinator: ENGINEERING INGEGNERIA INFORMATICA - Italy 30/09/2020 Project Presentation 24