Patton Visuality TM Platform Vehicular Communication Systems for

Patton Visuality. TM Platform Vehicular Communication Systems for Situational Awareness Gun Akkor Patton Electronics Co. gakkor@patton. com

2 EMERGENCY SERVICES WORKSHOP Roadmap • • • Situational awareness System architecture Application challenges Performance evaluation Future research Concluding remarks 9/29/2020

3 EMERGENCY SERVICES WORKSHOP What is Situational Awareness? And why do we need it? 9/29/2020

4 EMERGENCY SERVICES WORKSHOP 9/29/2020 Situational awareness • Collect real or near real-time sensory input from mobile and fixed assets: ▫ ▫ Video Audio GPS Other I/O (e. g. on-board car diagnostics, door locks, panic button) • Aggregate at a central location to provide a cognitive reaction to a condition or event.

5 EMERGENCY SERVICES WORKSHOP 9/29/2020 Situational awareness (cont’d) • Make data available to multiple users ▫ Support different platforms: desktops, laptops, handheld computers, PDAs and Smartphones ▫ Allow content sharing between local and remote users • Allow remote configurability of the system to adapt to changing conditions on the field.

6 EMERGENCY SERVICES WORKSHOP 9/29/2020 Applications • Support of law enforcement personnel ▫ In police cruisers replacing existing DVR solutions • Emergency response assistance ▫ In coordination of fire and rescue operations • Protection of personnel, vital equipment and goods ▫ School buses ▫ VIP protection ▫ Guarantee integrity of transferred goods • Transfer of valuable commodities ▫ Protection of armored vehicle fleets

7 EMERGENCY SERVICES WORKSHOP 9/29/2020 Components of Visuality Platform A high-level overview

8 EMERGENCY SERVICES WORKSHOP System Architecture 9/29/2020

9 EMERGENCY SERVICES WORKSHOP 9/29/2020 In-vehicle “remote” units Model T 7714 Model T 7712 • Up to 4 composite (NTSC/PAL) camera inputs • Up to 4 line-level mono audio sources • On-board GPS receiver with standard NMEA-0183 GPGGA and GPVTG output • Video encoded (MPEG 4) up to D 1 (720 x 480) resolution and 30 fps per channel for on-board storage on hard disk or SD/SDHC card • Video encoded (H. 264) up to SIF (352 x 240) resolution and 15 fps per channel for over the air transmission

10 EMERGENCY SERVICES WORKSHOP 9/29/2020 In-vehicle “remote” units (cont’d) Model T 7714 Model T 7712 • Audio sampled at 22. 05 k. Hz, encoded (AAC) at 16 -128 kbps • Metadata and audio multiplexed with video as an MPEG 2 TS • PTZ controller on RS-485 serial interface (supports Pelco-D and Rvision protocols) • Support for multiple cellular network technologies (GPRS, EDGE, 1 x. RTT, EVDO, HSPA) using PCI Express form factor modems • 12 V DC input via standard car or rechargeable lithium ion battery

11 EMERGENCY SERVICES WORKSHOP 9/29/2020 Command “headend” server • Compact PCI computer blade with accompanying 2 U rackmount chassis • Gatekeeper of the entire system • Terminates VPN connections • Aggregates received content and makes it available as multicast streams to local and remove users • Serves KML pages that can be network-linked from Google Earth • Supports asset sharing up to 24 remote units

12 EMERGENCY SERVICES WORKSHOP Command shift concept 9/29/2020

13 EMERGENCY SERVICES WORKSHOP 9/29/2020 Command shift concept (cont’d) • Remotes register with the primary command server (CS) at startup and pull their current assignments. • Secondary command servers also register with the primary and pull the list of remotes that are assigned to them. • Primary CS can move any asset to/from a secondary and make content available to operators of the secondary. • Primary CS can recall any unit when needed.

14 EMERGENCY SERVICES WORKSHOP Design Challenges Trade-offs and design choices 9/29/2020

15 EMERGENCY SERVICES WORKSHOP 9/29/2020 Video compression algorithms • Adaptive frame rate change (3. . 30) • Adaptive bit-rate change (40 Kbps – 2. 5 Mbps) • Adaptive quantization – dynamically adjust QP parameter to target constant bit rate (CBR) • Increased group of picture (GOP) size • Only P-frames, no B-frames are used to eliminate buffering needs at the receiver • Force an I-frame when scene change is above 40%

16 EMERGENCY SERVICES WORKSHOP 9/29/2020 Remove command control • Common Object Request Broker Architecture (CORBA) ▫ Standard defined by Object Management Group (OMG) ▫ Allows multiple machines to communicate over the network over IIOP protocol (Internet Inter-orb Protocol) ▫ Well defined uniform application programming interface (API) ▫ Transparent access whether local or remote

17 EMERGENCY SERVICES WORKSHOP 9/29/2020 Adaptive rate control Measured EV-DO Rev. A Rates Rate (kbit/sec) RSSI (d. Bm) 800 700 600 400 300 200 100 0 -100 -200 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 103 106 109 112 Rate/RSSI 500 Sample Points

18 EMERGENCY SERVICES WORKSHOP 9/29/2020 Adaptive rate-control (cont’d) • Light-weight probe packets sent from remotes to headend to record round trip time (every 2 seconds) • Queue backlog on the cellular network interface is measured (every 2 seconds) • A moving average window is used to smooth measurements (5 to 20 -sample size) • Moving average backlog value is compared to a configurable threshold

19 EMERGENCY SERVICES WORKSHOP 9/29/2020 Adaptive rate-control (cont’d) • Rate is adjusted such that maximum amount of data put on the queue in one RTT time never exceeds the available queue space – a simple delay-bandwidth product alike approach

20 EMERGENCY SERVICES WORKSHOP Prototype setup System evaluation 9/29/2020

21 EMERGENCY SERVICES WORKSHOP 9/29/2020 Experimental Setup • 5 -vehicle fleet • Each equipped with 4 NTSC cameras facing front, rear, left and right side • Cellular-GPS combo antennas places on the roof of each vehicle • Units are configured to transmit all 4 -streams, CIF size (352 x 240) , 10 fps and 100 kbps

22 EMERGENCY SERVICES WORKSHOP 9/29/2020 Experimental results Adjusted rate on the cellular uplink Instantaneous and moving average round-trip-delay (5 sample window)

23 EMERGENCY SERVICES WORKSHOP 9/29/2020 Visuality System in Action A short demonstration video

24 EMERGENCY SERVICES WORKSHOP 9/29/2020 Future directions • Alternative wireless technologies ▫ Wi. Max ▫ LTE • Incorporate vehicle’s on-board diagnostic computer (OBD) ▫ Engine codes ▫ Sudden acceleration/de-acceleration • Connecting tracking devices, PDAs, digital video recorders over 802. 11 or 802. 15

25 EMERGENCY SERVICES WORKSHOP 9/29/2020 Future directions (cont’d) • Improve adaptive rate-control ▫ Smooth out instantaneous rate spikes ▫ Investigate scalable video extensions for rate adaptation • Open-platform software and hardware ▫ Open to integration of other software and hardware modules