2015 Microsoft Indoor Localization Competition Dimitrios Lymberopoulos Microsoft
2015 Microsoft Indoor Localization Competition Dimitrios Lymberopoulos (Microsoft Research) Jie Liu (Microsoft Research) Xue Yang (Intel) Niki Trigoni (University of Oxford) Anthony Rowe (CMU) Ayman Naguib (Qualcomm)
Background • 48 systems from 44 teams submitted abstracts • 28 systems showed up • 23 systems actually evaluated • Academia, Industry • Two categories • Infrastructure-free (7 teams) • No hardware deployment • Conference venue Wi. Fi and phone sensors • Infrastructure-based( 16 teams) • Custom hardware deployment
Evaluation Process • 2 -day event • Day 1: Monday • Teams were given 7 hours to setup and calibrate their systems • Day 2: Tuesday • Each team was asked to provide the coordinates of 20 test points • Test points’ coordinates were manually measured using laser range finders • Evaluation Metric • Average localization error across the 20 test points
Y Axis 2000 m 2 Evaluation Area Origin Point (0, 0) X Axis
Fun Times…
Towards a standardized evaluation of indoor location technologies § Multi-institutional effort § NIST, TU Berlin, and i. Minds at University of Ghent § Goal: create a large repository of labeled indoor location data traces § 5 buildings on the NIST campus in Gaithersburg, MD § 1800 test points § More than 30000 m 2 § Data Collection § September – December 2015 § Available through a web service § Ideas/Feedback/Suggestions § Nader Moayeri (nader. moyaeri@nist. gov)
La t a l Ti zik e. m Qu e t al an D tit Cam om. ec p a Gm be in l b. H l et In al. t Si ran m on av Kl et a ip p l. Ch et a l. e Ze n e SP ng t a IR l IT en e. Na t a vi ga l. t Sy Elia ion m in s et gt on al. e Zo t al. u e Pi t al rk le. t W al. u e Ba t a ta l. et Qi al. u et al. Ni ko Na de vix m De et or al. a e M Ku t al. irs o he et ka al. ri et al. Ta ta ze he nc Sa Localization Error (m) Official Results 25 0 22. 99 20 15 11. 24 10 7. 39 5 1. 37 1. 63 1. 9 0. 78 0. 739 0. 73 0. 6 0. 2 0. 31 0. 39 infrastructure-free 2. 6 4. 92 4. 22 4. 31 4. 67 3. 93 3. 65 3. 5 Team industry 8. 24 5. 6
Infrastructure Based Infrastructure Free 1 st $1000 2 nd $600 3 rd $400 STe. AM: Sensor Tracking and Mapping Sanchez et al. (Institute for Transuranium Elements) 0. 2 m error ALPS: An Ultrasonic Localization System Lazik et al. 0. 31 m (CMU) error SPIRIT Navigation Berkovich et al. (SPIRIT Navigation) Time Domain Dewberry et al. (Time Domain) 1. 9 m error 2. 6 m Precise Indoor Location error D. Elias et al. (Fraunhofer Portugal Research Center) 0. 39 m error Exploring State of the Art Commercial Localization Campbell et al. 0. 6 m (Univ. of Michigan) error
Point-Level View Localization Error (m) 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 Sanchez et al. 9 10 11 12 Evaluation Point SPIRIT Navigation 13 14 15 16 17 18 19 20 Elias et al. Localization Error (m) 10 8 6 4 2 0 1 2 3 4 5 6 7 8 Lazik et al. 9 10 11 12 Evaluation Point Time Domain 13 14 Campbell et al. 15 16 17 18 19 20
- Slides: 9