Authentication services for big WiFi Eriks Rugelis Krzysztof

Authentication services for big Wi-Fi Eriks Rugelis, Krzysztof Adamski Network Development team York University, UIT

York. U context • Population – 47 K undergrad students, 6 K grad students, 7 K faculty and staff • Physical infrastructure – 2 primary campuses, 6 satellite locations – Total of 70 academic + admin buildings, approx. 600 K sq. m. – Plus… total of 16 residence buildings, 3400 beds

WLAN infrastructure at York. U • Academic and administration buildings – Cisco AP 1121, AP 3602 i, AP 3702 e, AP 3702 p • Residence buildings – Cisco AP 1252, AP 1142, AP 702 W, AP 3602 i • Total of 14 Cisco CT 5508 WLCs – One running Aire. OS 7. 0. 252. 0 for legacy APs – Balance running Aire. OS 8. 0. 120. 13

WLAN Services at York. U • Air. York. PLUS – The primary SSID for WLAN service to York. U users – 802. 1 x authentication, PEAP/MSCHAPv 2 – 27 K peak concurrent client devices in 2015 -16 • eduroam – Available to both guests and local users – 802. 1 x authentication, PEAP/MSCHAPv 2 • Air. York – Captive portal service – Hidden SSID, used for guest professional development users (difficult onboarding SSID with 802. 1 x credentials) • Wi-Fi Info – Open SSID used to provide online help information to Wi-Fi users

Identity management infrastructure • Passport York – Home grown identity management system – Is the administrative source-of-truth for identity for all centrally-supported IT services at York. U and many Faculty/Department supported services as well – Accessed via own API or via LDAP – Stores unique identifier code, userid, password, access rights attributes (PYflags, PYgroups)

Authentication/Authorization backend infrastructure Active Directory – YORKU domain – Root of enterprise identity tree – Used by WLAN services as backend authentication and authorization DB – Synchronized identity and passwords with PY – Receives group ID feed from PY – 5 servers (Sun. Fire 4170), 16 cores, 32 GB RAM

RADIUS server • York. U selected Packet. Fence • What is it? – Packaged Free. RADIUS + accounting database + NAC – FOSS project developed and maintained by Inverse. ca – Active community forum – Paid telephone support and custom SW development available from Inverse

Packetfence Components • PF front-end – Responds to RADIUS transactions from network • authorization / authentication / accounting – – Uses winbind to call Active Directory for authentication Uses LDAP to call Active Directory LDAP for authorization May be used as captive portal server May be used to probe and evaluate security posture of Wi. Fi client and apply NAC policy • PF database – Stores user credentials (user name, person name, MAC address of device, session login time, data volume accounting, history of policy violations)

The challenge • Poor performance of 802. 1 x authentication service for WLAN service has high impact on enduser perception of overall WLAN service – bad for student satisfaction: – Plan for performance – Plan for scale appropriate to your environment – Monitor performance and capacity metrics • 802. 1 x authentication transaction rates in higher education have high peak-to-average ratio

Total RADIUS requests/sec (x 0. 5)

York. U scale-out configuration • Two RADIUS load-balancer clusters (pfclust 10, pfclust 20), one cluster per VM host • 6 x PF guests per cluster – 8 v. Cores, 16 GB RAM • 1 st node acts as RADIUS LB only, nodes 2 thru 6 process the transactions • Odd numbered WLC’s prefer pfclust 10 first • Even numbered WLC’s prefer pfclust 20 first

York. U scale-out config - continued • If pfclust 11 fails then RADIUS LB function passes to next sequential pfclust 1 x guest; LB function returns to lowest numbered pfclust guest whenever it is ready to resume service • Any single pfclust* guest has capacity to handle up to 300 to 500 transactions per second • Pfdb 11 is a single VM guest; setup for autorestart by VMware HA • We have not yet observed a transaction rate limit on pfdb 11 – but pay attention to DB connected thread count!!

pfdb 11: thread stats

Other PF deployments tried • Active-Standby clusters (PF/RADIUS, DB) • Pair of Active-Active PF/RADIUS servers + Active-Standby DB cluster • Long list of Active PF/RADIUS servers + single pfdb 11 with VMware HA

Active-Standby clusters Pro: • Resilient/fault-tolerant authentication service Con: • Under-utilized idle capacity in Standby servers • DB cluster synchronization uses a unique and unfamiliar (for York. U) clustering technology • Difficult to upgrade SW without impacting entire authentication service • Limited to <500 RADIUS transactions/second

Active pair of PF/RADIUS Pro: • Steady-state capacity of 2 x <500 RADIUS transactions/sec. Con: • Fail-mode capacity is still only <500 transactions/sec (same as previous model) • DB synch technology is still unfamiliar to York. U

Active x 6 PF/RADIUS servers Pro: • Steady-state capacity 6 x <500 RADIUS transactions/sec • DB resiliency converted to familiar VMware HA mechanism Con: • Cisco WLC will send workload of a failed RADIUS server to the next sequential server – if total workload >500 transactions/sec then that server will also fail – congestive collapse is likely during peak loads!!

Be mindful of… • Peak projected RADIUS transaction rate per WLC (or equivalent) • Does WLC itself have sufficient CPU capacity to handle max. projected workload of RADIUS transactions? • How does WLC (e. g. Cisco, Aruba, HP, Meru, etc. ) distribute load when configured to use more than one RADIUS server? • How does WLC respond when preferred RADIUS server goes offline? Returns online? • Transaction rate capacity of back-end authentication DB (e. g. Active Directory) • Measuring/reporting transaction rates and latencies of all authentication system components

Future refinements • Move RADIUS LB function to dedicated (Layer 3 – Layer 7) LB appliance • Introduce LB appliance between pfclust* and ADLDAP to mitigate service outages of individual AD-DC’s (e. g. during monthly patching)