WiFi Conferencing System Locating Array Efficient experiment parameter

Wi-Fi Conferencing System: Locating Array Efficient experiment parameter screening Ingrid Moerman, Eli De Poorter, Michael Mehari (i. Minds) Violet Syrotiuk, Randy Compton, Charles Colbourn (Arizona state university)

Overview • Theoretical background • Applying these concepts in FIRE 2

Screen Experiments • A wireless system is characterized by a large amount of input parameters (factors) each with different configurable levels or settings. • The purpose of a screening experiment is to identify the significant factors and interactions between them that influence the overall performance 3

Screen Experiments • An experiment is given by an N × k array • The k columns correspond to the factors • Each factor Fi, 1 ≤ i ≤ k has a set of levels Li • Each of the N rows corresponds to a test in which each factor Fi is set to a level in Li • Example • 2 factors F 1 & F 2 (2 columns, k=2) • Tx power, 20 levels (0 -> 20 d. Bm) • Transmission rate, 5 levels (6, 9, 12, 24, 36 Mbps) • 5 experiments (5 rows, N=5) 4

Coverage • To measure a t-way interaction it must be covered • For example, if t=2, then all pairs of (factor, level) combinations must occur in the experiment • Covering arrays (CA) are designed to provide coverage • A CA(N; t, k, v) is an N × k array where each N × t subarray contains all ordered t-sets 5

Covering Arrays • An example covering array in which each 6 × 2 subarray contains all ordered (binary) pairs CA(6; 2, 5, 2) 0 1 0 1 1 0 0 1 0 1 0 0 0 1 1 0 6

Location is more than Coverage • However, the ability to locate factors and interactions is more than just coverage • A locating array must uniquely identify the rows in which significant t-way interactions are covered • The number of weak or faulty interactions must be limited (not known a priori) 7

Locating Arrays – How Large? • How does the array size N grow as a function of k for fixed v, t, and d? • Number of factors: k • Number of levels: v • “Size” of interaction: t • Maximum number of faulty interactions d • There are many sets of selection tuples to distinguish When t and d are fixed, and the number of factors k increases, the number of tests grows as a logarithmic function of k 8

Overview • Theoretical background • Applying these concepts in FIRE 9

From theory to experiment • The locating arrays have been used successfully in simulations [1] to identify parameter interactions • Reducing the design points from 1043 to only 421 • But is it also feasible on realistic systems? [1] Abraham N. Aldaco, Charles J. Colbourn, Violet R. Syrotiuk: Locating Arrays: A New Experimental Design for Screening Complex Engineered Systems. ACM SIGOPS Operating Systems Review - Special Issue on Repeatability and Sharing of Experimental Artifacts, 49(1): 31 -40 (2015) 10

Experiment set-up 1 transmitter 42 receivers 11

Factors and Ranges # Factor Description Range 1 band Frequency band (2. 4, 5) GHz 2 channel Frequency channel 1, 6, 11, 36, 40, 44 3 rate Transmission bitrate (6, 9, 12, 24, 36) Mbps 4 txpower Transmission power 20 d. Bm 5 mtu Maximum Transmission unit (256, 512, 1024, 1280, 1500) bytes 6 txqueuelen Transmission Queue length (10, 50, 100, 500, 1000) bytes 7 qdisc Queuing discipline pfifo, bfifo, pfifo_fast 8 ipfrag_low_thresh IP fragmentation lower threshold (0. 25 0. 75 1) 9 ipfrag_high_thresh IP fragmentation higher threshold (16384, 65536, 262144, 1048576, 4194304) bytes 10 udp_rmem_min Minimum UDP receive memory (0. 019231 0. 5) 11 rmem_default Default receive memory (0 0. 25 0. 75 1) 12 rmem_max Maximum receive memory (2304, 10418, 47105, 212992) bytes 13 udp_wmem_min Minimum UDP write memory (0. 019231 0. 5) PHY IP UDP APP 13

Factors and Ranges # Factor Description Range 14 wmem_default Default write memory (0 0. 25 0. 75 1) 15 wmem_max Maximum write memory (4608, 16537, 59349, 212992) bytes 16 udp_mem_min Minimum UDP memory (0. 25 0. 75) 17 udp_mem_pressure Pressure UDP memory (0 0. 33338 0. 5 0. 75 1) 18 udp_mem_max Maximum UDP memory (95, 9490, 94896) bytes 19 ROHC RObust Header Compression OFF 20 codec Audio codec type Opus, speex 21 codec. Bitrate Audio codec bitrate (8 16 24 32) kbps 22 Frame. Length. Aggregation Audio frame length aggregation (20 40 60) msec 23 interference. COR Interference Channel Occupancy Ratio 10%, 50%, 100% 24 sensing Background sensing activitity ON, OFF PHY IP UDP APP 14

Design space Range Total number of experiments (with 24 factors) l = 6*5*5*3*4*5*3*4*4*3*5*4*2*4*3*3*2 l = 5. 59872× 10¹² experiments l = 20 M years (if 2 min. / experiment) l � Locating Array experiments l = 109 experiments l = less then 4 hours � 15

Experiment Outputs � � Transmission Exposure Measures the electromagnetic exposure as felt by a receiver in SAR (Specific Absorption Rate) units � � Mean Opinion Score (MOS) � � Measures the audio quality of the speaker after the encoding and transmission path 16

Experimentation Challenges • Interference generation using USRP does not work consistently • Clients loose connection to Access point (speaker) when Tx. Power is reduced • There used to be a bug inside the Speex decoder • Continuously bringing the Wi-Fi interface ON and OFF sometime makes a node non-responding • Applying the LA factors on the system sometimes fail • Since the experiment takes very long time, watching for a failed experiment is time taking • Singe output criteria required reports from multiple sources (some devices might fail, other might not) • Variable outcomes (time dependent behavior) 17

THANK YOU FOR YOUR TIME

? Question s