July 2000 doc IEEE 802 15 00223 r

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Project: IEEE P 802. 15 Working Group for Wireless Personal Area Networks (WPANs) Submission Title: [TG 3 Common Simulation Framework Presentation] Date Submitted: [July 9. 2000] Source: [P. Taylor Salman, Manager, Market Research] Company [OPNET Technologies, Inc] Address [3400 International Drive, Washington, DC 2000] Voice: [202 -364 -4700 x 2297], FAX: [Add FAX number], E-Mail: [tsalman@mil 3. com] Re: [. ] Abstract: [The topic is a common simulation framework (CSF). A CSF allows data to be collected on competing standards proposals with identical usage scenarios, traffic loads and types, interfering sources, and methods of statistical collection. The goal is to be able to compare proposals, apples to apples, and to arrive at an objective conclusion on the best technical implementation. While other issues still play into the standards development process, CSF implementation can offer an approach to this task. ] Purpose: [The purpose of this presentation is to describe the benefits of implementing a CSF, how to best implement a CSF, what issues can be addressed by modeling and simulation in the standards process, and a review of past and present efforts within IEEE 802 using modeling and simulation. . ] Notice: This document has been prepared to assist the IEEE P 802. 15. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Release: The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by P 802. 15. Submission 1 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Using Modeling and Simulation to Support Communications Standard Development P. Taylor Salman, Manager, Market Research Submission 2 Taylor Salman, Opnet Technologies

July 2000 Introduction doc. : IEEE 802. 15 -00/223 r 0 Outline Modeling & Simulation? Process, Pitfalls, & Lessons Learned Common Applications of M&S in the Standard Development Process How To… Common Simulation Framework Coexistence Studies Standards Enhancement Summary Submission 3 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Introduction One of the biggest challenges in developing successful standards is determining the accuracy and performance of competing proposals. Subjective rating systems and unsubstantiated claims can inhibit progress and sometimes lead to stalemate within a working group. Modeling and simulation of proposals creates the opportunity for objective comparison and clear validation of proposals before final decisions are made. Submission 4 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Introduction How does M&S improve the standards development process? What is the best process for using M&S? What pitfalls should be avoided? When should modeling and simulation be used? How should M&S be applied? Submission 5 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Modeling and Simulation? Networking technology has become too complex for traditional analytical methods to yield accurate understanding of system behavior Submission 6 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Modeling and Simulation? Divides complex issues into understandable and resolvable components Addresses multiple layers of performance • Architecture • Application • Transport&Network • Access • Physical Submission 7 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Modeling and Simulation? Provides hard data from a variety of perspectives to evaluate performance Graphic depiction of performance eases communication of concepts and proofs Submission 8 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 The Process Scope the Problem Design Scenarios Divide & Conquer Build Models Format the Answer Run Simulations Compare Results Submission 9 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 The Process • Scope the Problem – What questions should be answered to evaluate the proposals? – 5 Criteria – Translate into data measures (MOEs) • Divide & Conquer – Split the questions into smaller problems – Group like issues – Isolate desired behavior – Abstract unrelated behavior • Format the Answer – What data will support the MOEs? – Where should the data be collected? – What method should be used to collect the data? – How should the data be presented? Submission 10 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 The Process • Design Scenarios – Determine the most likely conditions for use – Determine extreme environments – Design multiple scenarios to address both • Type of approach – Analytic vs. discrete-event, combination – Bit level vs. packet level vs. flow level performance • • • Topology Applications/traffic sources Higher-layer protocol interfaces Simulation run-time parameters Statistical distributions – Limit the scope of the scenario to address desired behavior – Determine how scenarios can “feed” each other Submission 11 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 The Process • Build Models – Data, Data… – Document! – Share and vet… Performance is just one of many factors contributing to a formal standard; • Run Simulations – New Issues? ? ? – Improve the process… M&S helps us understand the implications of our performance tradeoffs… • Compare Results – Data will NOT provide the answer… – Analysis is needed, arguments necessary… – Proof is available, arguments are objective and verifiable Submission 12 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Pitfalls & Lessons Learned • Start Early! • Get Commitments • Divide Responsibility • Communicate – Share models • Get agreement before proceeding • Re-evaluate Submission 13 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Applications of M&S in the Standards Process • Common Simulation Framework – Test different proposals for a common objective – Uses commons scenarios, traffic sources, higher layer protocols, data collection methods, and run time parameters – Enables “Apples to Apples” comparison • Co-existence Studies – Evaluate effect of new standards on existing systems – Physical and higher layer performance – Create complex environments without prototyping • Protocol Enhancement – Improve or “add on” to an existing protocol – Add features without affecting current performance – Test integration in a virtual environment Submission 14 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Examples of M&S Applications in the Standards Process • DOCSIS CSF (Cable. Labs) • Bluetooth Co-existence • Wireless LAN Enhancement Submission 15 Taylor Salman, Opnet Technologies

July 2000 doc. : IEEE 802. 15 -00/223 r 0 Summary • Modeling & Simulation provides verifiability and objectivity to the standards development process • Commitment, a plan, and adherence to a process are necessary to be successful • Don’t “bite off more than you can chew” and don’t succumb to the “ever-expanding scope”! • Additional Benefits: – The final models provide a common understanding of the standard – Models can be leveraged: – Promotion of the technology – Architecture planning – 2 nd generation technology baseline Submission 16 Taylor Salman, Opnet Technologies

July 2000 Submission doc. : IEEE 802. 15 -00/223 r 0 17 Taylor Salman, Opnet Technologies