Design of a Scalable Clearing House Architecture Lakshminarayanan

Design of a Scalable Clearing House Architecture Lakshminarayanan Subramanian Chen-Nee Chuah Ramakrishna Gummadi ICEBERG Design Review Jan 12, 2000

Basic Questions in Mind!!! v What is a Clearing House? v What are the basic requirements of the Clearing House? v What are the services it supports? v What are the goals of our design? v What are the assumptions we make in our design?

Clearing House v Coordinates interactions between the various ISPs in the network. v What kind of interactions? ¨ Performs path discovery and resource reservation. ¨ Services wide-area call requests. ¨ Provide Qo. S guarantees. ¨ Secure billing services. v Support for multicast and mobility.

Present Scenario ISP 3 H 3 ISP 1 ISP 2 H 1 ISP 4 H 2

Goals of our design Scalability- throughput, state maintained v Optimize network utilization v Dynamic call-routing v Continuous path monitoring for Qo. S v Reduce response time for call requests v Support multicast, mobility and secure billing v Recovery from link, node and packet failures v Security and Privacy v

How do we achieve it!!! v Build a logical hierarchy in the network v Distribute state and resources among the nodes in the hierarchy and create a distributed database v Aggregate requests and bound queue size v Hierarchical and dynamic routing of call requests v Continuous monitoring of resources v Separate resource reservation and call-setup

Assumptions v Edge routers can collect traffic statistics and estimate bandwidth requirements v Control and data paths are separated v Clearing House and ISP trust each other v Routers can measure queueing delay statistics v Possible to introduce a monitoring system into existing ISP architecture

Clearing House Structure ISP 3 ICH ECH ISP 1 ICH ISP 2 ICH ISP 4

Clearing House Infrastructure v External Clearing House(ECH) as third party agent to coordinate inter-ISP traffic. v Internal Clearing House(ICH) services intra. ISP traffic and acts as a monitoring agent for external traffic. v ECH organized as a hierarchy of nodes. v ECH stores inter-ISP network state and ICH stores intra-ISP network state in a distributed manner.

Hierarchical Structure v Divide network into non-intersecting basic domains(e. g. . Cluster area codes) v Recursively join physically adjacent domains to form larger logical domains. v Generate a hierarchical tree of domains in the network. v Associate a distributed ECH with every domain in the tree.

Hierarchical Clearing House ISP 3 ICH ECH ISP 1 Domain ECH ICH ISP 2 ISP 4

External Clearing House Performs hierarchical routing and computes near optimal path for call requests. v Aggregates call requests. v Collects statistics on resource reservations and delay statistics from ISPs. v Performs extra resource reservations for call requests if necessary. v Monitors performance of traffic. v Stores billing prices of ISPs within its domain v

Internal Clearing House v Every ISP has an ICH. v Routes intra-ISP calls. v Monitors and predicts incoming and outgoing traffic in edge routers v Performs advanced reservations for predicted traffic and updates ECH. v Determines link reservations in ISP and updates traffic routing table of routers.

Hierarchical Routing Inter-domain and Intra-domain paths Domain 1 1 a 1 b 1 c

Clearing house state v Billing information is present in CH of basic domain. v Each CH maintains aggregated state of its domain. ¨ Calls between two sub-domains of its domain. ¨ Aggregated connectivity graph between domains. ¨ Reservation and delay status along links and nodes in the graph. ¨ Pricing information between domains.

Other Enhancements v Caching ¨ v Cache computed inter-domain paths Rx. W scheduling ¨ Maximize throughput without affecting response time. Measuring Qo. S parameters v Multicast support v Dynamic path routing to support mobility v Secure billing architecture v Fault tolerance v

Support for Multicast and Broadcast Trees Edge router • Nodes up in the hierarchy find inter-domain multicast tree. • Local nodes find intra-domain optimal tree.

Scalable Infrastructure for supporting Mobility Moving Object L 1 Level 0 Domain Structure

Strengths State of network distributed among various CH nodes. v Aggregation of call requests. v Response time depends on locality. v Bounded queue size. v Path discovery is distributed. v Localized billing – makes it real-time. v Core routers do not maintain much state info. v Caching, scheduling improve performance. v

Clearing House Design: Resource Reservation Strategies Chen-Nee Chuah ICEBERG Design Review Jan 12, 2000

Example Scenario H 1 H 3 ISP 2 ISP 1 H 2 Resource Reservation v Quality of Service? ISP 3

Example Scenario H 1 H 3 ISP 1 SLA ISP 2 SLA ISP 3 H 2 SLA: Agreements that describe the volume of traffic exchanged, bandwidth reserved and price

Challenges How is the SLA between two ISPs established? v How do SLAs reflect dynamic traffic patterns? v What happens when it involves more than 2 ISPs? v => Clearing House provides a scalable approach to address these questions

Hierarchical Clearing House destination source Edge Router ISP n ISP 2 ICH ICH ISP 1 Updates CH 1 Adapt Reservation CH 1 CH 2 v Distributed database & bandwidth brokering agent • reservation status, % link utilization, traffic predictor • establish advanced reservation (based on traffic predictor)

Resource Reservation Infrastructure H 1 Edge Router H 2 Assume the Edge Router v collects traffic statistics ¨ ICH ISP 1 v estimates dynamic change of bandwidth requirements ¨ Edge Router v v ICH ISP 2 e. g. average aggregate incoming and outgoing traffic volume statistical techniques (Kalman filter) sends regular updates to LCH aggregates reservation requests

Static & Dynamic Reservations H 1 Edge Router ISP 1 H 2 ICH Edge Router CH Static Reservation Dynamic Reservation Internal Clearing House v Maintain intra-ISP reservation status v Establish static reservations based on mean aggregate traffic for different time of the day. v Adapt reservations on a smaller time-scale based on existing reservation and bandwidth predictor. v Send regular update to GCH

Properties v Aggregation of Signaling ¨ v Resource reservation requests are aggregated at various levels (ER -> ICH, ICH-> CH 1 etc. ) De-couple notifications & reservation requests ¨ ¨ notifications: updates on reservation status, % link utilization, traffic predictor reservation requests: initiation of SLA renegotiations Hierarchical Approach v Static and Dynamic Reservations v ¨ ¨ reduce reservation setup time compensate for the coarse granularity of the notifications

Clearing House Hierarchical Tree Adapt Reservations Notifications (every Du s) - Reservation status - Link utilization - Bandwidth predictor - Advance reservations - Process reservation requests CH 1 ICH LCH CH 2 CH 1 ICH ICH LCH Aggregate reservation requests (Ta)

Int-Serv Approach destination source BB ISP n BB ISP 1 v BB ISP 2 End-to-end notifications & reservation requests ¨ ¨ ISP 2 notifies next-hop ISPs and negotiate new SLA with them. When all downstream ISPs accept the SLAs, an ISP notifies upstream ISPs and set up new SLAs. When original SLA is accepted, all SLAs from source to sink are updated.

Diff-Serv Approach destination source Edge Router ISP n ISP 1 ISP 2 BB v BB BB Limited or no notifications ¨ Trade-off end-to-end Qo. S for scalability BB

Evaluation v Overall Performance Metrics ¨ Trade-offs between scalability, Qo. S and signaling complexity • Effect of aggregation on Qo. S – e. g. % blocking/dropping ¨ • Choice of signaling between CHs Link efficiency v Bandwidth Estimator ¨ How well do the predictors track the traffic fluctuation? • Window of measurement?

Clearing House Design: Billing, Security and Privacy Ramakrishna Gummadi ICEBERG Design Review Jan 12, 2000

Basic Goals v Support Scalable, Secure and Correct Billing v Support Trust Management for Traffic Monitoring v Support Privacy Management

Tasks while supporting Secure and Scalable Billing v Must scale to millions of calls per day v Must perform authentication (in both directions), authorization, and correct billing

Approaches to support Secure and Scalable Billing Use a level of indirection through authorization and billing tickets v Generate these tickets offline v Perform offline settlements with the user and various ISPs v Use aggregation for storing and verifying tickets to reduce storage space v Use X. 509 certificates, passwords or Public-key challenge/response for mutual authentication v

Notes on Authorization and Billing Tickets v Both used as level of indirection, for achieving scalability, while maintaining high security and requiring little trust v Both like Kerberos, a scalable security service, using tickets for authentication and secrecy v Both acquired by the user once at the beginning, and used as needed

Notes on Authorization and Billing Tickets (contd. . ) v Authorization tickets used to establish that call corresponds to resources reserved v Billing tickets used to charge the user for time spent on the call v Billing tickets can be returned by the user at end of call, or more can be acquired during duration of call, as needed, to maintain correct billing records

Performance Optimizations Can use shared-key techniques in using authorization tickets v A lot depends on the degree of trust between the CH and the ISPs (though the ISPs themselves don’t need to trust each other) v If trust possible, we can use shared-key cryptography for billing (no non-repudiation support) v Lots of performance and storage improvement through aggregation v

Trust Management v CH can incorporate a Trust Management module to: ¨ ¨ Provide a standard, general-purpose mechanism for specifying application security and credentials Directly authorize security-critical actions, like network monitoring Bind keys directly to authorization records to perform specific tasks Describe delegation of trust and subsume the role of public key certificates

Privacy Management v Privacy management very difficult in the current Internet, more so in ICEBERG (because of billing) v Privacy of users and participating ISPs needed v User privacy with respect to participating ISPs achieved by anonymization in the form of indirect authorization and billing tickets