SAML Conformance SubGroup Report Bob Griffin Facetoface meeting

SAML Conformance Sub-Group Report Bob Griffin Face-to-face meeting August 29, 2001

Topics • • • Status Binding. Conformance issue Browser partition issue Testing optional elements Building test clients Interoperability testing

Status • Conformance Clause – Version 0. 006 in SAML spec; no change from June F 2 F – Describes goals, validation/certification, partitions • Conformance Program – Version 0. 005 available – Includes cut at test cases for Authentication Authority Partition, including traceability to requirements; other test cases tdb • Certification – Decision at June F 2 F to focus on validation for SAML V 1 • Test suite – Detailed design to be included in test cases following F 2 F • Traceability of SAML to S 2 ML, Auth. XML… – Prateek to do SAML/S 2 ML traceability – Darren to do SAML/Auth. XML traceability – Dave O. to do SAML/ITML traceability

Issue: Binding. Conformance • “Should protocol bindings be the subject of conformance? ” • Concerns raised in issue write-up: – Profile is “a complete specification of the SAML aspects of some use case” while binding is “a building block in one or more profiles”. – Specifying conformance in terms of protocol/bindings increases the number of test cases – Conformance claims based on protocol/binding would not necessarily mean interoperability. • Discussion – Current model of “conformance partitions” does not define conformance in terms of use cases, but in terms of system entities in the static domain model: (authentication authority, attribute authority, policy decision point, policy enforcement point) – Conformance claims by SAML implementation or application would be for one or more partitions; – Within partition, subsets further allowed – For each system entity, test cases related to profiles and to protocols/bindings are defined

Authentication Authority Partition Test Cases HTTP Web Server Profile(s) – – Valid Authentication Assertion Produced Valid Authentication Assertion Consumed (eg, from another domain)? Valid Authentication Assertion Artifact Produced Valid Authentication Assertion Artifact Consumed Authentication Request/Response Protocol: HTTP, SOAP bindings – Valid Authentication Assertion Produced – Valid Authentication Assertion Artifact Consumed

Authentication Authority Partition Tests Test case 1 -4: Web Server Profile Request for authentication assertion embedded in HTTP SAML Conformance Test Client Web Server Authentication assertion returned, embedded in HTTP, and checked by test client for validity Test case 5 -16: Protocol/Binding Request for authentication assertion, bound to HTTP Authentication assertion returned via HTTP and checked by test client for validity SAML implementation or application

Binding. Conformance (cont’d) • Discussion: – Should conformance be tested in terms of use cases rather than entities in domain and producer-consumer models? – Are profiles equivalent to use cases and therefore the way conformance should be specified? – Does testing protocols/bindings test interoperability? – Are protocols specific enough so that conformance can be defined in terms of protocols rather than protocol + binding? • Recommendation: – For now, continue with “conformance partitions”. – Include behavior related to both profiles and protocol/bindings in conformance partition testing. . – Implementation or application claiming conformance to a given partition must specify which bindings it supports for the protocol(s) in that partition – Not all bindings have to be supported to claim conformance to that partition.

Issue: Browser Partition • Should there be a conformance partition for browser, comparable to those for authentication authority, attribute authority, PEP, PDP? • Discussion: – The other partitions represent implementations (eg. Multi-domain single-signon product) or applications (eg, application server as PEP accepting assertions) of SAML. – Insofar as unmodified browser is goal, no comparable implementation/application related to browser? – Will all SAML-specific behavior related to profiles be captured by the other partitions? • Recommendation: – Do not define browser partition. – Check that all SAML behavior is captured by other partitions

Optional Elements • Issue: Test cases have to include sub-cases based on optional elements, encryption/signing, etc. • Discussion – Test suite should validate that SAML implementation or application has correctly specified an extension, but should not test the extension itself – No expectation of having registry for SAML extensions • Recommendation: following F 2 F, develop sub-cases for variants and optional elements to explore this issue further.

Saml Conformance Test Client Design • Test client has to evaluate responses received from SAML implementation or application to determine validity of assertions and (in case of protocols) response messages • Validity can be determined in several ways: – Inspection of returned assertion and/or message – Submission of assertion and/or message to open-source reference implementation – Follow-on action based on assertion or message (eg, test succesful consuming of authentication assertion by requesting protecting resource)

Test Client Design - 2 Test case 1: Web Server Profile Request for authentication assertion embedded in HTTP header SAML Conformance Test Client Web Server Authentication assertion returned, embedded in HTTP header, and checked by test client for validity Alternative 3. Resource request submitted to SAML implementation, using authentication assertion SAML Reference Implementation Alternative 1. Received authentication assertion submitted to reference implementation in resource request Reference Assertion Alternative 2. Received authentication assertion compared to reference assertion SAML implementation or application

Test Client Design – 3 • Recommendation: investigate options in terms of detailed design of test cases, to determine how feasible each approach is across all test cases

Interoperability Testing • Several options: – Test implementations/applications against open-source reference implementation (eg, submit assertions to reference implementation PEP) – Cross-vendor testing in customer situations – Cross-vendor testing in independent lab, etc – Cross-vendor testing at vendor sites