A Sociology of the Grid Ian Foster Carl

A Sociology of the Grid? Ian Foster Carl Kesselman Computation Institute Information Sciences Institute, University of Southern California Argonne National Lab & University of Chicago

Why We Are Here “With the establishment of large scale multidisciplinary production Grid infrastructures such as the EGEE, OSG, DEISA, Tera. Grid, or NAREGI, the concept of Virtual Organizations (VO) has been constantly refined and efficient management of VOs and their policies is becoming one of the central topics for these infrastructures. ” 3

“The Anatomy of the Grid, ” 2001 The … problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource -brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization (VO). 4

Examples “The application service providers, storage service providers, cycle providers, and consultants engaged by a car manufacturer to perform scenario evaluation during planning for a new factory” “Members of an industrial consortium bidding on a new aircraft” “A crisis management team and the databases and simulation systems that they use to plan a response to an emergency situation” “Members of a large, international, multiyear high-energy physics collaboration” 5

From the Organizational Behavior and Management Community “[A] group of people who interact through interdependent tasks guided by common purpose [that] works across space, time, and organizational boundaries with links strengthened by webs of communication technologies” — Lipnack & Stamps, 1997 l Yes—but adding cyber-infrastructure: u People computational agents & services u Communication technologies IT infrastructure Collaboration based on rich data & computing capabilities 6

NSF Workshops on Building Effective Virtual Organizations [Search “BEVO 2008”] 7

Two Perspectives l Technology used to enhance collaboration (Computer Supported Collaborative Work) l Collaboration used to enhance technology 8

What is an Organization? l A organization has an identity and a purpose, which it seeks to fulfill within its environment l The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services l The organization’s performance can be evaluated with respect to various metrics Is a virtual organization any different? 9

Identity Legal aspects. Credentials. Purpose Anything legal … Environment Available service & resource providers. Legal &organizational constraints Participants Identity-based or attribute-based. People, services, resources, sensors. Structure Centralized, decentralized, … Activities Business processes. Workflows. Deliverables Data products. Services. Instrument operations. … Performance Throughput, responsiveness, growth, happiness, security, … 10

“Where am I going to eat tonight? ” It looks like you’re creating a VO “I can’t solve this problem alone—I need to involve my buddies Erwin and Miron” Get help with creating the VO Just create the VO without help From: Ian To: Erwin, Miron Subject: Help me find a restaurant 11

Identity Boston-Diner-VO. Informal, so no legal status. Purpose Find a good restaurant in Boston. Avoid eating with loud sports fans. Environment News feeds. Satellite data feeds. Analytic services. EGEE, OSG, Tera. Grid. Participants Erwin, Ian, Miron. Anyone else we trust to make good recommendations. Structure Central database and associated services (membership, …) Activities Restaurant identification workflow. Sport fan tracking workflow. Deliverables Restaurant recommendation for today. Recommendation service. Business plan? Performance Food quality. Monitor Miron’s contributions. 12

Building a Virtual Organization l Determine policy u l l Determine membership and roles u Terms of engagement u Virtualization & integration of providers Create VO-wide services u l Global behaviors Manage work u l Negotiation, trust management Collaborative problem solving, workflow management Manage the VO u Monitor performance, report metrics 13

Defining Community: Membership and Laws l Identify VO participants and roles u l And map participants to attributes and roles Specify and control actions of members u Empower members delegation u Enforce restrictions federate policy B A 1 1 10 A 1 2 10 1 B 16 1 2 14

Security Services Objectives l l It’s all about “policy” u Define the VO’s operating rules u Security services facilitate the enforcement Policy facilitates “business objectives” u l Related to goals/purpose of the VO Security policy often delicate balance u Legislation may mandate minimum security u More security Higher costs u Less security Higher exposure to loss u Risk versus rewards 15

Policy Challenges in VOs l Restrict VO operations based on characteristics of requestor u l l VO dynamics create challenges Intra-VO u VO-specific roles u Mechanisms to specify/enforce policy at VO level Inter-VO u Entities/roles in one VO not necessarily defined in another VO 16

Core Security Mechanisms l Attribute Assertions u l Authentication and digital signature u l C asserts that S can perform O on behalf of C. Attribute mapping u l Allows signer to assert attributes Delegation u l C asserts that S has attribute A with value V {A 1, A 2… An}vo 1 {A’ 1, A’ 2… A’m}vo 2 Policy u Entity with attributes A asserted by C may perform operation O on resource R 17

Trust in VOs l Do I “believe” an attribute assertion u u Used to evaluate cost vs. benefit of performing an operation E. g. , perform untrusted operation with extra auditing l Look at attributes of assertion signer l Rooting trust u Externally recognized source, e. g. , CA u Dynamically via VO structure delegation u Dynamically via alternative sources, e. g. , reputation 18

Building Blocks l Attribute Authority (ATA): u Issue signed attribute assertions (including identity, delegation, & mapping) ATA User A is an admin User B is a member User B can use service X l Authorization Authority (AZA) u Makes decisions based on assertions & policy 19

VO Policy at a Service ATA: Attribute Authority AZA: Authorization Authority VO ATA Resource AZA Resource ATA WS-Subject WS Resource GT 4 authorization and delegation services provide first implementations 20

Establishing VO-Wide Policy ATA: Attribute Authority AZA: Authorization Authority Subject ATA Subject AZA WS-Subject VO AZA VO ATA Resource AZA Resource ATA WS Resource GT 4 authorization and delegation services provide first implementations 21

Attribute Mapping VO 1 VO Resource Admin User A Attribute VO ATA Delegation Assertion User B can use Service X Mapping ATA VO Me mber Attribu te VO Member Attribute VO User B VO AZA Service X VO-1 Attr VO-2 Attr VO 2 Service 22

Protected Health Information Problem l l What do we want? u Use clinical data for research u Share clinical data, make research data available u Reuse same infrastructure u Image exchange between health providers Patient authorizes use of data – consent process u u u Intact unmodified DICOM workflow for diagnostics De-identified DICOM workflow for research (Modality profiles) Group authorization problem: Patient data–touser (Physician/Researcher) relationship not manageable! 23

24

Patient Authorized Grid Image Workflow MEDICUS (Erberich et al. ) 25

HIPAA Compliant Research Access MEDICUS (Erberich et al. ) 26

PHI safe workflow with PCA 27

VO as a Service (VOaa. S) Function Resource l l Virtual organizations integrate participants and resource providers u Participants are selected or self assemble u Select “best of breed” providers for VO services Much of this process can be automated u Provisioning of enabling services, at least 28

VOs Assemble Services l Integrate services from various sources u l Virtualize external services as VO services Deploy new services for the VO Content 3 2 Services Provider 5 Services Capacity Community 4 1 Capacity Provider 29

VOs Assemble Services Domain-dependent Expt Content design Services Tele-op monitor Expt output Domain-independent Simulation code Electronic notebook Portal server Simulation code Simulation server Metadata catalog Certificate authority Data archive Capacity Experimental apparatus Servers, storage, networks 30

Providing VO Services l Integrate existing services l Delegate and deploy capabilities/services l u Provision service to deliver defined capability u Configure execution environment u Host higher-level functions u GRAM, Workspace Service, EC 2, … Coordinate and compose u Build new functions from individual services 31

Virtualization and VOs: Its Turtles all the Way Down Application Interface Mgmt Interface WS-Agreement Mgr Mgmt Interface WS-Agreement Application Embedded VO management Workflow DAGMan Capability provisioned for VO GRAM Managed Service Glidein Cluster Provisioning, management and monitoring at all levels 32

Service Authoring and Deployment l l l grid Remote Application Virtualization Infrastructure Builds on Introduce u Define service u Create skeleton u Discover types u Add operations u Configure security Wrap arbitrary executables Create Appln Service Introduce Index service Ravi Madduri Store Advertize Discover Repository Service Transfer GAR Invoke; Container Deploy get results g. RAVI: Ravi Madduri et al. , Argonne/U. Chicago & OSU 33

Registries (E. g. , ca. BIG) 34

Workspace Service (For When You Want a Virtual Machine) request VM EPR VM Factory create new VM image Client use existing VM image inspect and manage Create VM image VM Repository deploy & suspend VM Manager Resource VM Kate Keahey start program Tim Freeman 35

Composing Services 36

Service Composition: Data Replication Service Pull “missing” files to a storage system Data Location Data Movement Reliable File Transfer Service Data Replication List of required Files Data Replication Service Grid. FTP Local Replica Catalog Replica Location Index Ann Chervenak “Design and Implementation of a Data Replication Service Based on the 37 Lightweight Data Replicator System, ” Chervenak et al. , 2005

Decomposition Enables Separation of Concerns & Roles S 1 User D S 3 “Provide access to data D at S 1, S 2, S 3 with performance P” Service Provider “Provide storage with performance P 1, network with P 2, …” Resource Provider S 2 S 1 D S 2 S 3 Replica catalog, User-level multicast, … S 1 D S 2 S 3 38

Policy, Revisited l Traditionally policy is enforced at end points, integrated with application u l E. g. , PDP call-out in GT container We can also apply policy at the VO level u u Define interactions between services at the organizational level Factor policy out of service implementations 39

Policy-Driven Service Oriented Architecture l Need stand-alone policy engine to coordinate at VO level l Connection between application policy and infrastructure policy (dynamic provisioning) l Policy extension points designed into services allow u u Coordination at VO level Dynamic policy enforcement across services and service oriented infrastructure Web Services 2. 0: Policy-driven Service Oriented Architectures Thomas B Winans and John Seely Brown 40

A Traditional View of the “Grid Problem” Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations Too limited a view 41

We Need an End-to-End Perspective l A organization has an identity and a purpose, which it seeks to fulfill within its environment l The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services l The organization’s performance can be evaluated with respect to various metrics Then focus on clear identification of roles, separation of concerns, isolation of policy 42