Noorisyam Hamid Fazilah Haron and Chan Huah Yong

今回の紹介論文 Noorisyam Hamid, Fazilah Haron and Chan Huah Yong, “Resource Discovery using Page. Rank Technique in Grid Environment”, In Proceedings of Cluster Computing and Grid 2006 (CCGrid 2006), 2006 l Tania Gomes Ramos and Alba Cristina Magalhaes Alves de Melo, “An Extensible Resource Discovery Mechanism for Grid Computing Environments”, In Proceedings of Cluster Computing and Grid 2006 (CCGrid 2006), 2006 l 9/19/2021 2

Resource Discovery using Page. Rank Technique in Grid Environment Noorisyam Hamid, Fazilah Haron and Chan Huah Yong 9/19/2021 3

背景 Resource selection in large-scale environment has been a challenging task especially in meeting user’s demand for a quality of service (Qo. S) l Most grid resource discovery or schedulers focus on selecting and allocating jobs to suitable resources by mapping the requirements and constraints between users and resources. None considers the background and quality of the users and resources involved l 9/19/2021 4

Condor Class. Ad A matching technique currently implemented in Condor scheduler l A semi-structured data model or language that can be used to specify the characteristics, constraint and preference of principles l Ignores the quality of the principals 9/19/2021 5

Page. Rank Re-ranking method of the searched results from search engine l “Importance” ranking of every web pages base on the importance of its parent pages l A successful link analysis behind l 2 1 3 9/19/2021 6

Quality-based Grid Resource Discovery (Q-GRe. D) l. A resource discovery technique, based on Condor Class. Ad but incorporate the idea of Page. Rank (Resource. Rank) in determining the quality and reliability of the grid resources 9/19/2021 7

Resource Rank Algorithm l Resource can obtain higher Resource. Rank score, if many users from different organizations submit jobs to that resource or there exist users with high Resource. Rank using the resource RR (A) = (1 -d) + d (RR (Ti)/N (Ti) + … + RR (Tn)/N (Tn) ) RR (A) - Resource. Rank of resource A RR (Ti) - Resource. Rank of resource Ti which uses the resource A N (Ti) - the number of times user in organization Ti submits job to current resource in an organization d - damping factor which usually set to 0. 85 9/19/2021 8

Resource Rank Calculation 9/19/2021 Didn’t match with the algorithm… 9

Matching Technique in Q-GRe. D l Class. Ad relies on the following expression to perform matching and find similarity between requestor and provider ¡ Constraint = attribute type which principal would like to be matched Where, Similarity (a, b) = {True if a=b else false} ¡ l Rank = preferences Incorporates Resource. Rank into the rank equation in Condor Class. Ad. Hence, Resource. Rank becomes a new constraint that must be considered when matchmaking is performed. 9/19/2021 10

Matchmaking Process The discovery and matchmaking is strictly based on Resource. Rank value on each principal l Requestor with high Resource. Rank has a high possibility to claim and to be placed into quality and reliable resource l For the case of requestor, if Resource. Rank = N then the claim will only allowed to provider with Resource. Rank = N and below l For the case of provider, the claim is valid and accepted to any requestors which have the same Resource. Rank value or above l 9/19/2021 11

まとめ l Q-GRe. D Provide a better resource discovery in a challenging large-scale grid environment ¡ Concerns with selecting quality and reliable resources for job allocation ¡ l Future work includes implementing the QGRe. D technique and studying other potential matchmaking algorithms 9/19/2021 12

感想 l There’s some mistakes (in formula and examples) l Not implemented yet, so the effect of this method is still unknown whether is good or not for the resource discovery but the idea is quite interesting 9/19/2021 13

An Extensible Resource Discovery Mechanism for Grid Computing Environments Tania Gomes Ramos Alba Cristina Magalhaes Alves de Melo 9/19/2021 14

背景 (1) Resource Discovery in Grids ¡ A directory service directed to the spontaneous network’s environment ¡ Resources are not only computers, but also software, instruments and data, among others (Heterogeneous) ¡ A huge number of resources, spread over multiple administrative domains that are geographically distributed (Scalability) l The kind of data to be retrieved by resource discovery mechanisms is statically defined l In a highly heterogeneous and dynamic environment such as a grid, statically defined searches are usually inappropriate l 9/19/2021 15

背景 (2) l Globus Toolkit ¡ ¡ l Emerged in 1997 as an open source project and quickly became a de facto standard for grid computing infrastructure Defines and implements a set of protocols, APIs and services used by hundreds of grid applications Open Grid Service Architecture (OGSA) ¡ ¡ 9/19/2021 Introduced by the Global Grid Forum (GGF) to expand standardization Provided a new architecture for grid applications based on Web Services in order to achieve interoperability 16

Resource Discovery Mechanisms 9/19/2021 17

Extensible Resource Discovery Mechanism for Grids l The basic resource information retrieval can be extended to include user-defined specific resource searches l Designed as a GT 3 grid service. 9/19/2021 18

Hierarchical Topology l Grid is divided into different virtual organizations (VOs), each VO is structures following the Master x Slave paradigm 9/19/2021 19

Basic Resource Discovery The master is responsible to update the resource database l The slaves are responsible to actually retrieve information from each machine l 9/19/2021 20

Master Service & Slave Service l Master Service (Search. Master. Service) ¡ l Responsible for looking up the slaves services by reading configuration file and invoking resource information retrieval Slave Service (Search. Resource. Service) ¡ Responsible for retrieving default local information about the slave machine (both static and dynamic data) The definition of both services is made in a WSDD (Web Service Data Definition) 9/19/2021 21

Master Service: Configuration file 9/19/2021 22

Slave Service: Local Information 9/19/2021 23

Extensible Resource Discovery l Defined in a module-basis. ¡ Basic Module ¡ Personalized Module Can be coupled to Basic Module on-the-fly to perform specific searches l Need 2 tasks: deploy a method that retrieves personalized information and describe its characteristic in a configuration file l 9/19/2021 24

Extensible Resource Discovery Process (1) l Detect a new configuration file ¡ Made by the periodic verification of the existence of the configuration file (Personalized. Services. xml) Read configuration file and add this the new module to the personalized ones l Incorporate automatically a new resource search to the basic resource discovery mechanism l 9/19/2021 25

Extensible Resource Discovery Process (2) 9/19/2021 26

Personalized. Services. xml 9/19/2021 27

Master Service Process l Incorporation phase ¡ JARs defined in the configuration file are used to create the new GAR (Grid Archive) file ¡ GAR file and Personalized. Services. xml are sent to all slave machines by the Grid. FTP ¡ Begin the container manipulation phase l Container ¡ Undeploy manipulation phase old service, then deploy new service during this container must be stopped due to GT 3 restriction 9/19/2021 28

Slave Service Process l Retrieval phase ¡ Check whethere is any personalized module defined every time it receives a resource’s information request ¡ If there is any personalized method Read the personalized module’s configuration file to retrieve the class responsible for performing the personalized search l This class is instantiated and its methods are invoked by using the Java’s Reflection mechanism l 9/19/2021 29

Resources information with a new service definition 9/19/2021 30

Implementation & Evaluation l Implemented using ¡ JAVA version 1. 4. 2_04 ¡ Jakarta ant 1. 6. 2 ¡ Globus Toolkit 3. 2. 1 9/19/2021 31

Incorporation/Unavailable Master Times (1) Incorporation time is the time between the detection of the personalized configuration file and the termination of the old resource discovery grid service in the master machine l Unavailable time is measured from the time the resource discovery service is stopped in the master to the time the extended resource discovery service is restarted l 9/19/2021 32

Incorporation/Unavailable Master Times (2) l Incorporation time depend on number of slaves while unavailable time doesn’t 9/19/2021 33

Incorporation/Unavailable Master Times (for 3. 2, 6. 4 MB files lengths) l Incorporation time increases a little when GAR size increases 9/19/2021 34

まとめ Proposed and evaluated an extensible resource discovery mechanism for grid environments l The results presented very reasonable times for the incorporation of a new resource search l From the results, a master should not manage a big number of slave machines, if it is necessary to retrieve information from a high number of machines, the system’s architecture permits an organization of these machines in different sets of master x slave machines l 9/19/2021 35

感想 l Use too much time (nearly 15 mins for incorporation of 7 slave machines) l This method is a kind of a value-added for resource discovery mechanism, so if the trade-off doesn’t cost this much, it is a good idea to implement this method 9/19/2021 36