WMS ISSUES CREAM CE ALICE TF MEETING 240708

WMS ISSUES CREAM CE ALICE TF MEETING (24/07/08)

WMS Issues (I) By construction the WMS has an internal task queue which holds the jobs that cannot be submitted to the CE By default the WMS will not abort a job immediately when the matchmaking fails Instead it retries the job after a while (6 hours by default) up to a maximum time (24 hours by default) wms 103. cern. ch and wms 109. cern. ch (Alice use only) have both the following values: �Expiry. Period = 7200 �Match. Retry. Period = 3500

WMS Issues (II) The jobs stay in status « waiting » in the mean time But they do not appear in the IS The counting of all submitted jobs is therefore no exact Thanks to Stefano that pointed out the issue We have asked at CERN to disable such feature And the rest of the sites? This is a service feature, not a VO feature If the WMS is shared with ATLAS and/or LHCb (which want the feature) we face an issue

CREAM CE tests As mentioned last week we have the whole testing setup placed at FZK-PPS vobox fully configured CREAM CE access with 30 WNs behind �Number of CPUs increased on Wed. to 330 Access to the local WMS ensured We tested the submission following 2 different approaches Submission through the WMS Direct submission to the CREAM CE

CREAM CE: WMS submission Take into account that we submit from a VOBOX This has an effect already faced: Number of delegations in the user proxy: From the UI �subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=pmendez/CN=477458/CN=Patricia Mendez Lorenzo/CN=proxy Entering the VOBOX �subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=pmendez/CN=477458/CN=Patricia Mendez Lorenzo/CN=proxy Delegating it �/CN=proxy/CN=proxy Passing through WMS Passing through the CREAM CE Passing through BLAH into the CE

CREAM CE: WMS submission In total there are 10 delegations The limit is 9 Direct CREAM CE does not have this problem because the WMS delegation is avoided Submission through the WMS to the LCG-CE neither because we do not face the BLAH delegation The WMS submission for the moment cannot be tested A bug has been submitted and M. Litmaath has already proposed a code which will solved this problem which affects to Alice only.

CREAM CE: Direct Submission

CREAM CE: Direct Submission Jul 23 17: 33: 21 info LCG Job. ID is https: //ppscream-fzk. gridka. de: 8443/CREAM 571043603 Jul 23 17: 33: 21 info Submission took 3 sec. Comparable with WMS (voalice 03. cern. ch) vs. 10 sec average for LCG-RB

CREAM CE: Direct Submission Principal changes in LCG. pm right now: Implementation of the commands �CREAM => {SUBMIT_CMD => 'glite-ce-job-submit', STATUS_CMD => 'glite-ce-job-status', KILL_CMD => 'glite-ce-job-cancel'}}; Sub wrapsubmit �CE_RBLIST must be empty in LDAP �$config. Opt. Name = "" if ($ENV{Ali. En_WMS} eq 'CREAM'); �$job. Id =~ m/(https: //[A-Za-z 0 -9. -]*: 8443/[A-Za-z 09_-]{14})/ if ($ENV{Ali. En_WMS} eq 'CREAM');

CREAM CE: direct submission The agent jdl also changes: Outputsandboxbasedesturi= « gsiftp: //localhost» This field is mandatory id the Outputsandbox is specified � It defined the gridftp server where the outputs have to be retrieved at the end of the job Our definition means: leave the OSB into the CREAM CE Via globus-url-copy (low level tool) we can later retrieve the output from the CREAM CE Retrieval procedure This solution is very dirty � We can fulfill the CREAM CE in a no effective way � The 4 LHC experiments asked to eliminate the gridftp server of the VOBOX with the g. Lite 3. 1 version � We need it back: In the vobox or outside we need a gridftp server at all sites providing CREAM CE � Already asked to the FZK CREAM experts for Alice use � PROVIDED TODAY IN THE MORNING

CREAM CE: Direct submission time is compatible to WMS for the following reason: We are doing the following: export X 509_USER_PROXY=delegated one �glite-ce-job-submit -a -r <ceid> myjdl 1 We must do the following: export X 509_USER_PROXY=delegated one � glite-ce-delegate-proxy -e <ce endpoint> <delid> �glite-ce-job-submit -D <delid> -r <ceid> myjdl 1 �glite-ce-job-submit -D <delid> -r <ceid> myjdl 2

CREAM CE: Direct submission The 2 nd procedure will speed the submission time since the delegation procedure is made just once It has however a point to remark: We begin the 1 st submission with a 48 h proxy We do not delegated anymore, so in 2 hours the proxy user with which we submit will be 46 h �From time to time we will have to refresh the proxy

CREAM CE: Final remarks We have made a 1 st test with the system Very dirty implementation in LCG. pm which now should be taken by Stefano for a clean setup We have to stop the production before the weekend Otherwise we will fulfill the CREAM CE with the outputs The retrieval procedure has to be implemented once the gridftp server is provided Already asked to FZK Purging procedure has to be implemented all the storage on the CE for each job is cleared, and it means that you can't check anymore the status of that job Automatic done via WMS submission, but not through a direct submission �glite-ce-job-purge An automatic purging procedure will be available with the next CREAM release