GRID Parallel Processing GRIDrelated activity in Japan Go

GRID & Parallel Processing GRID-related activity in Japan Go Iwai, K. Murakami (KEK) Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 1

GRID & Parallel Processing Outline n Geant 4 Parallelization n GRID deployment around KEK n GRID portal on Web ü Radiotherapy simulation with GRID Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 2

GRID & Parallel Processing Geant 4 Parallelization Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 3

GRID & Parallel Processing ü implemented with MPI. » tested with MPICH and LAM. ü based on a master-slave model. » UI commands are scattered to the slaves from the master. » The results in each slave are gathered into the master at the end of every event/run. send commands n Parallel software beam-on master slave receive results Geant 4 Parallelization event loop n Thread-out of event loop ü enables the slaves to respond to request from the master. ü The master can get informationprocess of slaves during event loop. Koichi Murakami request sub-thread for event loop 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 4 time

GRID & Parallel Processing Benchmarking Results Parallel efficiency (right-hand scale) The inverse of real elapsed time (left-hand scale) 30 times faster than single process execution Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 5

GRID & Parallel Processing Summary on Geant 4 Parallelization n Geant 4 parallel components have been implemented using MPI based on master-slave model. ü Applications based on these components are executed with one master and one or more slaves. ü Existing applications can be parallelized rapidly and easily. » just switching G 4 Run. Manager and G 4 UIsession n An existing our medical application based on Geant 4 was parallelized and tested on our PC cluster. ü A parallelized application show the performance gain of nearly 30 times faster on our PC cluster 20 nodes (40 CPUs). ü 2 nd loop of software improvement ü will commit to extended example Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 6

GRID & Parallel Processing GRID Deployment around KEK Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 7

GRID & Parallel Processing Grid Services provided by KEK n CA Service ü http: //gridca. kek. jp/ ü started on Feb. 2006. ü service for Japanese HEP activity ü # of issued user certificates: 25 / # of issued host certificates: 74 n VO Membership Service ü http: //voms. kek. jp/ ü Supported VOs: » » “apdg” is the VO for Asia-Pacific Data Grid. “belle” is the VO for Belle experiments. “atlasj” is the VO for Atlas experiments in Japan. “g 4 med” is the VO for Geant 4 medical application. Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 8

GRID & Parallel Processing GRID System n JP-KEK-CRC-01 (Pre-Production System) ü since Nov. 2005. ü registered to GOC. ready to WLCG (World wide LCG). ü Site role: » practice for production system JP-KEK-CRC 02. » test use among university groups in Japan. ü Resource and Component: » SL-3. 0. 5 w/ LCG-2. 7 » upgrade to g. Lite-3. 0 is done. » CPU: 14, Storage: 1 TB ü Supported VOs: » belle, apdg, dteam and ops n JP-KEK-CRC-02 (Production System) ü since early 2006. ü registered to GOC. ready to WLCG. ü Resource and Component: » SL or SLC w/ LCG-2. 7 » upgrade to g. Lite-3. 0 is done. » CPU: 48, Storage: 6 TB (w/o HPSS) ü Supported VOs: » belle, apdg, atlasj, ilc, dteam and ops n JP-KEK-CRC-00 (Testbed System) ü since Jun. 2005. ü Resource and Component: » SL-3. 0. 5 w/ g. Lite-3. 0 (100% pure) ü Supported VOs: » belle, apdg, atlasj and g 4 med Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 9

GRID & Parallel Processing GRID Portal on Web Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 10

GRID & Parallel Processing Usecase of Medical Application with GRID DICOM formatted data firewall getting results parameter settings GRID JDL generation job submission Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 11

GRID & Parallel Processing More Structure of GRID Web Interface User Applications GRID access • issue of proxy certificate • job management • job monitoring • post-process for job outputs GRID middleware g. Lite/LCG + web server • parameter inputs • JDL auto generation Koichi Murakami UI node Element 11 th Geant 4 Collaboration Workshop / Storage LIP - Lisboa (10 -14/Oct. /2006) 12

GRID & Parallel Processing GRID Access Layer n GRID API/commands ü issue of proxy certificates » xxx-proxy-init/info/destroy ü job management » submission/cancellation ― xxx-job-submit/xxx-job-cancel ü job monitoring » xxx-job-status ü post-process for job outputs » merging job outputs (histogram, etc. ) » collection/replication of results ― xxx-job-get-output, lfc-xxx, lcg-cp, lcg-cr, etc. ü These staffs are used for communication with the UI node, on which a web server runs. n HTML generation ü as a user front-end. » providing front-ends for accessing GRID resources. ü web pages for job submission, monitoring, etc. » providing functions to automatically generate related web forms. Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 13

GRID & Parallel Processing Radiotherapy Simulation on GRID n Typical situation of particle therapy simulation; ü ~3 days/1 M events » @ Pentium-4 3. 0 GHz processor / 1 GB memory ü massive computing power is required for precise simulation n Input parameters ü Selecting facility » HIBMC/NCC/NIRS-IHI/…. (Japanese facilities) ü Geometry (beamline modules) » collimator/wobler magnet/scatterer/range shifter/ridge filter/MLC/… ü Target » water phantom/human body (DICOM) ü Beam condition » beam energy/beam spread ü Simulation parameters » physics lists » cut values n Outputs ü dose distribution » ROOT file/GDD file/… Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 14

GRID & Parallel Processing Current Status & Future Prospects n Medical application of Geant 4 and GRID ü high speed MC-based dose calculation system in radiotherapy requires large-scale computing resources. n Application of GRID is a key component to boost simulation speed. ü We are developing an easy-to-use user interface on our GRID environment. ü We can get a secure and efficient way of sharing computing resources in the context of GRID technology. » Secure globally-distributed job execution » Web application interface Koichi Murakami 11 th Geant 4 Collaboration Workshop / LIP - Lisboa (10 -14/Oct. /2006) 15