Image Management and Rain on Future Grid A
- Slides: 18
Image Management and Rain on Future. Grid: A practical Example Presented by Javier Diaz, Fugang Wang, Gregor von Laszewski http: //futuregrid. org
Motivation • Provide users with the ability to create their own environments to run their projects (OS, packages, software) • Users can deploy their environments in both baremetal and virtualized infrastructures • The security is important http: //futuregrid. org
Architecture http: //futuregrid. org
Image Repository • Service to query, store, and update images through a unique and common interface https: //portal. futuregrid. org
Image Repository Examples • Query the image repository – CLI: fg-repo –u jdiaz –q “* where os=centos_5” – Shell: list * where os=centos Authentication OK 2 items found img. Id=215369546596144595085417, os=centos_5, arch=x 86_64, owner=jdiaz, description=None, tag=jdiaz 2699012769, vm. Type=none, img. Type=machine, permission=private, status=available img. Id=68725515834828774883357, os=centos_5, arch=x 86_64, owner=jdiaz, description=None, tag=jdiaz 1786816389, vm. Type=none, img. Type=machine, permission=private, status=available • Upload an Image – CLI: fg-repo –u jdiaz –p imagefile. tgz “os=centos&vmtype=kvm&description=my image” – Shell: put imagefile. tgz os=centos&vmtype=kvm&description=my image Checking quota and Generating an Img. Id Authentication OK Uploading image. You may be asked for ssh/passphrase password Imagefile. tgz 100% 53 0. 1 KB/s 00: 00 Registering the image https: //portal. futuregrid. org The image has been uploaded and registered with id 211913675261934066702430
Image Repository Examples • Add User – CLI: fg-repo –u jdiaz --useradd user. Id – Shell: user –a user. Ids Authentication OK User created successfully. Remember that you still need to activate this user (see setuserstatus command) • Image Usage – CLI: fg-repo –u jdiaz --histimg – Shell: histimage Authentication OK img. Id=191563243441508818679593, created. Date(UTC)=2011 -10 -13 21: 43: 30, last. Access(UTC)=2011 -10 -24 17: 37: 45, access. Count=16, img. Id=111462205747829171557134, created. Date(UTC)=2011 -10 -14 20: 36: 40, last. Access(UTC)=2011 -10 -21 13: 48: 04, access. Count=4, img. Id=21870735808909675281040, created. Date(UTC)=2011 -10 -07 20: 36: 33, https: //portal. futuregrid. org last. Access(UTC)=2011 -10 -07 20: 36: 33, access. Count=0,
Image Generation • Creates and customizes images according to user requirements: o OS type o OS version o Architecture o Software Packages • Image is stored in the Image Repository or returned to the users • Images are not aimed to any specific infrastructure http: //futuregrid. org
Generate an Image • fg-generate –u jdiaz –o centos –v 5 –a x 86_64 –s python 26 Client output: Generate img Deploy VM And Gen. Img Image generator client. . . 1 Please insert the password for the user jdiaz 2 Password: Selected Architecture: x 86_64 Connecting server: i 120: 567913 Your image request is in the queue to be processed Store in the Repo ------wait here if too many request are being processed-----or Your image request is being processed Return it to user Generating the image ------wait here until finished-----Your image has be uploaded in the repository with ID=915678426632408832461797 The image and the manifest generated are packaged in a tgz file. Please be aware that this Future. Grid image does not have kernel and fstab. Thus, it is not built for any deployment type. To deploy the new image, use the IMDeploy command.
Image Deployment • Customizes (network IP, DNS, file system table, kernel modules, etc) and deploys images for specific infrastructures • Two main infrastructures types – HPC deployment: it means that we are going to create network bootable images that can run in bare metal machines – Cloud deployment: it means that we are going to convert the images in VMs http: //futuregrid. org
Deploy/Stage an Image for HPC • fg-deploy –u jdiaz –r 2131235123 –x india Client output: Starting image deployer. . . Deploy img Please insert the password for the user jdiaz from Repo Get img from Password: Repo 1 Connecting to x. CAT server 2 ------wait here if an image is being deployed----Register img in Authentication OK Customize img 5 Moab and Customizing and deploying image on x. CAT 6 3 recycle sched here until finished------wait Return info Connecting to Moab server about the img Register img in x. CAT Your image has been deployed in x. CAT as centosjavi 960524558. (cp files/modify tables) Please allow a few minutes for x. CAT to register the image before attempting to use it. 4 To boot an machine using your image: qsub -l os=<imagename> To check the status of the job you can use checkjob and showq commands
Deploy/Stage an Image stored in the Repository to Open. Stack • fg-deploy –u jdiaz –r 2131235123 -s –v ~/novarc Client output: Starting image deployer. . . Please insert the password for the user jdiaz Password: Deploy img Authentication OK ------wait here until finished----- from Repo Get img from Retrieving image. You may be asked for ssh/passphrase password Repo 1 centos 5 jdiaz 2250444196. img 100% 1496 MB 65. 0 MB/s 00: 23 2 euca-bundle-image …. Upload the euca-upload-image … Customize img to the euca-register … 5 4 Cloud 3 IMAGE emi-437 C 1239 Return img Your image has been registered on Open. Stack with the id emi-437 C 1239 to client To launch a VM you can use euca-run-instances -k keyfile -n <#instances> id Remember to load you Eucalyptus environment before you run the instance (source eucarc) More information is provided in https: //portal. futuregrid. org/tutorials/oss and in https: //portal. futuregrid. org/tutorials/eucalyptus
List Images Deployed on x. CAT/Moab • fg-deploy –u jdiaz -l –x india Client output: List deployed Images Starting image deployer. . . Please insert the password for the user jdiaz 1 Password: Return Images Connecting to x. CAT server both know Authentication OK 4 about Connecting to Moab server Tell The list of available images on x. CAT/Moab is: me what Tell me what centosjdiaz 960524558 images you 3 2 images you centosfuwang 1549296807 know You can get more details by querying the image repository using IRClient. py -q command the query string: "* where tag=imagename". NOTE: To query the repository you need to remove the OS from the image name (centos, ubuntu, debian, rhel. . . ). The real name starts with the username. https: //portal. futuregrid. org
RAIN • Rain (Runtime Adaptable INsertion Configurator) • We want to provide custom HPC environment, Cloud environment, or virtual networks ondemand with little effort http: //futuregrid. org
Rain an Image and execute a task (baremetal) • fg-rain –u jdiaz –r 123123123 –x india –j Run job in my testjob. sh –m 2 7 qsub, monitor status, Client output: image stored in the repo 1 completion status and Starting rain. . . indiacate output files Please insert the password for the user jdiaz Deploy img Password: ----- Deploy the image. Same logs as before --2 Get img from Job id is: 200941 Deploy img 4 Repo 3 Wait until the job finishes from Repo State: Idle Register img Customize img 7 State: Idle in Moab and 5 State: Running recycle Return 8 State: Running sched info about Register img in x. CAT State: Completed the img (cp files/modify tables) Completion Code: 0 Time: Fri Oct 28 15: 02 6 The Standard output is in the file: salida. txt The Error output is in the file: jobscript. e 200941
Rain: Execute a task in a deployed image (Open. Stack) Client output: • fg-rain –u jdiaz –i i-00000061 –s –v ~/novarc –j Starting rain. . . Please insert the password for the user jdiaz Password: Run. Launching image job via ssh Run job in my and. Waiting for running state in all the VMs wait until image that is --- Prints status each X seconds--finished deployed 1 Number of instances booted 2 Instance i-000009 e 9 associated with address 149. 165. 146. 152 5 Boot VMs Instance i-000009 ea associated with address 149. 165. 146. 155 2 Waiting to have access to Instance i-000009 e 9 associated with address 149. 165. 146. 152 Waiting to have access to Instance i-000009 ea associated with address 149. 165. 146. 155 Wait for Configure VMs All VMs are accessible: True Cloud access to VMs 4 3 for. Creating temporal sshkey files user and Framework Copying temporal private and public ssh-key files to VMs mount his VM VM Configuring ssh in VM and mounting home directory (assumes that sshfs and ldap is installed) home directory Copying temporal private and public ssh-key files to VMs VM in /tmp using Configuring ssh in VM and mounting home directory (assumes that sshfs and ldap is installed) sshfs VM VM Running Job VM --- Waits until job is done--Job Done testjob. sh –m 2
Some Numbers… (I) https: //portal. futuregrid. org
Some Numbers… (II) https: //portal. futuregrid. org
Manual and Tutorials • Tutorial: https: //portal. futuregrid. org/tutorials/image_mana gement_and_rain • Manuals – fg-shell: https: //portal. futuregrid. org/man/fg-shell – fg-repo: https: //portal. futuregrid. org/man/fg-repo – fg-generate: https: //portal. futuregrid. org/man/fggenerate – fg-deploy: https: //portal. futuregrid. org/man/fg-deploy https: //portal. futuregrid. org
- Future perfect continuous tense examples
- Future perfect e future continuous
- Rain rain go away by isaac asimov summary
- Pga vs lga
- Abb statcom
- In futuregrid runtime adaptable
- Future perfect future continuous exercises
- Future tenses summary
- Future plans and finished future actions
- Future perfect and future continuous
- Future continuous and future perfect objasnjenje
- Future perfect presentation
- Past and present tense
- Future nurse future midwife
- Past present continuous examples
- Future continuous for future arrangements
- Analog image and digital image
- Smart grid enables distributed energy management
- 6 box talent grid