Introduction to Boston Universitys Scientific Computing Facility Doug

Introduction to Boston University’s Scientific Computing Facility Doug Sondak sondak@bu. edu Scientific Computing and Visualization

Information Services & Technology 12/12/2021 Outline § § § § Introduction Hardware Software Account Information Tutorials Some Basic Unix Commands How to Run Batch 2

Information Services & Technology 12/12/2021 Introduction § BU’s Scientific Computing Facility (SCF) addresses high-performance computing and visualization needs of the BU community § Large resource requirements § CPU time § Memory § One group used 3 centuries of CPU time last year! 3

Information Services & Technology 12/12/2021 Overview of SCV § Scientific Computing and Visualization § Computing § high-performance, parallel computers (SCF) § support § parallelization § optimization § tutorials § Visualization § support to create graphics, from publication figures to interactive 3 D displays and movies § 15’x 8’ tiled display for 3 D visualization § tutorials 4

Information Services & Technology 12/12/2021 Current Hardware - Blade. Center § IBM Blade. Center § katana § currently 272 processors § heterogeneous mixture of blades § Intel Xeon and AMD Opteron § 2. 4 -3. 0 GHz § blades contain 4 or 8 processors § May use up to 32 processors at a time § parallel or serial § Two blades each have 8 processors and 96 GB memory 5

Information Services & Technology 12/12/2021 Current Hardware – Blue Gene § IBM Blue Gene § 1024 nodes, each with 2 processors § Power. PC 440 processors § 700 MHz § 32 -bit § only runs MPI codes § each processor is relatively slow, so you need good scalability to make it worthwhile § May use 1024 processors during the day, all 2048 processors at night 6

Information Services & Technology 12/12/2021 Current Hardware - p. Series § IBM p 655 § shared-memory machines § Power 4 processors § 9 machines, each with 8 processors § 6 machines 1. 1 GHz, 16 GB memory § 3 machines 1. 7 GHz, 8 GB memory § login node is twister 7

Information Services & Technology 12/12/2021 Software § All machines run under Unix § C, C++, Fortran, Java, Perl, Python, Tcl § Matlab § all machines except Blue Gene § PCT (Parallel Computing Toolbox) § Maple § all machines except Blue Gene § Mathematica § all machines except Blue Gene 8

Information Services & Technology 12/12/2021 Software (cont’d) § Gauss § blade center, p. Series § R § all machines except Blue Gene § VTK § Paraview § Maya 9

Information Services & Technology 12/12/2021 Accounts and Policies § Accounts and allocations are based on “projects” § Must be faculty or research staff to apply for projects § Project PI then adds accounts for students, collaborators, etc. § May assign post-doc or admin. staff as “administrative contact” § Apply for project at http: //www. bu. edu/tech/accounts/special/research/accounts/applications/ § Click on “Boston University faculty and research staff may apply for a new project. ” § If you’re not sure how much time to ask for, 1000 katana hours is considered modest § The form has fields for requesting time on each machine, but the time is actually all in one pot, and can be spent on any machine § This will give you accounts on all machines except for Blue Gene 10

Information Services & Technology 12/12/2021 Accounts and Policies (cont’d) § For Blue Gene account, once you are awarded a “regular” account go back to § http: //www. bu. edu/tech/accounts/special/research/accounts/applications/ § There is a paragraph starting with “Blue Gene accounts. ” Click on the “SCF User Information Page” link. § You will be requested for your user name and password § Click on the “Update Personal Information” link 11

Information Services & Technology 12/12/2021 Disk Space § The disk space in your home directory is minimal § Most project PI’s request a “project” directory § project directory contains disk space allocated to the specific project § once your account has been activated go back to http: //www. bu. edu/tech/accounts/special/research/accounts/applications/ and click “request a Project Disk Space allocation. ” § a request of a few GB will be rubber-stamped § large requests (hundreds of GB) will require stronger justification 12

Information Services & Technology 12/12/2021 “Buy-In” Nodes § Departments/groups may purchase their own Blade. Center nodes § alternative to buying and administering their own machines § Group that owns node(s) gets priority for batch runs § can be restricted to specific users if desired § May also have log-in node(s) for interactive use § log-in node restricted to specified groups/users 13

Information Services & Technology 12/12/2021 Services § SCV has 3 components: 1. system administration § excellent administrators! § can often accommodate special requests 2. visualization § can help with creating visualizations, including animations, for presentations, publication, and display wall § wall has 3 D capability 14

Information Services & Technology 12/12/2021 Services (cont’d) 3. high-performance computing § code parallelization and optimization § MPI § Open. MP § PCT (Matlab) § help with any large-scale computing issues § algorithms § coding § There is no charge for SCV services 15

Information Services & Technology 12/12/2021 Tutorials § SCV offers tutorials on a variety of computing and visualization topics § If you have interest in an area that we don’t presently cover, let us know! § Current offerings are in three areas § Programming § High-Performance Computing § Visualization 16

Information Services & Technology 12/12/2021 Tutorials (cont’d) § Programming § Intro to Matlab § Intro to Fortran § Intro to C/C++ § High-Performance Computing § § § Code Tuning (C, Fortran) Tuning Matlab Codes Intro to MPI Parallelization with Open. MP Matlab Parallel Computing Toolbox (PCT) 17

Information Services & Technology 12/12/2021 Tutorials (3) § Visualization § § § Intro to Scientific Visualization using Paraview Scientific Visualization using VTK Scientific Visualization using Matlab Graphics Programming in C/C++: Open. GL and Open. Scene. Graphics and Images for Presentation and Publication 18

Information Services & Technology 12/12/2021 Some Basic Unix 19

Information Services & Technology 12/12/2021 Login § From a terminal (e. g. , Xwin-32, putty) ssh katana § this requests a “secure shell” on katana § you’ll be prompted for your username and password § You’re now logged on to the blade server “katana” § You’ll see something like “katana: ~ %” § this is a Unix prompt, which is where you type in commands 20

Information Services & Technology 12/12/2021 Directories § “directories” are analogous to PC “folders” § delimited by / rather than as on a PC § when you first log in, current directory is called your “home” directory § directory where you are located at any given time is your “working” directory § pwd means “print working directory” § type pwd § to create a subdirectory, use mkdir (make directory) § type mkdir sub 1, where sub 1 is the subdirectory name 21

Information Services & Technology 12/12/2021 File Management § ls lists all files and directories in your current directory § type ls § you should see your new directory listed § some shorthand directory names: ~. . . home directory current directory one level above current directory § cd dirname (change directory) moves you to dirname § type cd sub 1 § type pwd 22

Information Services & Technology 12/12/2021 File Management (cont’d) § type ls ~ § lists everything in your home directory § type ls. . § since you’re one directory below your home directory, this is the same as ls ~ § cp from to will copy a file § from is the file name you’re copying from § to is either a file name or a directory § if it’s a file name, the copy will be given that name § if it’s a directory, the file will retain the old name and be placed in the specified directory 23

Information Services & Technology 12/12/2021 File Management (3) § copy the file /scratch/sondak/emacs_file to your sub 1 directory § cp /scratch/sondak/emacs_file. § we will use it later § a path is a full set of directories leading to a file or directory § e. g. , when you type pwd, it shows the path to your current directory § file suffixes generally don’t have any special meanings as they do under Windows § there are some cases where they do have meaning, such as. m for a Matlab program 24

Information Services & Technology 12/12/2021 Emacs § text files are created with an editor § kind of like a fancy “notepad” § two most common Unix editors are vi and emacs § since I use emacs, that’s what we’ll use here § we’ll introduce some emacs commands using the file emacs_file § two ways to enter commands § menu § shortcuts 25

Information Services & Technology 12/12/2021 Emacs Exercise § make sure you’re in your sub 1 directory § type emacs_file § the file will appear in an emacs window § try migrating around the file with the arrow keys § you can also navigate by clicking on the desired location § click on the 0 in 0. 282 § to delete a character use Delete button § hit Delete 3 times to delete 0. 2 § type 1. 3 § you’ve now changed 0. 282 to 1. 382 26

Information Services & Technology 12/12/2021 Emacs Exercise (cont’d) § Highlight 0. 288 with the mouse § Don’t include spaces before or after § Edit -> Cut will delete the highlighted characters § can also use CTL-w § Oh no, we made a mistake! Edit -> Undo undoes the previous command. Try it; 0. 288 should reappear. § can also use CTL-x u § can keep repeating undo, and it will undo last command, second-tolast command, third-to-last command, etc. 27

Information Services & Technology 12/12/2021 Emacs Exercise (3) § The point is the location on the left side of the cursor; i. e. , between the character on which the cursor resides and the character to its left. § The mark is similar to the point, but it’s location is set explicitly (i. e. , doesn’t move with cursor). 28

Information Services & Technology 12/12/2021 Emacs Exercise (4) § Suppose we want to delete all the characters between (inclusively) 0. 288 and 0. 407. § Set the cursor on the 0 in 0. 288. § To set the mark, CTL-spacebar § The note “Mark set” will appear at the bottom of the screen. § Move the cursor to the right of 0. 407 § We place it to the right of the 7 rather than on it because the point is always to the left of the cursor § Edit -> Cut will delete all characters between the mark and the point § shortcut is CTL-w 29

Information Services & Technology 12/12/2021 Emacs Exercise (5) § For now, let’s put the characters back in by using Edit -> Undo (or CTL-x u) § Move the cursor to the start of the current line using CTL-a § (CTL-e moves to end of current line) § Delete (“kill”) the line with CTL-k § Another CTL-k deletes the newline character 30

Information Services & Technology 12/12/2021 Emacs Exercise (6) § “Meta” key is the Esc key § We will use M to refer to the Meta key § The Meta key is hit prior to the subsequent key(s), not simultaneously as with CTL § Place the cursor at the top of the file § M-> will move the cursor to the bottom of the file (don’t confuse notation with arrows I’m using for menu notation; this is “Meta greater-than”) § M-< will move it back to the top 31

Information Services & Technology 12/12/2021 Emacs Exercise (7) § Whatever we deleted last is available in a buffer § Move the cursor to the beginning of the “M 1 rel” line § Edit -> Paste “yanks” the current buffer § can also use CTL-y 32

Information Services & Technology 12/12/2021 Emacs Exercise (8) § CTL-h gets you into the Emacs help function § at prompt type ? to list types of help § I often use “a” (apropos) § § Type CTL-h Then type a at the prompt Then type “kill” at the prompt The window will split, listing all Emacs commands that have something to do with the word kill 33

Information Services & Technology 12/12/2021 Emacs Exercise (9) § Type CTL-x 1 to get you back to the un-split screen § Split screens can be very handy, especially for cutting from one file and pasting in another § Type CTL-x 2 to split screen horizontally § can open different files in each half of the screen § toggle across split with CTL-o (“other”) § CTL-x 0 eliminates the half screen in which the cursor currently resides (try it) § this is a zero, not an “oh” 34

Information Services & Technology 12/12/2021 Emacs Exercise (10) § To save the modified file, File -> Save (current buffer) § can also use CTL-x CTL-s § A note will appear at the bottom of the window saying the file has been saved § To save it under a new name, File -> Save Buffer As… § can also use CTL-x CTL-w § You’ll be prompted for the name at the bottom of the screen § Note that when you get these kinds of prompts, you can edit them using emacs commands § Type a file name and then move back and forth with arrow keys § File -> Exit Emacs (or CTL-x CTL-c) to quit § Previous version will appear with ~ suffix 35

Information Services & Technology 12/12/2021 Katana Batch System 36

Information Services & Technology 12/12/2021 Katana Batch System § If your run will take less than 10 minutes of CPU time, you can run interactively § Longer runs are submitted to the batch system § Sun Grid Engine § May use up to 32 processors at a time § can be 32 serial runs § can be one 32 -processor parallel run § can be combination § Serial runs are limited to 24 hours § Parallel runs are limited to 5 hours 37

Information Services & Technology 12/12/2021 Katana Batch System (cont’d) § Job Control File (JCF) § § this is file you submit to batch system line starting with #! defines Unix shell lines starting with #$ are batch system commands lines starting with # are comments § Here’s a “minimalist” JCF for a serial run: #!/bin/tcsh # run time limit hr: min: sec #$ -l h_rt=2: 00 mycode 38

Information Services & Technology 12/12/2021 Katana Batch System (3) § JCF must have execute permission § chmod 755 myjcf § qsub myjcf submits your job to the queue § qstat –u username checks status of all your jobs 39

Information Services & Technology 12/12/2021 Katana Batch System (4) § Create a trivial Matlab code with some output, e. g. , a = 3; disp([‘a = ‘ int 2 str(a)]); § I called mine “dumbcode. m” § Create a JCF #!/bin/tcsh # run time limit hr: min: sec #$ -l h_rt=0: 02: 00 matlab –r –nodisplay dumbcode 40

Information Services & Technology 12/12/2021 Katana Batch System (5) § Submit to batch system § When it’s done 2 files should be in your working directory § dumbcode. e###### § dumbcode. o###### where ###### represents a 6 -digit number § The. e file contains errors § The. o file contains output 41

Information Services & Technology 12/12/2021 References § General SCV/SCF information § http: //www. bu. edu/tech/research/ § Blade Server § http: //www. bu. edu/tech/research/computation/linux-cluster/katana-cluster/ § Account Application and Maintenance § http: //www. bu. edu/tech/accounts/special/research/accounts/applications/ § Slides from Live SCV Tutorials § http: //www. bu. edu/tech/research/training/presentations/list/ § Web-based SCV Tutorials § http: //www. bu. edu/tech/research/training/tutorials/list/ § Katana Batch System § http: //www. bu. edu/tech/research/computation/linux-cluster/katana-cluster/runningjobs/ 42

Information Services & Technology 12/12/2021 Human Help § If you ever have any questions, feel free to contact us § Doug Sondak, sondak@bu. edu § Kadin Tseng, kadin@bu. edu 43