Py Storm Tracker A Parallel ObjectOriented Cyclone Tracker

Py. Storm. Tracker: A Parallel Object-Oriented Cyclone Tracker in Python Albert Yau School of Marine and Atmospheric Sciences Stony Brook University Kevin Paul and John Dennis Application Scalability and Performance (ASAP) Group Computational and Information Systems Laboratory National Center for Atmospheric Research 11/5/2020 1

Outline • • • Scientific Background Motivation and Objectives Implementation Sample Output Benchmarks Summary and Future Work 11/5/2020 2

What are cyclones? 11/5/2020 A figure from a nineteenth century geography text showing storm frequency distribution (Hinman, 1888), adapted from Chang et al. (2002) 3

Scientific Background • Cyclones are synoptic scale (~1000 km) moving low pressure systems • Extra-tropical cyclones are responsible for much of the day-to-day weather variability and extreme weather events at mid-latitudes Evolution of an extratropical cyclone as in the Norwegian Cyclone Model. (Source: weather. gov) 11/5/2020 4

Motivation • Tracking extra-tropical cyclones is challenging (Neu et al. , 2013): – Cyclones vary greatly in size and shape, move with different velocities, split and merge over the lifetime • Most current cyclone trackers are written in FORTRAN, the codes are not readily available – Hard to modify, optimize, or compare existing cyclone trackers 11/5/2020 5

Objective 1 • Implement a user and developer friendly cyclone tracker in Python using an objectoriented approach – Flexible and extensible by inheriting or replacing the classes and methods – Use existing scientific computing libraries such as Num. Py and Sci. Py – The source codes will be publicly available at Git. Hub 11/5/2020 6

Motivation • Emergence of high resolution and multi-ensemble model data – CESM Large Ensemble project: 40 ensemble members from 1920 -2100 for CMIP 5 historical and RCP 8. 5 scenarios – Global Ensemble Forecast System (GEFS): up to 16 days of global forecasts from 21 ensemble members GEFS ensemble members Source: schumacher. atmos. colostate. edu 11/5/2020 7

Objective 2 • To be able to process data from different datasets – GRIB: most commonly used in reanalysis and weather models: NCEP/NCAR Reanalysis and GEFS – net. CDF: a more recent standard commonly used in climate models: CMIP 5 archive of more than 30 climate models – Py. NIO is chosen for file I/O • Utilize parallel computing technique to accelerate processing of large amount of data – MPI 4 Py as an interface to the MPI message passing library 11/5/2020 8

Implementation: Task Parallelism Task 0 grid centers tracks Task 1 grid centers tracks Task 2 grid centers tracks grid tracks (meta data only) Task 3 grid centers tracks Task 4 grid centers tracks (tree reduction) … Task n 11/5/2020 9

Implementation: Classes Rect. Grid class: • Rectilinear grid derived from Grid class • Does not read data until actually required Center class: • Split into smaller objects • Detect centers from Grid by time data using scipy. ndimage. filters • Calculate great circle distances between centers stormtracker. py from detector import Rect. Grid, Center from linker import Tracks If __name__ == “__main__”: grid = Rect. Grid(filepath=“…”) centers = grid. detect() tracks = Tracks() tracks. append_center(centers) Tracks class: stormtracker. py: • Input a list of Center • The main controller for • Match centers to existing generating Grid, Center Tracks using nearest and Track objects neighborhood method • Manage parallel processes • If matched, append to exisiting track; else append center as a new track • Merge two Tracks objects detector. py class Grid(object): class Center(object): class Rect. Grid(Grid): def __init__(self, filepath, trange): def get_time(self): def get_var(self, trange): def split(self, num): 11/5/2020 linker. py class Tracks(object): def __init__(self, time, lat, lon, var): def abs_dist(self, center): The code repository will be available at: https: //github. com/mwyau/Py. Storm. Tracker def __init__(self): def match_center(self, centers): def append_center(self, centers): def merge_tracks(self, tracks): 10

Sample Output CESM Large Ensemble #31: • 6 -hourly PSL • 1 -degree resolution • 192*288 grid 11/5/2020 • Year 1990, 769 tracks Tracks deeper than 975 h. Pa, moved 1000 km and longer than 2 days 11

Benchmark: CESM 1 year – 1460 time steps Number of MPI Tasks Detector Time Linker Total Speedup for CESM 1 -year PSL 1000 1 1028. 22 569. 55 1597. 78 2 517. 25 297. 34 814. 58 4 261. 58 160. 38 421. 96 8 136. 39 88. 17 224. 56 16 77. 21 48. 20 125. 40 32 46. 99 30. 73 77. 72 64 30. 44 21. 43 51. 86 128 20. 80 17. 59 38. 39 256 17. 67 15. 86 33. 53 512 15. 77 15. 17 30. 93 Ideal Speedup Detector 1024 15. 63 15. 20 30. 83 Linker Total 11/5/2020 Speedup 100 10 1 1 16 256 Number of MPI Tasks 12

Benchmark: CESM 15 years – 22360 time steps Time Linker Total 16 1057. 31 749. 45 1806. 76 32 537. 75 475. 40 1013. 15 64 272. 01 337. 67 609. 69 128 143. 80 264. 06 407. 86 256 80. 64 226. 78 307. 42 512 47. 35 209. 60 256. 95 1024 30. 71 200. 80 231. 50 2048 23. 16 197. 29 220. 45 4096 19. 33 196. 02 215. 36 11/5/2020 Speedup for CESM 15 -year PSL 100 Speedup Number of MPI Tasks Detector 10 1 16 256 4096 Number of MPI Tasks Ideal Speedup Detector Linker Total 13

Summary and Future Work • The cyclone tracker is implemented using an object-oriented approach, which is flexible and easily extended: – For example: implement a Cubed. Sphere. Grid class; track tropical cyclones 11/5/2020 14

Summary and Future Work • We achieved reasonably good speedup in the detector, while the bottleneck is in the linker: 11/5/2020 Speedup – Reduce the communication cost by preselect tracks before merging – Instead of serializing (pickling) the objects, use buffered send/receive in MPI 100 10 1 16 256 4096 Number of MPI Tasks Ideal Speedup Detector Linker Total 15

Summary and Future Work • Verify the quality of tracks: – Compare the tracks with other well established cyclone trackers NCEP/NCAR Reanalysis: Count of cyclone tracks deeper than 975 h. Pa, moved at least 1000 km and existed longer than two days in the north Pacific winter, compared with Hodges (1999) tracker 11/5/2020 16

Thank you! • Questions? • Email: albert. yau@stonybrook. edu • The code repository will be available at: https: //github. com/mwyau/Py. Storm. Tracker 11/5/2020 17