The HDF Group HDF Tools Tutorial Peter Cao

The HDF Group HDF Tools Tutorial Peter Cao, The HDF Group Jonathan Kim, The HDF Group September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 1 www. hdfgroup. org

Tools Tutorial • HDFView • HDF 5 Command-line tools September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 2 www. hdfgroup. org

HDFView • View file content • Edit file content • Show NPOESS/JPSS related features September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 3 www. hdfgroup. org

HDFView • Download HDF-Java 2. 7 beta from http: //www. hdfgroup. org/ftp/HDF 5/hdf-java-2. 7 • Download example files from ftp: //ftp. hdfgroup. uiuc. edu/pub/outgoing/Workshop. XIV/HDF 5 -Files/ September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 4 www. hdfgroup. org

HDFView • View file content • • • Open OMI-Aura_L 3. he 5 Open groups (try expand all) Stretch the metadata view for showing metadata Double click on “Column. Amount. O 3” to show data in table Use “Open As” to show “Column. Amount. O 3” in image • View a subset • Open weather. h 5 • Select from preview image or selection fields September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 5 www. hdfgroup. org

HDFView • Copy/paste a dataset and a group • Create a new file: tmp. h 5 • Copy/paste “Column. Amount. O 3” and show the image from the new file • Copy the group “GRIDS” to show the whole group is copied September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 6 www. hdfgroup. org

HDFView • Create a dataset and add an attribute • • • Create /d 1 (32 -bit float, 720 x 1440) Copy “Column. Amount. O 3” to /d 1 Add attribute “IMAGE_MINMAXRANGE = 123. 0, 447. 0” Export /d 1 to text file Create /d 2 and import d 1. txt September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 7 www. hdfgroup. org

HDFView • NPOESS/JPSS related features • Object reference • Region references • Quality flags September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 8 www. hdfgroup. org

HDFView • Object reference 1) 2) 3) 4) 5) 6) Open file: SVI-NPP. h 5 Open “VIIRS-I 1 -SDR_Aggr” Click on object ID and shows object name Right mouse to click to open data as image Open data in table Select two cells and open them at once September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 9 www. hdfgroup. org

HDFView • Region references 1) 2) 3) 4) Open “VIIRS-I 1 -SDR_Gran” Right mouse to click to open data as image Open data in table Select two cells and open at once September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 10 www. hdfgroup. org

HDFView • Quality flags 1) Quality flags • • Land/Water Background SST Skin Quality SST Bulk Quality Aerosol Correction -- 1 -bit -- 2 -bits -- 3 -bits 2) Open QF_VIIRSI 1 SDR_Array 3) Use "Open As" on QF_VIIRSI 1 SDR_Array 4) Select first bit September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 11 www. hdfgroup. org

HDFView • Advanced features 1) Import/export images 2) Use HDF-EOS plugin September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 12 www. hdfgroup. org

HDF 5 Command-line tools • • • h 5 ls h 5 dump h 5 repack h 5 diff h 5 copy t Wha September 28 -30, 2010 HDF and HDF-EOS Workshop XIV o o t e thes u o y r o fo d n a ls c 13 www. hdfgroup. org

h 5 ls • Show file content list • h 5 ls ami. h 5 • h 5 ls –r ami. h 5 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 14 www. hdfgroup. org

h 5 dump • • • September 28 -30, 2010 Structure Dataset Binary XML Region references HDF and HDF-EOS Workshop XIV 15 www. hdfgroup. org

h 5 dump • Structure • h 5 dump -H tools_diff 1. h 5 • h 5 dump -H ami. h 5 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 16 www. hdfgroup. org

h 5 dump • Dataset • Content • h 5 dump -d /En. Sight_model/geometry/parts/part_2/quad 4 ami. h 5 • Information • h 5 dump –p –H -d /En. Sight_model/geometry/parts/part_2/quad 4 ami. h 5 • Subset • H 5 dump -d /En. Sight_model/geometry/parts/part_2/quad 4 -s 10, 0 -c 20, 2 ami. h 5 • <-s and –c must be used after –d> September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 17 www. hdfgroup. org

h 5 dump • Binary • h 5 dump -d /En. Sight_model/geometry/parts/part_2/quad 4 -b LE –o out. bin ami. h 5 • XML • h 5 dump –x ami. h 5 • Region • • h 5 dump -d /Data_Products/VIIRS-I 1 -SDR_Gran_0 -s 3 -c 1 SVI-NPP. h 5 dump –R -d /Data_Products/VIIRS-I 1 -SDR_Gran_0 -s 3 -c 1 SVI-NPP. h 5 • Packed fields <only 1. 8. 5+> September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 18 www. hdfgroup. org

h 5 repack • Remove inaccessible objects / junk spaces • Change storage layout • Apply compression filter Copies a file to a new file with different storage layouts and compression filters September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 19 www. hdfgroup. org

h 5 repack • Remove inaccessible objects • h 5 dump tools_junk. h 5 • h 5 repack tools_junk. h 5 tmp. h 5 • Change layout • h 5 dump -H tools_bad_layout. h 5 • h 5 repack tools_bad_layout. h 5 tmp. h 5 • h 5 repack -l CHUNK=16 x 16 tools_bad_layout. h 5 tmp. h 5 • Change compression • h 5 repack -f GZIP=6 tmp. h 5 tmp 2. h 5 • More: use latest version, link storage, shared message, and etc September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 20 www. hdfgroup. org

h 5 diff Show differences between two files or two objects • Dataset • Whole file September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 21 www. hdfgroup. org

h 5 diff • Dataset • h 5 diff -r tools_diff 1. h 5 tools_diff 2. h 5 /g 1/uint 8 /x • Whole file • Run “h 5 ls –r tools_diff 1. h 5” and “h 5 ls -r tools_diff 2. h 5” • h 5 diff tools_diff 1. h 5 tools_diff 2. h 5 • h 5 diff –v tools_diff 1. h 5 tools_diff 2. h 5 • Follow symlinks • h 5 diff -r --follow-symlinks tools_diff 1. h 5 tools_diff 2. h 5 /sl 1 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 22 www. hdfgroup. org

h 5 copy • Dataset • Group Copies objects to files… September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 23 www. hdfgroup. org

h 5 copy • Dataset • h 5 copy -i tools_input. h 5 -o tmp. h 5 -s /g 1/g 2/f -d /float • Group • Default: h 5 copy -i tools_input. h 5 -o tmp. h 5 -s /g 1 -d /g 1 • Parent groups: h 5 copy -p -i tools_input. h 5 -o tmp. h 5 -s /g 1/g 2/f -d /g 1/g 2/f September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 24 www. hdfgroup. org

Thank You! Questions/comments? September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 25 www. hdfgroup. org