The HDF Group HDF 5 Tools Updates Peter

The HDF Group HDF 5 Tools Updates Peter Cao, The HDF Group September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 1

Tools Updates • HDF 5 Command-line tools • HDF-Java Products s e t a d p u d n a s e g n a Ch September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 2 www. hdfgroup. org

HDF 5 Command-line tools • Look or check file content h 5 dump, h 5 diff, h 5 ls, h 5 check, etc. • Change file content or create a new file h 5 repack, h 5 import, h 5 jam, h 5 copy, etc. • Convert an HDF 5 file to a different file h 4 toh 5, h 5 toh 4, gif 2 h 5, h 52 gif, etc. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 3 www. hdfgroup. org

h 5 ls Prints information about one or more HDF 5 files or objects September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 4 www. hdfgroup. org

h 5 ls • New features • ‘--follow-symlinks’ option to follow soft and external link. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 5 www. hdfgroup. org

h 5 ls • New features • ‘--no-dangling-links’ option to check dangling links September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 6 www. hdfgroup. org

h 5 dump Dumps contents of HDF 5 file to stdout ASCII text • XML • Binary • September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 7 www. hdfgroup. org

h 5 dump • New features • Show data pointed by region references September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 8 www. hdfgroup. org

h 5 dump • New features • Bit(s) values packed in a byte/short integer E. g. Quality flags Land/Water Background SST Skin Quality SST Bulk Quality Aerosol Correction September 28 -30, 2010 HDF and HDF-EOS Workshop XIV -- 1 -bit -- 2 -bits -- 3 -bits 9 www. hdfgroup. org

h 5 dump • Bug fixes • • Datatype of object references was displayed as the datatype of region references. Does not export data pointed by a region reference as a binary file. Ignore –m <set the floating point output format> option when generating XML with –u. Subsetting does not check dimension bounds. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 10 www. hdfgroup. org

h 5 diff Show differences between… two objects or two files September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 11 www. hdfgroup. org

h 5 diff • New features • ‘--follow-symlinks’ option to follow soft and external link. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 12 www. hdfgroup. org

h 5 diff • New features • ‘--no-dangling-links’ option to check dangling links September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 13 www. hdfgroup. org

h 5 diff • New features • ‘--exclude-path’ option to exclude specified object pointed by a given path for comparison. • Will apply to attributes in the future. E. g. h 5 diff --exclude-path /g 1/g 2/f tools_diff 1. h 5 tools_diff 2. h 5 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 14 www. hdfgroup. org

h 5 diff • Bug fixes • Improved to traverse groups recursively for comparing child objects when the groups are specified to be compared via command line. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 15 www. hdfgroup. org

h 5 repack Copies an entire file to a new file… Changes storage layouts and apply compression filters September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 16 www. hdfgroup. org

h 5 repack • Bug fixes • • • Attributes on the datasets with H 5 T_REFERENCE type were not present in the repacked file A data element which was a reference to a group became NULL in the repacked file Display compression information incorrectly. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 17 www. hdfgroup. org

h 5 copy Copies objects to a different file / / floats integers i 1 i 2 64 -bit f 32 f 1 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV f 2 18 www. hdfgroup. org

h 5 copy • Bug fixes • • • Failed to copy dangling links when it’s specified directly as the source object. Couldn’t deal with NULL elements of object references Copying references was not tested and properly documented September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 19 www. hdfgroup. org

New Tools in development h 5 watch September 28 -30, 2010 h 5 edit HDF and. HDF/HDF-EOS Workshop XIVXIII 20 www. hdfgroup. org

h 5 watch • • • Allows the user to monitor the growth of a dataset Prints out the new elements appended to a dataset Includes options on what to be monitored. September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 21 www. hdfgroup. org

h 5 edit • Motivation • Create an attribute • Change the value of an attribute in a dataset • Edit other objects in an HDF 5 file • Requirements • Familiar syntax for the input data (DDL or h 5 import) • Scriptable (can be used in a shell script ) • Extendible to support other HDF 5 object September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 22 www. hdfgroup. org

h 5 edit • New tool for editing an HDF 5 file RFC: http: //www. hdfgroup. org/pubs/rfcs/ • Initial implementation will include creating and deleting attributes • Prototype will be released soon • Please send us your comments! September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 23 www. hdfgroup. org

Example: h 5 edit Ø H 5 edit --command “delete ATTRIBUTE Percentage_per_Volume /m 1; ” file 1. h 5 Ø h 5 edit --command “create September 28 -30, 2010 ATTRIBUTE Percentage_per_Volume HDF and HDF-EOS Workshop XIV 40 24 /m 1; ” file 1. h 5 www. hdfgroup. org

The HDF Group HDF-Java Products Support for HDF 5 1. 8 HDF-Java 2. 7 release September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 25

HDF-Java Products HDF-Java HDF Java Wrapper September 28 -30, 2010 Object Package HDF and HDF-EOS Workshop XIV Viewer Editor 26 www. hdfgroup. org

Java Wrapper HDF JNI A programming framework that allows Java code to call and be called by applications and libraries written in other languages HDF September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 27 www. hdfgroup. org

Object Package: basic objects File Group Dataset Datatype Attribute http: //www. hdfgroup. org/hdf-java-html/hdf-object/ September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 28 www. hdfgroup. org

HDFView September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 29 www. hdfgroup. org

Support for HDF 1. 8 in HDF-Java • HDF 5 JNI (funded by Sandia National Lab) • Added about 118 (out of 169) new functions • Added unit tests for the. JNI functions Package H 5 S H 5 T H 5 D H 5 O H 5 L H 5 A H 5 R H 5 I H 5 P 1. 6 compatable APIs Total September 28 -30, 2010 No. of new APIs 3 7 5 17 18 14 1 2 34 17 118 HDF and HDF-EOS Workshop XIV 30 www. hdfgroup. org

Support for HDF 1. 8 in HDF-Java • Added HDF 5 1. 8 related features to the Object Layer and HDFView • • • Set the bounds of library versions Set link creation order Set link storage type Show groups and attributes in creation order (not in HDFView) Create soft and external links Retrieve link information September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 31 www. hdfgroup. org

HDF-Java 2. 7 • New features (17), e. g. • • • Show bit(s) values in byte or short integers Move objects within the same file Import/export binary data Use H 5 Ocopy() API (important for copying large datasets) Rename Attributes Support for 64 -bit Windows September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 32 www. hdfgroup. org

HDF-Java 2. 7 • Bug fixes (29), e. g. • • Java binary jar files are not the same for 32 -bit and 64 -bit Datasets displayed incorrectly if you swap dimensions H 5 Sset_extent_simple in Java does not seem to take effect Possible useful information for HDF-EOS 2 files are not displayed by HDFView September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 33 www. hdfgroup. org

HDF-Java 2. 7 • • Support for HDF 5 1. 8 Bug fixes/other new features Beta is out for you to try Official release – Nov. 2010 September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 34 www. hdfgroup. org

HDF-Java Features/Roadmap Nov. 2010 • Support for HDF 5 1. 8 • N-dimension Array • Buffered Array • Function pointers • Unit tests for JNI • Creation Order * • Buffered Array • Edit complex data • Bug fixes • Import/Export Geo. Tiff * • Import/export XML * • 3 D Graphic view * • Other new features • GUI Widgets * • Drag/drop (in & ex) * • Automatic GUI tests * * For HDFView only Please give us your feedback September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 35 www. hdfgroup. org

Thank You! Questions/comments? September 28 -30, 2010 HDF and HDF-EOS Workshop XIV 36 www. hdfgroup. org