Committed datatypes in HDFView/h5dump

Hi all,
I'd like to be able to tell in HDFView/h5dump when a dataset has a
committed datatype. Ideally I'd like a path to the committed type be
provided (a la H5Iget_name), but it would be sufficient to identify it as
being a committed type. It would be very useful to me for the debugging
tools to display this and not have to use my tools to check their files.

I'm using committed types to implement a type of units handling, and it
works great, except that the standard tools are not very helpful in working
with them. It would be excellent to add this feature to HDFView and/or
h5dump, but I would also be interested in other approaches and/or
workarounds.

Cheers,
Martijn

Hi Martijn,

  if you use h5ls, then it will indicate named data types as being "shared" such as:

  h5ls -rv LOD.f5/t=000000000.0000000000/Triangulation/Faces/Points/Positions
Opened "LOD.f5" with sec2 driver.
t=000000000.0000000000/Triangulation/Faces/Points/Positions Dataset {94525/94525}
     Location: 1:1442029
     Links: 1
     Modified: 2014-09-17 01:34:18 Central Daylight Time
     Storage: 1134300 logical bytes, 1134300 allocated bytes, 100.00% utilization
     Type: shared-1:1440965 struct {
                    "i" +0 native 32-bit field
                    "j" +4 native 32-bit field
                    "k" +8 native 32-bit field
                } 12 bytes

If you do an h5ls on the type itself, it will show its attributes and the shared-identifier as well:

  h5ls -rv LOD.f5/Charts/triangular/SinglePrecision/Point
Opened "LOD.f5" with sec2 driver.
Charts/triangular/SinglePrecision/Point Type
     Attribute: ChartDomain scalar
         Type: 11-byte null-terminated ASCII string
         Data: "triangular"
     Attribute: F5::CellDimensionality scalar
         Type: native int
     Attribute: MemoryOrder {3}
         Type: native int
     Location: 1:1440965
     Links: 2
     Modified: 2014-09-17 01:34:18 Central Daylight Time
     Type: shared-1:1440965 struct {
                    "i" +0 native 32-bit field
                    "j" +4 native 32-bit field
                    "k" +8 native 32-bit field
                } 12 bytes

Cheers,

       Werner

···

Data: 2
         Data: 1, 0, -1

On 16.11.2014 02:12, Martijn Jasperse wrote:

Hi all,
I'd like to be able to tell in HDFView/h5dump when a dataset has a committed datatype. Ideally I'd like a path to the committed type be provided (a la H5Iget_name), but it would be sufficient to identify it as being a committed type. It would be very useful to me for the debugging tools to display this and not have to use my tools to check their files.

I'm using committed types to implement a type of units handling, and it works great, except that the standard tools are not very helpful in working with them. It would be excellent to add this feature to HDFView and/or h5dump, but I would also be interested in other approaches and/or workarounds.

Cheers,
Martijn

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

Hi Martijn,
  In h5dump, I see that datasets that use a committed types have the path to the type identified. E.g.:

DATASET "dset1" {

      DATATYPE "/dtype1"

      DATASPACE SIMPLE { ( 10 ) / ( 10 ) }

      DATA {

      (0): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

      }

}

DATATYPE "dtype1" H5T_IEEE_F32LE;

Is that not sufficient to see how the committed type is used?

In HDFView, the committed types are shown in the Tree View, but dataset's that reference them don't indicate this. So that would be a good feature to add. I've created an issue to track this request.

Regards,

John Readey

The HDF Group

···

From: Werner Benger <werner@cct.lsu.edu<mailto:werner@cct.lsu.edu>>
Reply-To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Sunday, November 16, 2014 at 6:41 AM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] Committed datatypes in HDFView/h5dump

Hi Martijn,

if you use h5ls, then it will indicate named data types as being "shared" such as:

h5ls -rv LOD.f5/t=000000000.0000000000/Triangulation/Faces/Points/Positions
Opened "LOD.f5" with sec2 driver.
t=000000000.0000000000/Triangulation/Faces/Points/Positions Dataset {94525/94525}
    Location: 1:1442029
    Links: 1
    Modified: 2014-09-17 01:34:18 Central Daylight Time
    Storage: 1134300 logical bytes, 1134300 allocated bytes, 100.00% utilization
    Type: shared-1:1440965 struct {
                   "i" +0 native 32-bit field
                   "j" +4 native 32-bit field
                   "k" +8 native 32-bit field
               } 12 bytes

If you do an h5ls on the type itself, it will show its attributes and the shared-identifier as well:

h5ls -rv LOD.f5/Charts/triangular/SinglePrecision/Point
Opened "LOD.f5" with sec2 driver.
Charts/triangular/SinglePrecision/Point Type
    Attribute: ChartDomain scalar
        Type: 11-byte null-terminated ASCII string
        Data: "triangular"
    Attribute: F5::CellDimensionality scalar
        Type: native int
        Data: 2
    Attribute: MemoryOrder {3}
        Type: native int
        Data: 1, 0, -1
    Location: 1:1440965
    Links: 2
    Modified: 2014-09-17 01:34:18 Central Daylight Time
    Type: shared-1:1440965 struct {
                   "i" +0 native 32-bit field
                   "j" +4 native 32-bit field
                   "k" +8 native 32-bit field
               } 12 bytes

Cheers,

      Werner

On 16.11.2014 02:12, Martijn Jasperse wrote:
Hi all,
I'd like to be able to tell in HDFView/h5dump when a dataset has a committed datatype. Ideally I'd like a path to the committed type be provided (a la H5Iget_name), but it would be sufficient to identify it as being a committed type. It would be very useful to me for the debugging tools to display this and not have to use my tools to check their files.

I'm using committed types to implement a type of units handling, and it works great, except that the standard tools are not very helpful in working with them. It would be excellent to add this feature to HDFView and/or h5dump, but I would also be interested in other approaches and/or workarounds.

Cheers,
Martijn

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

Hi John,
Thanks for the reply and making the feature request for HDFView. You and
Werner are correct that the dump/ls tools do produce information about
committed types - and this is the debug information I was after - unless
they are part of a H5T_COMPOUND.

In the case of a compound, only raw information about the datatype is
presented. It would be good if it were able to present the location of
committed types in the same way, e.g.

   DATASET "test" {

      DATATYPE H5T_COMPOUND {
         H5T_IEEE_F64LE "a";
         H5T_IEEE_F64LE "b";
         H5T_IEEE_F64LE "c";
      }
      ...

becomes

   DATASET "test" {

      DATATYPE H5T_COMPOUND {
         "/unit1" "a";
         "/unit2" "b";
         "/unit3" "c";
      }
      ...

or similar.

Cheers,
Martijn

···

On 9 December 2014 at 18:13, John Readey <jreadey@hdfgroup.org> wrote:

Hi Martijn,
  In h5dump, I see that datasets that use a committed types have the path
to the type identified. E.g.:

DATASET "dset1" {

      DATATYPE "/dtype1"

      DATASPACE SIMPLE { ( 10 ) / ( 10 ) }

      DATA {

      (0): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

      }

}

DATATYPE "dtype1" H5T_IEEE_F32LE;

Is that not sufficient to see how the committed type is used?

In HDFView, the committed types are shown in the Tree View, but
dataset’s that reference them don’t indicate this. So that would be a good
feature to add. I’ve created an issue to track this request.

Regards,

John Readey

The HDF Group

  From: Werner Benger <werner@cct.lsu.edu>
Reply-To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Date: Sunday, November 16, 2014 at 6:41 AM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] Committed datatypes in HDFView/h5dump

  Hi Martijn,

if you use h5ls, then it will indicate named data types as being "shared"
such as:

h5ls -rv
LOD.f5/t=000000000.0000000000/Triangulation/Faces/Points/Positions
Opened "LOD.f5" with sec2 driver.
t=000000000.0000000000/Triangulation/Faces/Points/Positions Dataset
{94525/94525}
    Location: 1:1442029
    Links: 1
    Modified: 2014-09-17 01:34:18 Central Daylight Time
    Storage: 1134300 logical bytes, 1134300 allocated bytes, 100.00%
utilization
    Type: shared-1:1440965 struct {
                   "i" +0 native 32-bit field
                   "j" +4 native 32-bit field
                   "k" +8 native 32-bit field
               } 12 bytes

If you do an h5ls on the type itself, it will show its attributes and the
shared-identifier as well:

h5ls -rv LOD.f5/Charts/triangular/SinglePrecision/Point
Opened "LOD.f5" with sec2 driver.
Charts/triangular/SinglePrecision/Point Type
    Attribute: ChartDomain scalar
        Type: 11-byte null-terminated ASCII string
        Data: "triangular"
    Attribute: F5::CellDimensionality scalar
        Type: native int
        Data: 2
    Attribute: MemoryOrder {3}
        Type: native int
        Data: 1, 0, -1
    Location: 1:1440965
    Links: 2
    Modified: 2014-09-17 01:34:18 Central Daylight Time
    Type: shared-1:1440965 struct {
                   "i" +0 native 32-bit field
                   "j" +4 native 32-bit field
                   "k" +8 native 32-bit field
               } 12 bytes

Cheers,

      Werner

On 16.11.2014 02:12, Martijn Jasperse wrote:

Hi all,
I'd like to be able to tell in HDFView/h5dump when a dataset has a
committed datatype. Ideally I'd like a path to the committed type be
provided (a la H5Iget_name), but it would be sufficient to identify it as
being a committed type. It would be very useful to me for the debugging
tools to display this and not have to use my tools to check their files.

I'm using committed types to implement a type of units handling, and it
works great, except that the standard tools are not very helpful in working
with them. It would be excellent to add this feature to HDFView and/or
h5dump, but I would also be interested in other approaches and/or
workarounds.

Cheers,
Martijn

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.orghttp://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org

http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5