H5TB and H5PT support in HDF5DotNet

Hdf-forum archives
1

Hi, Expert,

I am new to HDF5.

I noticed that H5TB and H5PT two APIs were not supported in HDF5DotNet .

Is there a plan to support them in HDF5DotNet ?

What is the best dataset to support real time 10HZ acquisition with HDF5
technology?

Thanks for any help!

JC

···

______________________________________________________
Jianxin Chen

2

Hi Jianxin, how are you? There’s currently no plan
to support H5TB or H5PT in HDF5DotNet.
The “best” dataset configuration for your acquisition
problem depends on several factors, most of which only you know.
What’re the attributes of the of the events you are trying
to record and what’s their size? Are they the same across
your event stream or do they vary from event to event?
Do you plan on applying some form of compression?
What’s your buffering strategy to deal with rate fluctuations?
How will the data be used or processed?

Before spending too much time worrying about performance,
you should make sure that your layout is easy to
use downstream. You can do prototyping and generate sample
data in Python or MATLAB. Once you have that settled,
you can tune and revise it for performance.

Best, G.

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Jianxin Chen
Sent: Tuesday, May 26, 2015 9:13 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi, Expert,

I am new to HDF5.

I noticed that H5TB and H5PT two APIs were not supported in HDF5DotNet .

Is there a plan to support them in HDF5DotNet ?

What is the best dataset to support real time 10HZ acquisition with HDF5 technology?

Thanks for any help!

JC
______________________________________________________
Jianxin Chen

3

Hi, Gerd,

Thanks for your quick response.

I will write some test code first on my acquisition data.

JC

···

______________________________________________________
Jianxin Chen
832-512-6788 (C)

On Tue, May 26, 2015 at 11:11 AM, Gerd Heber <gheber@hdfgroup.org> wrote:

Hi Jianxin, how are you? There’s currently no plan

to support H5TB or H5PT in HDF5DotNet.

The “best” dataset configuration for your acquisition

problem depends on several factors, most of which only you know.

What’re the attributes of the of the events you are trying

to record and what’s their size? Are they the same across

your event stream or do they vary from event to event?

Do you plan on applying some form of compression?

What’s your buffering strategy to deal with rate fluctuations?

How will the data be used or processed?

Before spending too much time worrying about performance,

you should make sure that your layout is easy to

use downstream. You can do prototyping and generate sample

data in Python or MATLAB. Once you have that settled,

you can tune and revise it for performance.

Best, G.

*From:* Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] *On
Behalf Of *Jianxin Chen
*Sent:* Tuesday, May 26, 2015 9:13 AM
*To:* hdf-forum@lists.hdfgroup.org
*Subject:* [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi, Expert,

I am new to HDF5.

I noticed that H5TB and H5PT two APIs were not supported in HDF5DotNet .

Is there a plan to support them in HDF5DotNet ?

What is the best dataset to support real time 10HZ acquisition with HDF5
technology?

Thanks for any help!

JC

______________________________________________________

Jianxin Chen

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

4

I haven't done anything with H5TB, but PT is relatively easy to wrap using the existing examples.

···

________________________________________
From: Hdf-forum [hdf-forum-bounces@lists.hdfgroup.org] on behalf of Gerd Heber [gheber@hdfgroup.org]
Sent: Tuesday, May 26, 2015 12:11 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi Jianxin, how are you? There’s currently no plan
to support H5TB or H5PT in HDF5DotNet.
The “best” dataset configuration for your acquisition
problem depends on several factors, most of which only you know.
What’re the attributes of the of the events you are trying
to record and what’s their size? Are they the same across
your event stream or do they vary from event to event?
Do you plan on applying some form of compression?
What’s your buffering strategy to deal with rate fluctuations?
How will the data be used or processed?

Before spending too much time worrying about performance,
you should make sure that your layout is easy to
use downstream. You can do prototyping and generate sample
data in Python or MATLAB. Once you have that settled,
you can tune and revise it for performance.

Best, G.

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Jianxin Chen
Sent: Tuesday, May 26, 2015 9:13 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi, Expert,

I am new to HDF5.

I noticed that H5TB and H5PT two APIs were not supported in HDF5DotNet .

Is there a plan to support them in HDF5DotNet ?

What is the best dataset to support real time 10HZ acquisition with HDF5 technology?

Thanks for any help!

JC
______________________________________________________
Jianxin Chen

________________________________

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.

5

I use PT in C++ at higher frequencies than mentioned (20-30hz). It's worth
noting it builds on TB and optimizes some parts so it's more suitable to
online-appends which you encounter in acquisition but otherwise it's the
same foundation. You didn't mention what you were logging at 10hz so you
have to figure that in... I wouldn't try recording high resolution video to
HDF at that frequency but maybe it'll work for some people and at some
resolution it will be acceptable. Make check sure your filter pipeline is
nil for real-time purposes no time spent converting and compressing
online. Also a quick note that make sure the system doesn't get shut down
abruptly - HDF5 does not deal with power loss or crashes very well at the
moment but it's something they're working on...

As an alternative I would suggest just dumping to a binary file with all
the metadata you need for ever sample. Then just write a python/H5py
script to convert that to HDF5 [just] after acquisition - this lets you be
very efficient on disk IO, more deterministic performance, potentially log
to disk in a way more robust to to power failure/crash, and in your case
not have to jump through hoops for the PT.

-Jason

···

On Tue, May 26, 2015 at 2:19 PM, Mitchell, Scott - Exelis < Scott.Mitchell@exelisinc.com> wrote:

I haven't done anything with H5TB, but PT is relatively easy to wrap using
the existing examples.
________________________________________
From: Hdf-forum [hdf-forum-bounces@lists.hdfgroup.org] on behalf of Gerd
Heber [gheber@hdfgroup.org]
Sent: Tuesday, May 26, 2015 12:11 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi Jianxin, how are you? There’s currently no plan
to support H5TB or H5PT in HDF5DotNet.
The “best” dataset configuration for your acquisition
problem depends on several factors, most of which only you know.
What’re the attributes of the of the events you are trying
to record and what’s their size? Are they the same across
your event stream or do they vary from event to event?
Do you plan on applying some form of compression?
What’s your buffering strategy to deal with rate fluctuations?
How will the data be used or processed?

Before spending too much time worrying about performance,
you should make sure that your layout is easy to
use downstream. You can do prototyping and generate sample
data in Python or MATLAB. Once you have that settled,
you can tune and revise it for performance.

Best, G.

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
Of Jianxin Chen
Sent: Tuesday, May 26, 2015 9:13 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] H5TB and H5PT support in HDF5DotNet

Hi, Expert,

I am new to HDF5.

I noticed that H5TB and H5PT two APIs were not supported in HDF5DotNet .

Is there a plan to support them in HDF5DotNet ?

What is the best dataset to support real time 10HZ acquisition with HDF5
technology?

Thanks for any help!

JC
______________________________________________________
Jianxin Chen

________________________________

This e-mail and any files transmitted with it may be proprietary and are
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this e-mail in error please notify the
sender. Please note that any views or opinions presented in this e-mail are
solely those of the author and do not necessarily represent those of Exelis
Inc. The recipient should check this e-mail and any attachments for the
presence of viruses. Exelis Inc. accepts no liability for any damage caused
by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5