HDF Project for storing Time-Series storage of Physiological Waveforms - ECG etc

Hdf-forum archives
1

Hello Everyone,
I had posted earlier ago requesting any information on HDF based projects that were developed for Time-Series based storage of Physiological waveforms such as ECG etc.

I would really appreciate your help in this matter.

Thanks
Tim

Taimoore Rajah (Tim)
Biomedical Engineer
POIS-SIS IT Support Group|DPM IT & Medical Device Integration, HW & SW Support
Department of Perioperative Medicine (DPM)
National Institute of Health, Clinical Center
9000 Rockville Pike, MSC-1512
Building 10, Room 2C563
Bethesda, MD 20892-1512
Tel: O (301) 827-9402
Fax: (301) 480-1699
[cid:image001.jpg@01CE56E6.8FCBD490]

2

Hi Tim,

I was curious about the topic too. From memory (earlier searches, a few years
ago), the was no standard for time series data.

By googling today, I find a few results

TsTables
http://andyfiedler.com/projects/tstables-store-high-frequency-data-with-pytables/

it is aimed at financial data series and is based on Pandas and PyTables. So, it
might no be very useful as a general purpose (i.e. usable by other software than
Python).

Modelica Association Time Series File Format

Has been designed explicitly for generic time series data in the framework of
https://modelica.org/

NSDF
http://nsdf.readthedocs.org/en/latest/nsdf.html

Neuroscience Simulation Data Format, it seems a bit simpler in its definition

It is just a bunch of links, but I hope that it helps.

Regards,

Pierre

···

On Wed, Apr 08, 2015 at 01:24:50PM +0000, Rajah, Tim (NIH/CC/SURG) [E] wrote:

Hello Everyone,
I had posted earlier ago requesting any information on HDF based projects that were developed for Time-Series based storage of Physiological waveforms such as ECG etc.

I would really appreciate your help in this matter.

Thanks
Tim

3

I do quite a bit of heterogeneous sensor data in time series. A common time format would be nice. I ended up using Windows .NET DateTime.Ticks. Users complained that it was unreadable (shocking) and I had to add a parallel text string. Some of the sensors provide their own timestamp typically in Unix Time (plus milliseconds).

The timestamps are in dimension scales to the actual dataset. Dimension scales seem to be relatively unsupported... not by HDFView anyway. So they end up as a secondary, independent dataset. It works in my code though.

Scott

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Pierre de Buyl
Sent: Wednesday, April 08, 2015 3:27 PM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] HDF Project for storing Time-Series storage of Physiological Waveforms - ECG etc

Hi Tim,

I was curious about the topic too. From memory (earlier searches, a few years ago), the was no standard for time series data.

By googling today, I find a few results

TsTables
http://andyfiedler.com/projects/tstables-store-high-frequency-data-with-pytables/

it is aimed at financial data series and is based on Pandas and PyTables. So, it might no be very useful as a general purpose (i.e. usable by other software than Python).

Modelica Association Time Series File Format http://www.ep.liu.se/ecp/076/050/ecp12076050.pdf

Has been designed explicitly for generic time series data in the framework of https://modelica.org/

NSDF
http://nsdf.readthedocs.org/en/latest/nsdf.html

Neuroscience Simulation Data Format, it seems a bit simpler in its definition Frontiers | Neuroscience Simulation Data Format (NSDF) : HDF-based format for large simulation datasets

It is just a bunch of links, but I hope that it helps.

Regards,

Pierre

On Wed, Apr 08, 2015 at 01:24:50PM +0000, Rajah, Tim (NIH/CC/SURG) [E] wrote:

Hello Everyone,
I had posted earlier ago requesting any information on HDF based projects that were developed for Time-Series based storage of Physiological waveforms such as ECG etc.

I would really appreciate your help in this matter.

Thanks
Tim

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

________________________________

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.

4

Scott

The timestamps are in dimension scales to the actual dataset. Dimension scales seem to be relatively unsupported... not by HDFView anyway.

HDF Explorer supports dimension scales. To see an example, please go to

http://www.space-research.org/

go to menu "Online help" and search "HDF5 Dimension scales"

The example shows maps generated for a dataset that has 2 sets of dimension scales attached. Choosing one set over the otther results in a different map.

Direct link:

http://www.space-research.org/help/help.htm

more info

http://ftp.hdfgroup.org/HDF5/Tutor/h5dimscale.html

···

----------------------
Pedro Vicente
pedro.vicente@space-research.org
http://www.space-research.org/

----- Original Message ----- From: "Mitchell, Scott - Exelis" <Scott.Mitchell@exelisinc.com>
To: "HDF Users Discussion List" <hdf-forum@lists.hdfgroup.org>
Sent: Thursday, April 09, 2015 10:05 AM
Subject: Re: [Hdf-forum] HDF Project for storing Time-Series storage of Physiological Waveforms - ECG etc

I do quite a bit of heterogeneous sensor data in time series. A common time format would be nice. I ended up using Windows .NET DateTime.Ticks. Users complained that it was unreadable (shocking) and I had to add a parallel text string. Some of the sensors provide their own timestamp typically in Unix Time (plus milliseconds).

The timestamps are in dimension scales to the actual dataset. Dimension scales seem to be relatively unsupported... not by HDFView anyway. So they end up as a secondary, independent dataset. It works in my code though.

Scott

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Pierre de Buyl
Sent: Wednesday, April 08, 2015 3:27 PM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] HDF Project for storing Time-Series storage of Physiological Waveforms - ECG etc

Hi Tim,

I was curious about the topic too. From memory (earlier searches, a few years ago), the was no standard for time series data.

By googling today, I find a few results

TsTables
http://andyfiedler.com/projects/tstables-store-high-frequency-data-with-pytables/

it is aimed at financial data series and is based on Pandas and PyTables. So, it might no be very useful as a general purpose (i.e. usable by other software than Python).

Modelica Association Time Series File Format http://www.ep.liu.se/ecp/076/050/ecp12076050.pdf

Has been designed explicitly for generic time series data in the framework of https://modelica.org/

NSDF
http://nsdf.readthedocs.org/en/latest/nsdf.html

Neuroscience Simulation Data Format, it seems a bit simpler in its definition http://www.frontiersin.org/10.3389/conf.fninf.2014.18.00026/event_abstract

It is just a bunch of links, but I hope that it helps.

Regards,

Pierre

On Wed, Apr 08, 2015 at 01:24:50PM +0000, Rajah, Tim (NIH/CC/SURG) [E] > wrote:

Hello Everyone,
I had posted earlier ago requesting any information on HDF based projects that were developed for Time-Series based storage of Physiological waveforms such as ECG etc.

I would really appreciate your help in this matter.

Thanks
Tim

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

________________________________

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5