Hi,
What is a good way to store arrays of complex numbers?
Particularly MKL (Intel^® Math Kernel Library) has a data type MKL_Complex16 which is two doubles side by side and I make very large arrays of them.
What is a good way to store arrays of complex numbers?
Particularly MKL (Intel® Math Kernel Library) has a data type MKL_Complex16
which is two doubles side by side and I make very large arrays of them.
I do not know that it is The Right Way (TM) to do it, but I have been
using H5Tarray_create to create a datatype consisting of two floating
point types. The HDF5 user guide shows another option
(http://www.hdfgroup.org/HDF5/doc/UG/UG_frame11Datatypes.html\).
- Rhys
Hi,
to read/write with complex numbers you can create a new datatype, to
be used in the dataset read/write function.
franZ
typedef struct {
double re; /*real part*/
double im; /*imaginary part*/
} complex_t;
hid_t complex_id = H5Tcreate (H5T_COMPOUND, sizeof (complex_t));
H5Tinsert (complex_id, “real”, HOFFSET(complex_t,re),
H5T_NATIVE_DOUBLE);
H5Tinsert (complex_id, “imaginary”, HOFFSET(complex_t,im),
H5T_NATIVE_DOUBLE);
···
On Fri, Dec 10, 2010 at 10:39 AM, Roger Martin <roger@quantumbioinc.com> wrote:
Hi,
What is a good way to store arrays of complex numbers?
Particularly MKL (Intel® Math Kernel Library) has a data type MKL_Complex16
which is two doubles side by side and I make very large arrays of them._______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
A Friday 10 December 2010 16:39:48 Roger Martin escrigué:
Hi,
What is a good way to store arrays of complex numbers?
There is not a standard way (it would be nice if there was one). The
usual thing is to save them as a compound datatype, but people choose
different options for the names of the fields. Here are what I know:
* ("real", "imag") for Octave
* ("r", "i") for PyTables (but ("real", "imag") is ok too)
Particularly MKL (Intel^® Math Kernel Library) has a data type
MKL_Complex16 which is two doubles side by side and I make very large
arrays of them.
May be there is a possibility to define an atomic datatype in HDF5, but
provided that you don't introduce padding between the fields of the
compound type, this works pretty similar to a larger, atomic type.
Hope this helps,
···
--
Francesc Alted
Just curious...A complex number is sort of native to the Fortran
language. Are we saying there is no 'native' support in HDF5 for
Fortran's complex type?
Mark
···
On Fri, 2010-12-10 at 07:56, Francesc Alted wrote:
A Friday 10 December 2010 16:39:48 Roger Martin escrigu�:
> Hi,
>
> What is a good way to store arrays of complex numbers?There is not a standard way (it would be nice if there was one). The
usual thing is to save them as a compound datatype, but people choose
different options for the names of the fields. Here are what I know:* ("real", "imag") for Octave
* ("r", "i") for PyTables (but ("real", "imag") is ok too)> Particularly MKL (Intel^� Math Kernel Library) has a data type
> MKL_Complex16 which is two doubles side by side and I make very large
> arrays of them.May be there is a possibility to define an atomic datatype in HDF5, but
provided that you don't introduce padding between the fields of the
compound type, this works pretty similar to a larger, atomic type.Hope this helps,
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
Ok, thanks for that clarification. Didn't know that.
If I recall, I wasn't sure if I saw complex number support planned in
1.10 or 1.11? You say here '1.10.*' which sounds potentially like a
minor release AFTER 1.10 release which presumably extends into 2011?
In the interim, sounds like potentially a lot of other users are
frequently hit by this. Why not add a helper method to define a defacto
type for it to HDF5 just as many of these other users are currently
doing ABOVE HDF5? That way, they all don't wind up essentially
duplicating the same effort?
Mark
···
On Fri, 2010-12-10 at 12:21, Elena Pourmal wrote:
Yes, at this point there is no support for complex type in Fortran and in C. We are planning to add it in 1.10.* along with other C99 types that currently are not supported.
Elena
On Dec 10, 2010, at 12:06 PM, Mark Miller wrote:> Just curious...A complex number is sort of native to the Fortran
> language. Are we saying there is no 'native' support in HDF5 for
> Fortran's complex type?
>
> Mark
>
> On Fri, 2010-12-10 at 07:56, Francesc Alted wrote:
>> A Friday 10 December 2010 16:39:48 Roger Martin escrigu�:
>>> Hi,
>>>
>>> What is a good way to store arrays of complex numbers?
>>
>> There is not a standard way (it would be nice if there was one). The
>> usual thing is to save them as a compound datatype, but people choose
>> different options for the names of the fields. Here are what I know:
>>
>> * ("real", "imag") for Octave
>> * ("r", "i") for PyTables (but ("real", "imag") is ok too)
>>
>>> Particularly MKL (Intel^� Math Kernel Library) has a data type
>>> MKL_Complex16 which is two doubles side by side and I make very large
>>> arrays of them.
>>
>> May be there is a possibility to define an atomic datatype in HDF5, but
>> provided that you don't introduce padding between the fields of the
>> compound type, this works pretty similar to a larger, atomic type.
>>
>> Hope this helps,
> --
> Mark C. Miller, Lawrence Livermore National Laboratory
> ================!!LLNL BUSINESS ONLY!!================
> miller86@llnl.gov urgent: miller86@pager.llnl.gov
> T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
Sorry, I meant '2012' not '2011'
Mark
···
On Fri, 2010-12-10 at 13:02, Mark Miller wrote:
Ok, thanks for that clarification. Didn't know that.
If I recall, I wasn't sure if I saw complex number support planned in
1.10 or 1.11? You say here '1.10.*' which sounds potentially like a
minor release AFTER 1.10 release which presumably extends into 2011?In the interim, sounds like potentially a lot of other users are
frequently hit by this. Why not add a helper method to define a defacto
type for it to HDF5 just as many of these other users are currently
doing ABOVE HDF5? That way, they all don't wind up essentially
duplicating the same effort?Mark
On Fri, 2010-12-10 at 12:21, Elena Pourmal wrote:
> Yes, at this point there is no support for complex type in Fortran and in C. We are planning to add it in 1.10.* along with other C99 types that currently are not supported.
>
> Elena
> On Dec 10, 2010, at 12:06 PM, Mark Miller wrote:
>
> > Just curious...A complex number is sort of native to the Fortran
> > language. Are we saying there is no 'native' support in HDF5 for
> > Fortran's complex type?
> >
> > Mark
> >
> > On Fri, 2010-12-10 at 07:56, Francesc Alted wrote:
> >> A Friday 10 December 2010 16:39:48 Roger Martin escrigu�:
> >>> Hi,
> >>>
> >>> What is a good way to store arrays of complex numbers?
> >>
> >> There is not a standard way (it would be nice if there was one). The
> >> usual thing is to save them as a compound datatype, but people choose
> >> different options for the names of the fields. Here are what I know:
> >>
> >> * ("real", "imag") for Octave
> >> * ("r", "i") for PyTables (but ("real", "imag") is ok too)
> >>
> >>> Particularly MKL (Intel^� Math Kernel Library) has a data type
> >>> MKL_Complex16 which is two doubles side by side and I make very large
> >>> arrays of them.
> >>
> >> May be there is a possibility to define an atomic datatype in HDF5, but
> >> provided that you don't introduce padding between the fields of the
> >> compound type, this works pretty similar to a larger, atomic type.
> >>
> >> Hope this helps,
> > --
> > Mark C. Miller, Lawrence Livermore National Laboratory
> > ================!!LLNL BUSINESS ONLY!!================
> > miller86@llnl.gov urgent: miller86@pager.llnl.gov
> > T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
> >
> >
> > _______________________________________________
> > Hdf-forum is for HDF software users discussion.
> > Hdf-forum@hdfgroup.org
> > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
Yes, at this point there is no support for complex type in Fortran and in C. We are planning to add it in 1.10.* along with other C99 types that currently are not supported.
Elena
···
On Dec 10, 2010, at 12:06 PM, Mark Miller wrote:
Just curious...A complex number is sort of native to the Fortran
language. Are we saying there is no 'native' support in HDF5 for
Fortran's complex type?Mark
On Fri, 2010-12-10 at 07:56, Francesc Alted wrote:
A Friday 10 December 2010 16:39:48 Roger Martin escrigué:
Hi,
What is a good way to store arrays of complex numbers?
There is not a standard way (it would be nice if there was one). The
usual thing is to save them as a compound datatype, but people choose
different options for the names of the fields. Here are what I know:* ("real", "imag") for Octave
* ("r", "i") for PyTables (but ("real", "imag") is ok too)Particularly MKL (Intel^® Math Kernel Library) has a data type
MKL_Complex16 which is two doubles side by side and I make very large
arrays of them.May be there is a possibility to define an atomic datatype in HDF5, but
provided that you don't introduce padding between the fields of the
compound type, this works pretty similar to a larger, atomic type.Hope this helps,
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Why not add a helper method to define a defacto
type for it to HDF5 just as many of these other users are currently
doing ABOVE HDF5? That way, they all don't wind up essentially
duplicating the same effort?
To be fair, because there's no one-size-fits-all complex number
solution outside Fortran, a one-size-fits-all cross-language HDF5
helper method to define a complex type would be difficult to get
right. Not technically difficult, just socially difficult.
- Rhys
Agree.
···
On Dec 10, 2010, at 4:26 PM, Rhys Ulerich wrote:
Why not add a helper method to define a defacto
type for it to HDF5 just as many of these other users are currently
doing ABOVE HDF5? That way, they all don't wind up essentially
duplicating the same effort?To be fair, because there's no one-size-fits-all complex number
solution outside Fortran, a one-size-fits-all cross-language HDF5
helper method to define a complex type would be difficult to get
right. Not technically difficult, just socially difficult.- Rhys
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Ok, thanks for that clarification. Didn't know that.
If I recall, I wasn't sure if I saw complex number support planned in
1.10 or 1.11? You say here '1.10.*' which sounds potentially like a
minor release AFTER 1.10 release which presumably extends into 2011?
Many users asked to support complex and boolean types in both HDF5 C and Fortran. We hope to implement at least those two types for 1.10.0 and add more types as we go.
We definitely do not want to wait until 2012 
Elena
···
On Dec 10, 2010, at 3:02 PM, Mark Miller wrote:
In the interim, sounds like potentially a lot of other users are
frequently hit by this. Why not add a helper method to define a defacto
type for it to HDF5 just as many of these other users are currently
doing ABOVE HDF5? That way, they all don't wind up essentially
duplicating the same effort?Mark
On Fri, 2010-12-10 at 12:21, Elena Pourmal wrote:
Yes, at this point there is no support for complex type in Fortran and in C. We are planning to add it in 1.10.* along with other C99 types that currently are not supported.
Elena
On Dec 10, 2010, at 12:06 PM, Mark Miller wrote:Just curious...A complex number is sort of native to the Fortran
language. Are we saying there is no 'native' support in HDF5 for
Fortran's complex type?Mark
On Fri, 2010-12-10 at 07:56, Francesc Alted wrote:
A Friday 10 December 2010 16:39:48 Roger Martin escrigué:
Hi,
What is a good way to store arrays of complex numbers?
There is not a standard way (it would be nice if there was one). The
usual thing is to save them as a compound datatype, but people choose
different options for the names of the fields. Here are what I know:* ("real", "imag") for Octave
* ("r", "i") for PyTables (but ("real", "imag") is ok too)Particularly MKL (Intel^® Math Kernel Library) has a data type
MKL_Complex16 which is two doubles side by side and I make very large
arrays of them.May be there is a possibility to define an atomic datatype in HDF5, but
provided that you don't introduce padding between the fields of the
compound type, this works pretty similar to a larger, atomic type.Hope this helps,
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org