I have some data, that was created by a C program and thus all data is in rowMajor format.
We would like to read this into another C program that would then call LAPACK. Because lapack for C is really a shim to fortran (which expects ColumnMajor) we would like to read the data in transposed.
Is there a simple way to do this? Any tricks? We have some Macros that let us access malloc()'ed memory in fortran style, but I can for the life of me figure out any way to use them with H5Dread() .
Any comments?
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
···
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Just read your data "as is" in row-major format, then call LAPACK with 'T' for transpose in the parameter list
···
On Sep 17, 2008, at 12:35 PM, Brock Palen wrote:
I have some data, that was created by a C program and thus all data is in rowMajor format.
We would like to read this into another C program that would then call LAPACK. Because lapack for C is really a shim to fortran (which expects ColumnMajor) we would like to read the data in transposed.
Is there a simple way to do this? Any tricks? We have some Macros that let us access malloc()'ed memory in fortran style, but I can for the life of me figure out any way to use them with H5Dread() .
Any comments?
drew
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Welcome to the club!
There is no easy way because HDF5 does not care about this issue, which I very
much regret.
I resolve it by always adding an attribute to any arrays rank>1 how it is
stored. Then it is the responsibility of the client code to read it in
correctly.
And transposing in-place is not difficult, only will not allow a const
pointer, which is annoying.
regards,
- Dominik
···
On Wednesday 17 September 2008 09:35:40 pm Brock Palen wrote:
I have some data, that was created by a C program and thus all data
is in rowMajor format.
We would like to read this into another C program that would then
call LAPACK. Because lapack for C is really a shim to fortran (which
expects ColumnMajor) we would like to read the data in transposed.
Is there a simple way to do this? Any tricks? We have some Macros
that let us access malloc()'ed memory in fortran style, but I can for
the life of me figure out any way to use them with H5Dread() .
Any comments?
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org. To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Welcome to the club!
There is no easy way because HDF5 does not care about this issue, which I very
much regret.
Well, actually we do care about implementing this feature in HDF5, but it has never been a high enough priority by someone with money to get funded.
Quincey
···
On Sep 17, 2008, at 2:54 PM, Dominik Szczerba wrote:
I resolve it by always adding an attribute to any arrays rank>1 how it is
stored. Then it is the responsibility of the client code to read it in
correctly.
And transposing in-place is not difficult, only will not allow a const
pointer, which is annoying.
regards,
- Dominik
On Wednesday 17 September 2008 09:35:40 pm Brock Palen wrote:
I have some data, that was created by a C program and thus all data
is in rowMajor format.
We would like to read this into another C program that would then
call LAPACK. Because lapack for C is really a shim to fortran (which
expects ColumnMajor) we would like to read the data in transposed.
Is there a simple way to do this? Any tricks? We have some Macros
that let us access malloc()'ed memory in fortran style, but I can for
the life of me figure out any way to use them with H5Dread() .
Any comments?
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org. To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Yes it's not that hard. But as a general rule, anything I write to do this in memory would not be as fast as a BLAS1 operation even. For large matrices this has to be a memory bound operation.
Is there a standard way to do in place transpose that is high performance? Is there a way to do it with a blas2 call?
I was hoping that I could do it while reading from disk, because disk would be slower than memory anyway thus rearranging in memory while reading from disk would not be a performance hit.
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
···
On Sep 17, 2008, at 3:54 PM, Dominik Szczerba wrote:
Welcome to the club!
There is no easy way because HDF5 does not care about this issue, which I very
much regret.
I resolve it by always adding an attribute to any arrays rank>1 how it is
stored. Then it is the responsibility of the client code to read it in
correctly.
And transposing in-place is not difficult, only will not allow a const
pointer, which is annoying.
regards,
- Dominik
On Wednesday 17 September 2008 09:35:40 pm Brock Palen wrote:
I have some data, that was created by a C program and thus all data
is in rowMajor format.
We would like to read this into another C program that would then
call LAPACK. Because lapack for C is really a shim to fortran (which
expects ColumnMajor) we would like to read the data in transposed.
Is there a simple way to do this? Any tricks? We have some Macros
that let us access malloc()'ed memory in fortran style, but I can for
the life of me figure out any way to use them with H5Dread() .
Any comments?
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org. To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
I saw that also.
I was looking more for a catch all way for reading data right in for C programmers for use by LAPACK, which has no (if poor) C interface.
macros like this don't work with H5read()
#define A(I,J) a[((J)-1)*lda+(I)-1]
Thanks I will check that out.
Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp@umich.edu
(734)936-1985
···
On Sep 18, 2008, at 2:37 AM, Andrew Cunningham wrote:
Hmm, seems to me we should call in a linear algebra guru....
dgesv is the combination of two calls..wild guess maybe dgetrs with 'T' may work.
// Compute the LU factorization of A.
dgetrf( n, n, a, lda, ipiv, info );
if( info == 0 ) {
// Solve the system A*X = B, overwriting B with X.
dgetrs( 'N'/* No transpose */, n, nrhs, a, lda, ipiv, b, ldb,
info );
}
On Sep 17, 2008, at 6:31 PM, Brock Palen wrote:
The function we were looking at using (dgesv()) does not have a transpose option.
Andrew
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
One solution might be for your users to use a C++ Matrix library that hides the actual implementation of the storage of the data so that while the data is stored column-major ( thus FORTRAN/LAPACK friendly) your C/C++ users see a familiar interface m(r,c)=val
e.g. Boost::uBLAS
···
On Sep 18, 2008, at 6:31 AM, Brock Palen wrote:
I was looking more for a catch all way for reading data right in for C programmers for use by LAPACK, which has no (if poor) C interface.
Andrew
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
I get the following linking error compiling my stuff with Visual Studio 2008
using hdf5 (binaries downloaded from the hdf5 download section). Can someone
point me out in the right direction please?
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_UCHAR_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@@V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_INT_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@@V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_DOUBLE_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@@V?
$allocator@D@2@@std@@@Z)
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
This looks to me like a mismatch of debug/release settings in the
compiler, and/or a mismatch of multi-threaded/single-threaded. Also, I
don't believe that HDF releases binaries built with VS2008, and that may
be another complicating factor. I never attempt to link code built
with different compiler versions, so I don't know if this will work.
I would recommend building the binaries yourself. That way you can
control all of the settings and make sure they match between the HDF5
libraries and your code.
George Lewandowski
(314)777-7890
Mail Code S270-2204
Building 270-E Level 2E Room 20E
P-8A
I get the following linking error compiling my stuff with Visual Studio
2008 using hdf5 (binaries downloaded from the hdf5 download section).
Can someone point me out in the right direction please?
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_UCHAR_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_INT_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_DOUBLE_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
with regards,
Dominik
--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich http://www.vision.ee.ethz.ch/~domi
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
This looks to me like a mismatch of debug/release settings in the
compiler, and/or a mismatch of multi-threaded/single-threaded. Also, I
don't believe that HDF releases binaries built with VS2008, and that may
be another complicating factor. I never attempt to link code built
with different compiler versions, so I don't know if this will work.
I would recommend building the binaries yourself. That way you can
control all of the settings and make sure they match between the HDF5
libraries and your code.
George Lewandowski
(314)777-7890
Mail Code S270-2204
Building 270-E Level 2E Room 20E
P-8A
I get the following linking error compiling my stuff with Visual Studio
2008 using hdf5 (binaries downloaded from the hdf5 download section).
Can someone point me out in the right direction please?
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_UCHAR_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_INT_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
HDF5++.lib(HDF5Reader.obj) : error LNK2019: unresolved external symbol
_H5T_NATIVE_DOUBLE_g referenced in function "private: int __thiscall
HDF5::HDF5Reader::readData(class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > const &)"
(?readData@HDF5Reader@HDF5@@AAEHABV?$basic_string@DU?$char_traits@D@std@ @V?
$allocator@D@2@@std@@@Z)
with regards,
Dominik
--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich http://www.vision.ee.ethz.ch/~domi
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
