hdf work with hadoop map reduce

Hdf-forum archives
1

Hi all,
Is there a pure java HDF file reader impletation? We are working on hadoop distributed file system, and only it's specific api
can access HDF file stores in it (which cannot be read by hdf native library).
So, i need a pure java HDF file reader. Thanks!

2

For pure HDF5 Java reader check NetCDF-Java library

http://www.unidata.ucar.edu/software/netcdf-java/

Elena

···

On Jan 12, 2010, at 8:56 AM, shensl wrote:

Hi all,
Is there a pure java HDF file reader impletation? We are working on hadoop distributed file system, and only it's specific api
can access HDF file stores in it (which cannot be read by hdf native library).
So, i need a pure java HDF file reader. Thanks!

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

3

Thanks Elena,
I checked out NetCDF-Java library, but i found it also depends on HDF native library.

Our problem is that we have about 2000T HDF files. we have to find a distributed computing platform to run our application. Is there anyone worked with Hadoop map/reduce framework with HDF files before ? Thanks.

在2010-01-12?23:30:43，"Elena?Pourmal"?<epourmal@hdfgroup.org>?写道：

···

For?pure?HDF5?Java?reader?check?NetCDF-Java?library

http://www.unidata.ucar.edu/software/netcdf-java/

Elena
On?Jan?12,?2010,?at?8:56?AM,?shensl?wrote:

?Hi?all,
?Is?there?a?pure?java?HDF?file?reader?impletation???We?are?working?on?hadoop?distributed?file?system,?and?only?it's?specific?api
?can?access?HDF?file?stores?in?it?(which?cannot?be?read?by?hdf?native?library).?
?So,?i?need?a?pure?java?HDF?file?reader.??Thanks!
?
?
?_______________________________________________
?Hdf-forum?is?for?HDF?software?users?discussion.
?Hdf-forum@hdfgroup.org
?http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum?is?for?HDF?software?users?discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

4

Sorry, I am probably missing something.

Thanks Elena,
I checked out NetCDF-Java library, but i found it also depends on HDF native library.

Could you please point me to this information (I hope we are talking about HDF5 and not about HDF4, but it may be a wrong assumption :wink:
To my knowledge NetCDF HDF5 Java reader doesn't use HDF5 library (compare with the HDF-Java library and tools that come from our group).

here is the info from the UCAR's netcdf-java web site

Software libraries used by the Netcdf-Java library:

Apache Commons HttpClient for HTTP access, which also use commons-logging and commons-codec.
Java-OPeNDAP library for reading OPeNDAP.
JDOM for parsing XML.
JGoodies for UI stuff
JPEG2000 library for decoding GRIB2 files.
Simple Logging Facade for Java (SLF4J) for logging
Spring lightweight application framework. Used in ToolsUI application.
VisAD for reading McIDAS AREA files.

Thank you!

Elena

···

On Jan 13, 2010, at 10:23 AM, shensl wrote:

Our problem is that we have about 2000T HDF files. we have to find a distributed computing platform to run our application. Is there anyone worked with Hadoop map/reduce framework with HDF files before ? Thanks.
在2010-01-12 23:30:43，"Elena Pourmal" <epourmal@hdfgroup.org> 写道：
>For pure HDF5 Java reader check NetCDF-Java library
>
>http://www.unidata.ucar.edu/software/netcdf-java/
>
>Elena
>On Jan 12, 2010, at 8:56 AM, shensl wrote:
>
>> Hi all,
>> Is there a pure java HDF file reader impletation? We are working on hadoop distributed file system, and only it's specific api
>> can access HDF file stores in it (which cannot be read by hdf native library).
>> So, i need a pure java HDF file reader. Thanks!
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> Hdf-forum@hdfgroup.org
>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
>_______________________________________________
>Hdf-forum is for HDF software users discussion.
>Hdf-forum@hdfgroup.org
>http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org