----------------------------------------------------------------------
Message: 1
Date: Mon, 4 Feb 2013 10:00:25 -0600
From: "H. Joe Lee" <hyoklee@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: [Hdf-forum] Open Source Hadoop Project for both HDF4 and
HDF5?
Message-ID:
<CADeM3-y2j25ctf=NogU-exFFzbHW0hn5P_O4ULm75Kaec_0+Xw@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1I'm wondering if there has been any open source software project for
Hadoop that can work with *both* HDF4 and HDF5 directly as described
in SciMATE[1] (i.e., HDF5 Adapter in Figure 2 from the paper) I think
[2] is close but it seems to be only for NetCDF.p.s. I'm not looking for a solution that involves exporting data
into XML from HDF.[1] http://www.cse.ohio-state.edu/~wayi/papers/SciMATE.pdf
[2] https://github.com/four2five/SciHadoop--
HDF: Software that Powers Science
I'm a member of the lab that did the SciHadoop work and I wanted to mention that we're working on adding HDF5 support (along with a few other formats) but this requires using a distributed file system that supports mutable data (HDFS is append-only, which is problematic for many scientific file formats). Once we have something working in the lab, I'll post to this mailing list with the details and a path to the code.
Best,
-Joe Buck