Open Source Hadoop Project for both HDF4 and HDF5?


Message: 1
Date: Mon, 4 Feb 2013 10:00:25 -0600
From: "H. Joe Lee" <>
To: HDF Users Discussion List <>
Subject: [Hdf-forum] Open Source Hadoop Project for both HDF4 and
Content-Type: text/plain; charset=ISO-8859-1

I'm wondering if there has been any open source software project for
Hadoop that can work with *both* HDF4 and HDF5 directly as described
in SciMATE[1] (i.e., HDF5 Adapter in Figure 2 from the paper) I think
[2] is close but it seems to be only for NetCDF.

p.s. I'm not looking for a solution that involves exporting data
into XML from HDF.


HDF: Software that Powers Science

I'm a member of the lab that did the SciHadoop work and I wanted to mention that we're working on adding HDF5 support (along with a few other formats) but this requires using a distributed file system that supports mutable data (HDFS is append-only, which is problematic for many scientific file formats). Once we have something working in the lab, I'll post to this mailing list with the details and a path to the code.

-Joe Buck