Hi,
I have a scenario when HDF5 throws Access Violation when saving to disk with insufficient disk space or when there is network disconnect after H5File has been created.
Environment: Windows 7
Compiler: Visual Studio 2012
Platform: x64
HDF5 version: 1.8.5 (also tested with HDF5 1.8.11)
Language: C++ version of HDF5
Precondition: Insufficient disk space or Network disconnect after H5File has been created.
I was expecting H5::Exception instead of Access Violation in these scenarios. I could check for disk space or network connection check before the operation, but there is a chance that at the time of writing by HDF5, there is not enough disk space or network has been disconnected.
I would like to know if I am missing something or if there is any workaround to avoid access violations.
1. I am looking for a solution/workaround that could avoid access violation during the write operation especially for version 1.8.5.
2. Both 1.8.5 and 1.8.11 throw access violation when unloading HDF5 library during H5close. Is there any way to avoid this access violation?
Here is the test code that I used:
#include <stdio.h>
#include <iostream>
#include <vector>
#include <cpp/H5Cpp.h>
int main(int argc, char* argv[])
{
try {
// Preconditions
// 1. DataSet::write exception - Insufficient disk space or Network disconnect after H5File creation; Data size is greater than CHUNK_CACHE (default 1 MB)
// 2. DataSet::close exception - Insufficient disk space or Network disconnect after H5File creation; Data size is less than CHUNK_CACHE
// Expectation just H5 exceptions, but Actual result is Access Violation in both 1.8.5 and 1.8.11 at different points
// Used DYNAMIC library of HDF5 1.8.5;
// Used STATIC library of HDF5 1.8.11; (so as to not affect deployed 1.8.5)
// Used VS 2012 in Windows 7 environment for this test.
std::string fileName("G:\\output.volcan.h5");
std::cout << "Creating H5File at " << fileName << " ....." << std::endl;
H5::FileCreatPropList create_plist = H5::FileCreatPropList::DEFAULT;
H5::FileAccPropList access_plist = H5::FileAccPropList::DEFAULT;
unsigned int flags = H5F_ACC_TRUNC;
H5::H5File h5File(fileName, flags, create_plist, access_plist);
std::cout << "Created H5File at " << fileName << std::endl;
//Create DataSet
std::string datasetName("FloatData");
std::cout << "Creating float datatype DataSet " << datasetName << " ....." << std::endl;
H5::FloatType datatype(H5::PredType::NATIVE_FLOAT);
datatype.setOrder(H5T_ORDER_LE);
const unsigned long long count = 10000000; //100;
hsize_t dimsf[1]; // dataset dimensions
dimsf[0] = count;
hsize_t maxdims[1] = {H5S_UNLIMITED};
H5::DataSpace dataspace(1, dimsf, maxdims);
H5::DSetCreatPropList cparms;
cparms.setDeflate(1);
cparms.setFletcher32();
//Setup Chunking
const int DEFAULT_CHUNK_SIZE = 32768;
unsigned long long chunkFactor = DEFAULT_CHUNK_SIZE/4;
hsize_t chunk_dims[1] ={chunkFactor};
cparms.setChunk(1, chunk_dims);
float fill_val = 0.0;
cparms.setFillValue(H5::PredType::NATIVE_FLOAT, &fill_val);
H5::DataSet h5DataSet = h5File.createDataSet(datasetName, datatype, dataspace, cparms);
std::cout << "Created float datatype DataSet " << datasetName << std::endl;
//Get data ready
std::vector<float> data(count);
for (unsigned long long i = 0; i < count; i++) {
data[i] = (float)i;
}
//Write data to DataSet
std::cout << "Writing float datatype DataSet " << datasetName << " ....." << std::endl;
/*************************************************************************************************************************************/
//If there is insufficient disk space or Network disconnect after H5File creation and
//If data size is bigger than CHUNK_CACHE (default 1 MB),
//Access Violation in 1.8.5 - BAD
//H5::Exception in 1.8.11 - GOOD
h5DataSet.write(data.data(), h5DataSet.getDataType());
/*************************************************************************************************************************************/
std::cout << "Finished writing float datatype DataSet " << datasetName << std::endl;
//Close
std::cout << "Closing DataSet and File " << " ....." << std::endl;
//If there is insufficient disk space or Network disconnect after H5File creation and
//If data size is less than CHUNK_CACHE (default 1 MB), H5::Exception in both 1.8.5 and 1.8.11 - GOOD
h5DataSet.close();
h5File.close();
std::cout << "Closed DataSet and File " << std::endl;
}
catch(H5::Exception ex)
{
std::cout << ex.getDetailMsg() << std::endl;
}
std::cout << "Done" << std::endl;
return 0;
/*************************************************************************************************************************************/
} // Access Violation in H5F_close called from H5close in both 1.8.5 and 1.8.11 - BAD
/*************************************************************************************************************************************/
PS: Attached Visual Studio 2012 solution with two different project properties (one for 1.8.5 and 1.8.11). They need to be edited as per where the solution is copied to. Also please change the platform to x64.
Thank you,
Arun
HDF5WriteWithFullDisk.zipx (6.44 KB)