Hi,
I have 3000 variables need to be write to 3 datasets (each 1000 variables, because one dataset can only have less than 2000 columns for column titles, so I set 1000 columns). I have 3 compound datasets. After writing out data into the first dataset, H5DWrite function always get exceptions when writing to the 2nd or the 3rd dataset for memory allocation error. I feel strange because each variable only has 12 values. Memory should not be a problem.
I got the following exception:
ncsa.hdf.hdf5lib.exceptions.HDF5ResourceUnavailableException: No space available for allocation
at ncsa.hdf.hdf5lib.H5.H5Dwrite_double(Native Method)
at ncsa.hdf.hdf5lib.H5.H5Dwrite(H5.java:2290)
at ncsa.hdf.hdf5lib.H5.H5Dwrite(H5.java:2217)
at wrimsv2.hdf5.HDF5Writer.writeMonthlyTimestepDvarAlias(HDF5Writer.java:264)
at wrimsv2.hdf5.HDF5Writer.writeTimestepDvarAlias(HDF5Writer.java:201)
at wrimsv2.hdf5.HDF5Writer.writeTimestepData(HDF5Writer.java:195)
at wrimsv2.components.ControllerBatch.runModelXA(ControllerBatch.java:353)
at wrimsv2.components.ControllerBatch.runModel(ControllerBatch.java:212)
at wrimsv2.components.ControllerBatch.<init>(ControllerBatch.java:66)
at test.test_ilp.Example_CL_SharingFix_092812.Future_BO_36steps(Example_CL_SharingFix_092812.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:73)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:667)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:840)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1141)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:135)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:119)
at org.testng.TestRunner.runWorkers(TestRunner.java:1108)
at org.testng.TestRunner.privateRun(TestRunner.java:737)
at org.testng.TestRunner.run(TestRunner.java:596)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:315)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:310)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:272)
at org.testng.SuiteRunner.run(SuiteRunner.java:221)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:946)
at org.testng.TestNG.runSuitesLocally(TestNG.java:883)
at org.testng.TestNG.run(TestNG.java:814)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:89)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:144)
HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
#000: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5Dio.c line 271 in H5Dwrite(): can't prepare for writing data
major: Dataset
minor: Write failed
#001: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5Dio.c line 352 in H5D__pre_write(): can't write data
major: Dataset
minor: Write failed
#002: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5Dio.c line 644 in H5D__write(): unable to set up type info
major: Dataset
minor: Unable to initialize object
#003: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5Dio.c line 1045 in H5D__typeinfo_init(): memory allocation failed for background conversion
major: Resource unavailable
minor: No space available for allocation
#004: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5FL.c line 954 in H5FL_blk_calloc(): memory allocation failed
major: Resource unavailable
minor: No space available for allocation
#005: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5FL.c line 886 in H5FL_blk_malloc(): memory allocation failed for chunk
major: Resource unavailable
minor: No space available for allocation
#006: C:\autotest\HDFJAVAjreReleaseTAR\build\native\HDF5-prefix\src\HDF5\src\H5FL.c line 206 in H5FL_malloc(): memory allocation failed for chunk
major: Resource unavailable
minor: No space available for allocation
The following is my code:
public static void writeMonthlyTimestepDvarAlias(){
long[] dims={12};
if (gidMonthly>=0){
try {
for (int k=0; k<3; k++){
int offset=0;
String dName="Timestep Table "+k;
int tidCompound = H5.H5Tcreate(HDF5Constants.H5T_COMPOUND, 8*1000);
for (int i=k*1000; i<(k+1)*1000; i++){
H5.H5Tinsert(tidCompound, String.valueOf(i), offset, HDF5Constants.H5T_NATIVE_DOUBLE);
offset=offset+8;
}
int sidTDA = H5.H5Screate_simple(1, dims, null);
if (sidTDA >= 0){
int didTDA = H5.H5Dcreate(gidMonthly, dName, tidCompound, sidTDA, HDF5Constants.H5P_DEFAULT, HDF5Constants.H5P_DEFAULT, HDF5Constants.H5P_DEFAULT);
if (didTDA >= 0){
offset=0;
for (int i=k*1000; i<(k+1)*1000; i++){
int tidCompoundTmp = H5.H5Tcreate(HDF5Constants.H5T_COMPOUND, 8);
H5.H5Tinsert(tidCompoundTmp, String.valueOf(i), offset, HDF5Constants.H5T_NATIVE_DOUBLE);
double[] data = DataTimeSeries.dvAliasTS.get(name).getData();
H5.H5Dwrite(didTDA, tidCompoundTmp,
HDF5Constants.H5S_ALL, HDF5Constants.H5S_ALL, HDF5Constants.H5P_DEFAULT, data);
H5.H5Tclose(tidCompoundTmp);
}
}
H5.H5Dclose(didTDA);
}
H5.H5Sclose(sidTDA);
H5.H5Tclose(tidCompound);
}
}catch(Exception e){
e.printStackTrace();
}
}
}
Hao Xie