Possible bug with hyperslab selection in 1.10.7

Hi,

I’ve encountered some strange behaviour when using H5S_SELECT_OR to merge two hyperslab selections.

If I have two regular hyperslabs (A & B) the order in which I provide them to H5Sselect_hyperslab() (A OR B vs B OR A) seens to determine whether I get either the expected behaviour or an error. I only see this behaviour with HDF5 1.10.7 and not with 1.10.6.

The code in the link below demonstrates the issue.

In summary the code tries to:

  • First create a file and write a 10x10 dataset and confirm that my two sets of hyperslab parameters do what’s expected when used independently.
  • Then reset the file dataspace to hyperslab 1 selection and use a further call to H5Sselect_hyperslab(op = H5S_SELECT_OR) to combine the selections. H5Dread() works fine with this.
  • Finally, reset to hyperslab 2 selection and this time use H5Sselect_hyperslab() to combine with the hyperslab 1 parameters. H5Dread() works in 1.10.6, and errors with 1.10.7

Here’s the output using 1.10.6:

HDF5 Version: 1.10.6

Data Written to File:
	1	11	21	31	41	51	61	71	81	91
	2	12	22	32	42	52	62	72	82	92
	3	13	23	33	43	53	63	73	83	93
	4	14	24	34	44	54	64	74	84	94
	5	15	25	35	45	55	65	75	85	95
	6	16	26	36	46	56	66	76	86	96
	7	17	27	37	47	57	67	77	87	97
	8	18	28	38	48	58	68	78	88	98
	9	19	29	39	49	59	69	79	89	99
	10	20	30	40	50	60	70	80	90	100

Hyperslab 1:
	22	32
	25	35

Hyperslab 2:
	23	33
	26	36

Hyperslab 1 OR Hyperslab 2:
	22	32	23	33
	25	35	26	36

Hyperslab 2 OR Hyperslab 1:
	22	32	23	33
	25	35	26	36

However if I run the same code compiled against HDF5 1.10.7, I get the following output when it gets to reading the combined hyperslabs. The first seems to work fine, the second errors:

HDF5 Version: 1.10.7
...
Hyperslab 1 OR Hyperslab 2:
	22	32	23	33
	25	35	26	36
HDF5-DIAG: Error detected in HDF5 (1.10.7) thread 0:
  #000: H5Dio.c line 198 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: H5Dio.c line 599 in H5D__read(): can't read data
    major: Dataset
    minor: Read failed
  #002: H5Dcontig.c line 622 in H5D__contig_read(): contiguous read failed
    major: Dataset
    minor: Read failed
  #003: H5Dscatgath.c line 520 in H5D__scatgath_read(): file gather failed
    major: Low-level I/O
    minor: Read failed
  #004: H5Dscatgath.c line 250 in H5D__gather_file(): read error
    major: Dataspace
    minor: Read failed
  #005: H5Dcontig.c line 957 in H5D__contig_readvv(): can't perform vectorized sieve buffer read
    major: Dataset
    minor: Can't operate on object
  #006: H5VM.c line 1422 in H5VM_opvv(): can't perform operation
    major: Internal error (too specific to document in detail)
    minor: Can't operate on object
  #007: H5Dcontig.c line 861 in H5D__contig_readvv_sieve_cb(): block read failed
    major: Dataset
    minor: Read failed
  #008: H5Fio.c line 116 in H5F_block_read(): read through page buffer failed
    major: Low-level I/O
    minor: Read failed
  #009: H5PB.c line 736 in H5PB_read(): read through metadata accumulator failed
    major: Page Buffering
    minor: Read failed
  #010: H5Faccum.c line 260 in H5F__accum_read(): driver read request failed
    major: Low-level I/O
    minor: Read failed
  #011: H5FDint.c line 193 in H5FD_read(): addr overflow, addr = 2496, size = 400, eoa = 2448
    major: Invalid arguments to routine
    minor: Address overflowed

Many thanks for any feedback on what I might be doing wrong if this isn’t a bug.

Hello!

Thank you for the example! I can reproduce the issue and can see that it runs successfully with 1.8.22 and 1.10.6, but fails with 1.10.7 and 1.12.0.

I entered Jira issue HDFFV-11219 to investigate it. It would be good to understand why it runs successfully with one version of HDF5 but not the other.

The issue is open to the community.

-Barbara
The HDF Group

Thanks for confirming the behaviour and adding it to Jira.

For my own testing I confirmed I see the same problem if I combine the two dataspaces with H5Scombine_select() rather than H5Shyperslab_select(). That suggests to me its something in how the selection is used in the reading, rather than the way I was creating the selection.

There’s some code for the approach at https://gist.github.com/grimbough/effec0aa0b071bdb8817b9ad96c42a53

This only runs on 1.10.7 (and I guess 1.12.0) as H5Scombine_select() was introduced then.

Thanks again.

Thank you for this additional example! I added it to the bug report.
-Barbara

I’ve found another way to produce the error, perhaps with some additional information that might be useful.

This time I was trying to understand the difference between H5Sselect_hyperslab() and H5Scombine_hyperslab() when creating an irregular hyperslab selection. I’ve found what seems to be an inconsistency in their behaviours, and ultimately this can lead to the same error I reported previously.

There’s another example program at https://gist.github.com/grimbough/c9a59b6aba450204f6c7ad9702391821

Summary of the example code:

First create a file and write a 10x10 dataset and confirm that my two sets of hyperslab parameters do what’s expected when used independently. The two hyperslabs are both single 1-column, 2-row blocks, and if combined they should be irregular. The program will print details of the selections including the number of point, number of blocks, coordinates of the blocks and whether the selection is a regular hyperslab:

Hyperslab 1:
Number of points: 2
Number of hyperslab blocks: 2
(0,0) - (0,0)
(0,2) - (0,2)
Is regular hyperslab: 1
     start  = [0, 0] 
     stride = [1, 2] 
     count  = [1, 2] 
     block  = [1, 1] 

Select hyperslab 1 and combine with hyperslab 2 with a call to H5Sselect_hyperslab(). This appears to work and the details are consistent with what I would expect:

Combining with H5Sselect_hyperslab
Number of points: 4
Number of hyperslab blocks: 4
(0,0) - (0,0)
(0,2) - (0,2)
(2,2) - (2,2)
(2,4) - (2,4)
Is regular hyperslab: 0
Selection values:
	1	21
	23	43

Repeat; select hyperslab 1 and combine with hyperslab 2 via H5Scombine_hyperslab(). This time the returned dataspace has a selection that I don’t understand and is inconsistent with the result before. Why does it report that 4 points are selected, but only 2 blocks each of which are singular? Why does it confirm the new hyperslab is regular, when it should not be?

Combining with H5Scombine_hyperslab
Number of points: 4
Number of hyperslab blocks: 2
(2,2) - (2,2)
(2,4) - (2,4)
Is regular hyperslab: 1
     start  = [2, 2] 
     stride = [1, 2] 
     count  = [1, 2] 
     block  = [1, 1] 

Trying to use H5Dread() with this hyperslab selection leads to the same error as reported before.

 #000: H5Dio.c line 198 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
...
 #011: H5FDint.c line 193 in H5FD_read(): addr overflow, addr = 2536, size = 400, eoa = 2448
    major: Invalid arguments to routine
    minor: Address overflowed

Maybe that’s helpful, and I’m also grateful is someone finds a mistake in my code!

1 Like