Hi Werner,
On Apr 26, 2017, at 12:34 AM, Werner Benger <werner@cct.lsu.edu> wrote:
Probably correct, Quincey might know better how far the hyperslab
implementation could be modified to allow a negative stride. Maybe it
wouldn't be much of a change, just there was no need so far to do it. If
that is the case, then at least it would mean that the "hyperslab grammar"
would indeed allow operations like transpose (and rotation) of a dataset,
just the implementation does not support it. That aside, it's probably not
efficient as it would require multiple hyperslab operations, so a point
selection would be faster, but for big data and out-of-core data
management, sometimes seemingly inefficient operations turn out to be
better.
I like the idea quite a lot. However, it might be quite difficult to
implement. The hyperslabs are “compiled” into [complex] internal data
structures as they are built up, in order to make the I/O operations as
fast as possible. It would be possible to defer that compilation until
when the actual I/O operation occurs, but that would be another layer of
data structures, etc. I believe it would be worthwhile, particularly so
that transposes are possible, but it would definitely be a fair bit of work…
Quincey
On 26.04.2017 00:13, Isaac Gerg wrote:
I am not sure if you are allowed a negative stride but even if you are, I
believe the transpose still won't work because the stride doesnt stride
dimension like numpy but strides across the memory space and wrap is not
allowed. Does this line of thinking make sense?
On Tue, Apr 25, 2017 at 6:06 PM, Werner Benger <werner@cct.lsu.edu> wrote:
Hi Isaac,
it's just thought that it could be done like this:
Copy operation: start = 0, stride = 1
target_index = 0 + source_index * 1
Reflection: start = max_index, stride = -1
target_index = max_index + source_index * -1
So a negative stride in one dimensions would go "backwards", thus do a
reflection in this direction. In practice the stride parameter is probably
an unsigned integer and thus running out of bounds instead of going
backwards, as you say.
Werner
On 25.04.2017 23:48, Isaac Gerg wrote:
Hi Werner, thanks for the reply. Let's wrestle with this a bit. So
numpy does their indexing by strides, the stride for each dimension. hdf5
does their "striding" across the data in the sense of more of a selection
than a stride. I would be inclined to say that even a negative stride
wouldnt allow this. The problem is the stride is a simply stride and not
one based on dimension index so the stride can't wrap so you end up with
out of bounds issues with the datasets. Does this seem right?
On Tue, Apr 25, 2017 at 5:29 PM, Werner Benger <werner@cct.lsu.edu> >> wrote:
Hi Isaac,
without proof, but I could imagine it's possible to implement a shear
operation via hyperslabs. If so, then hyperslabs could be used to implement
rotations:
e.g. https://www.ocf.berkeley.edu/~fricke/projects/israel/paeth/r
otation_by_shearing.html
To do a transpose, one would need a reflection in addition to rotation
(e.g. http://techieme.in/matrix-rotation/ ) , and I guess doing this
operation of a reflection is not possible via hyperslabs, so doing a
transpose is out of reach. Probably it would require to allow something
like a negative stride value...
Werner
On 25.04.2017 22:24, Isaac Gerg wrote:
Hi Quincy, I just was reading about point selections and was wondering
if that's the way we would go. Thanks for confirming!
On Tue, Apr 25, 2017 at 4:21 PM, Quincey Koziol <koziol@lbl.gov> wrote:
> On Apr 25, 2017, at 1:02 PM, Isaac Gerg <isaac.gerg@gergltd.com> >>>> wrote:
>
> A coworker and I are mulling over if its possible to take a simple
3x3 matrix (stored row major) in a dataspace and write it out transposed to
disk using a hyperslab. It seems that the grammer provided by start,
stride, blocks, and count won't allow for this. We hope we are wrong! It
is possible to do this?
Hyperslabs won’t do this, but you could use a point selection
instead.
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.Hdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 <%28225%29%20578-4809> Fax.: +1 225 578-5362 <%28225%29%20578-5362>
_______________________________________________ Hdf-forum is for HDF
software users discussion. Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.Hdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 <%28225%29%20578-4809> Fax.: +1 225 578-5362 <%28225%29%20578-5362>
_______________________________________________ Hdf-forum is for HDF
software users discussion. Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.Hdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 <(225)%20578-4809> Fax.: +1 225 578-5362 <(225)%20578-5362>
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5