# Does h5py support masked numpy arrays?

**URL:** <https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865>\
**Category:** h5py\
**Created:** [December 9, 2024, 3:32am UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865 "2024-12-09T03:32:14Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nepomnyi](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/nepomnyi/32/2030_2.png) [@nepomnyi](https://forum.hdfgroup.org/u/nepomnyi)\
**Post date:** [December 9, 2024, 3:32am UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/1 "2024-12-09T03:32:14Z")

</div>

Hello, dear HDF5 team.  
I’m wondering if you can clarify for me, please, if `h5py` supports `np.ma` arrays (i.e., masked `numpy` arrays). In other words, when I create a dataset - for example as `dataSet = groupName.create_dataset("dataSetName", data=np.ma.masked_array([1,2],mask=[True,False]))` - will it store `data` as an `np.ma` array or as a regular `np` array? [Documentation](https://docs.h5py.org/en/stable/faq.html) does not specify that explicitly (or I don’t understand how to interpret that table in the referenced link).  
If follow up questions are allowed on this website, then how do I retrieve a masked array which I saved into an `.hdf5` file beforehand?  
Thank you in advance.  
Ivan

---

<div class="post-metadata">

**Author:** ![hyoklee](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/hyoklee/32/1528_2.png) [@hyoklee](https://forum.hdfgroup.org/u/hyoklee)\
**Post date:** [December 9, 2024, 3:33pm UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/2 "2024-12-09T15:33:51Z")

</div>

Hi,

Would you please try reporting an issue at [GitHub - h5py/h5py: HDF5 for Python -- The h5py package is a Pythonic interface to the HDF5 binary data format.](https://github.com/h5py/h5py)?

---

<div class="post-metadata">

**Author:** ![rosborn](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/rosborn/32/118_2.png) [@rosborn](https://forum.hdfgroup.org/u/rosborn)\
**Post date:** [December 9, 2024, 4:59pm UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/3 "2024-12-09T16:59:58Z")

</div>

I can’t claim to give an authoritative answer, but my understanding is that there is no mechanism to store masked arrays. I think it would only be implemented explicitly in h5py if HDF5 had a convention for storing dataset masks (which I doubt). I think you would have to implement your own convention by storing the field and masks as separate datasets, with some way of connecting them in your own code, e.g., by specifying the path to the mask as a dataset attribute or by naming the mask dataset by adding “\_mask” to the dataset name. We use the former in the [nexusformat package](https://nexpy.github.io/nexpy/), which read and writes HDF5 files written using the NeXus standard.

---

<div class="post-metadata">

**Author:** ![rosborn](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/rosborn/32/118_2.png) [@rosborn](https://forum.hdfgroup.org/u/rosborn)\
**Post date:** [December 9, 2024, 5:21pm UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/4 "2024-12-09T17:21:14Z")

</div>

There is a discussion of how we implement masks [here](https://nexpy.github.io/nexpy/pythonshell.html?highlight=mask#masked-arrays).

---

<div class="post-metadata">

**Author:** ![rosborn](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/rosborn/32/118_2.png) [@rosborn](https://forum.hdfgroup.org/u/rosborn)\
**Post date:** [December 9, 2024, 5:45pm UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/5 "2024-12-09T17:45:20Z")

</div>

By the way, the nexusformat package can be used to read and write HDF5 files that do not conform to the NeXus standard. To implement your example, you could use the following code:

```auto
from nexusformat.nexus import NXfield, nxopen

data = NXfield(np.ma.masked_array([1,2],mask=[True,False]))

with nxopen('mydata.h5', 'w') as root:
    root['data'] = data
    print(root.tree)

```

This produces the following HDF5 file.

```auto
root:NXroot
  data = [-- 2]
    @mask = 'data_mask'
  data_mask = [True False]

```

If you read it again, the datasets ‘data’ and ‘data\_mask’ are returned as a single NXfield, which just wraps the NumPy array with its attributes.

```auto
with nxopen('mydata.h5') as root:
    input_data = root['data']

```

The masked array is contained within the NXfield, accessible as the `nxvalue` attribute.

```auto
masked_array(data=[--, 2],
             mask=[True, False],
       fill_value=999999)

```

---

<div class="post-metadata">

**Author:** ![nepomnyi](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/nepomnyi/32/2030_2.png) [@nepomnyi](https://forum.hdfgroup.org/u/nepomnyi)\
**Post date:** [December 9, 2024, 10:50pm UTC](https://forum.hdfgroup.org/t/does-h5py-support-masked-numpy-arrays/12865/6 "2024-12-09T22:50:58Z")

</div>

Thank you very much,@rosborn! Your example with nexusformat looks pretty easy to work with. I think, I’ll stick with it.  
@hyoklee, I will convert my post to an issue on GitHub.  
Ivan