Hi there,all!
I’m new to HDF5, and am working on an AMR generator recently.
I want to use HDF5+XDMF to save the AMR as an .h5 and an .xdmf file.
The structure of my AMR is like this:
The mesh has N block, and each block has I*J*K cells.
The size between blocks differs.
The size of cells of one same block is the same.
Could anyone kindly teach me the typical format to save such an AMR?
THX a lot!
My colleague @mlarson develops a nice tool called Agentic HDF5. Watch his intro. Using this tool, here’s what I got in response to your question:
● AMR Storage in HDF5+XDMF
For block-structured AMR with variable block sizes, the standard approach is:
HDF5 Structure
Each block is a separate group (since blocks have different I×J×K dimensions, they can't share a single dataset):
/
├── metadata/
│ ├── block_origins [N, 3] # (x0, y0, z0) of each block
│ ├── block_spacings [N, 3] # (dx, dy, dz) per block
│ ├── block_dims [N, 3] # (I, J, K) per block
│ └── block_levels [N] # refinement level per block
└── fields/
├── block_0000/
│ ├── density [I, J, K]
│ └── velocity [I, J, K, 3]
├── block_0001/
│ ├── density [I', J', K']
│ └── velocity [I', J', K', 3]
└── ...
XDMF Structure
The XDMF file uses a spatial Collection of Uniform grids — one per block — with ORIGIN_DXDYDZ geometry (the natural
fit for uniform Cartesian blocks):
<?xml version="1.0" ?>
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd" []>
<Xdmf Version="3.0">
<Domain>
<Grid GridType="Collection" CollectionType="Spatial" Name="AMR">
<!-- Block 0 -->
<Grid GridType="Uniform" Name="block_0000">
<Topology TopologyType="3DCoRectMesh" Dimensions="K+1 J+1 I+1"/>
<Geometry GeometryType="ORIGIN_DXDYDZ">
<DataItem Format="HDF" Dimensions="3">mesh.h5:/metadata/block_origins[0]</DataItem>
<DataItem Format="HDF" Dimensions="3">mesh.h5:/metadata/block_spacings[0]</DataItem>
</Geometry>
<Attribute Name="density" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="K J I">mesh.h5:/fields/block_0000/density</DataItem>
</Attribute>
</Grid>
<!-- Block 1 ... -->
</Grid>
</Domain>
</Xdmf>
▎ Dimension ordering note: XDMF uses ZYX (slow-to-fast) order — so for an I×J×K cell array, write Dimensions="K J I"
▎ for cells and K+1 J+1 I+1 for nodes.
Key design decisions
┌───────────────────────┬─────────────────┬──────────────────────────────────────────────────────────────┐
│ Choice │ Recommendation │ Reason │
├───────────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ Grid topology │ 3DCoRectMesh │ Cells within a block are uniform size │
├───────────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ Geometry type │ ORIGIN_DXDYDZ │ Compact — stores origin + 3 spacings, not all coordinates │
├───────────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ Center │ Cell (not Node) │ AMR data is cell-centered by convention │
├───────────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ Per-block HDF5 groups │ Yes │ Blocks have different I×J×K, so separate datasets are needed │
├───────────────────────┼─────────────────┼──────────────────────────────────────────────────────────────┤
│ CollectionType │ Spatial │ Supported by ParaView/VisIt; Tree has limited tool support │
└───────────────────────┴─────────────────┴──────────────────────────────────────────────────────────────┘
Python skeleton
import h5py
import numpy as np
def write_amr_hdf5(blocks, field_name, filepath):
"""
blocks: list of dicts with keys:
origin [3] - (x0, y0, z0)
spacing [3] - (dx, dy, dz)
data [I,J,K] - field values
"""
N = len(blocks)
with h5py.File(filepath, "w") as f:
meta = f.create_group("metadata")
meta["block_origins"] = np.array([b["origin"] for b in blocks])
meta["block_spacings"] = np.array([b["spacing"] for b in blocks])
meta["block_dims"] = np.array([b["data"].shape for b in blocks])
fields = f.create_group("fields")
for i, b in enumerate(blocks):
grp = fields.create_group(f"block_{i:04d}")
grp.create_dataset(field_name, data=b["data"],
compression="gzip", chunks=True)
def write_xdmf(blocks, field_name, h5path, xdmfpath):
lines = [
'<?xml version="1.0" ?>',
'<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd" []>',
'<Xdmf Version="3.0"><Domain>',
' <Grid GridType="Collection" CollectionType="Spatial" Name="AMR">',
]
for i, b in enumerate(blocks):
I, J, K = b["data"].shape
tag = f"block_{i:04d}"
lines += [
f' <Grid GridType="Uniform" Name="{tag}">',
f' <Topology TopologyType="3DCoRectMesh" Dimensions="{K+1} {J+1} {I+1}"/>',
f' <Geometry GeometryType="ORIGIN_DXDYDZ">',
f' <DataItem Format="HDF" Dimensions="3">{h5path}:/metadata/block_origins[{i}]</DataItem>',
f' <DataItem Format="HDF" Dimensions="3">{h5path}:/metadata/block_spacings[{i}]</DataItem>',
f' </Geometry>',
f' <Attribute Name="{field_name}" AttributeType="Scalar" Center="Cell">',
f' <DataItem Format="HDF" Dimensions="{K} {J} {I}">{h5path}:/fields/{tag}/{field_name}</DataItem>',
f' </Attribute>',
f' </Grid>',
]
lines += [' </Grid>', '</Domain></Xdmf>']
with open(xdmfpath, "w") as f:
f.write("\n".join(lines))
Tools that read this format
- ParaView — open the .xdmf file directly; recognizes the spatial collection as multi-block AMR
- VisIt — same
- h5py / xarray — read the .h5 directly using the metadata group for geometry reconstruction
I have not verified the correctness of the code snippets, but given the information you’ve provided, this appears to be a reasonable starting point.
Best, G.
Here are a few clarifications regarding XDMF paths and handling:
The row-indexing notation used in the XDMF paths is invalid:
mesh.h5:/metadata/block_origins[{i}]
XDMF does not support this indexing method. To reference individual rows from XDMF, you must use the ugly verbose HyperSlab syntax, like this:
<DataItem ItemType="HyperSlab" Dimensions="3">
<DataItem Dimensions="3 2" Format="XML">2 0 1 1 1 3</DataItem> <!-- row 2, all 3 columns -->
<DataItem Format="HDF" Dimensions="N 3">mesh.h5:/metadata/block_origins</DataItem>
</DataItem>
A simpler solution is to store the origin and spacing for each block group, allowing for straightforward paths such as:
mesh.h5:/fields/block_0002/origin
mesh.h5:/fields/block_0002/spacing
Although this topic is primarily relevant to C++, it’s essential to understand the ZYX convention. The Python example provided may not be precise; XDMF specifies dimensions with the slowest-varying dimension listed first (KJI order). However, the Python code treats the dimensions incorrectly with I, J, K = b["data"].shape, assuming I is the slowest axis. Consequently, it writes the topology as {K+1} {J+1} {I+1}. This is inconsistent because if the data shape is (I, J, K), the topology should instead read {I+1} {J+1} {K+1}. This reversal of indices relative to what unpacking implies is one of the most common sources of confusion for newcomers, so it’s worth double-checking.
Since you are exploring XDMF, it seems you want to avoid adopting standard AMR frameworks such as Chombo, AMReX, FLASH, SAMRAI, or Enzo. If your goal is to remain flexible and not be locked into a specific framework, consider using Conduit and Blueprint. Conduit is a data description and I/O library, while Blueprint serves as its mesh schema specification, explicitly addressing AMR hierarchies, including parent-child relationships, refinement ratios, and patch topology. You can describe your AMR data using Blueprint terminology, and Conduit will manage writing it to an HDF5 file. An added bonus is that tools like ASCENT and other in-situ tools can consume this data natively. This approach is among the closest options for an “AMR I/O schema as a library,” and it is tailored specifically for C++ code. This said, you willl need to take on the toolchain dependency as neither Paraview or VisIT can handle a Blueprint HDF5 file without some work.
This short note outlines a simple and practical approach to storing block-structured Adaptive Mesh Refinement (AMR) data in HDF5, based on guidance from the HDF Group community.
Credit: The layout and initial idea were inspired by a response and Python sketch from Gerd Heber.
We consider an AMR mesh with:
Goal: store AMR data in HDF5 + XDMF such that it is compatible with tools like ParaView and VisIt.
The key design decision is simple: each block must be stored separately, because blocks with different shapes cannot share the same dense dataset cleanly.
/
├── metadata/
│ ├── block_origins [N,3] # (x0,y0,z0)
│ ├── block_spacings [N,3] # (dx,dy,dz)
│ ├── block_dims [N,3] # (I,J,K)
│ └── block_levels [N]
└── fields/
├── block_0000/
│ ├── density [I,J,K]
│ └── velocity [I,J,K,3]
├── block_0001/
└── ...
This layout is simple, explicit, and aligns well with common visualization tools.
Each block becomes a Uniform Grid, grouped into a Spatial Collection:
<Grid GridType="Collection" CollectionType="Spatial" Name="AMR">
<Grid GridType="Uniform" Name="block_0000">
<Topology TopologyType="3DCoRectMesh" Dimensions="K+1 J+1 I+1"/>
<Geometry GeometryType="ORIGIN_DXDYDZ">
<DataItem Format="HDF" Dimensions="3">mesh.h5:/metadata/block_origins[0]</DataItem>
<DataItem Format="HDF" Dimensions="3">mesh.h5:/metadata/block_spacings[0]</DataItem>
</Geometry>
<Attribute Name="density" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="K J I">
mesh.h5:/fields/block_0000/density
</DataItem>
</Attribute>
</Grid>
</Grid>
XDMF uses ZYX ordering, so:
[I,J,K] becomes "K J I""K+1 J+1 I+1"struct block_t {
std::array<double,3> origin, spacing;
std::array<std::size_t,3> dims;
std::uint32_t level;
std::vector<double> density, velocity;
};
h5::write(fd, "/metadata/block_origins", block_origins);
h5::write(fd, "/metadata/block_spacings", block_spacings);
h5::write(fd, "/metadata/block_dims", block_dims);
h5::write(fd, "/metadata/block_levels", block_levels);
for (std::size_t i = 0; i < blocks.size(); ++i) {
const auto tag = std::format("/fields/block_{:04d}", i);
h5::write(fd, tag + "/density", b.density, h5::current_dims{I,J,K}, h5::chunk{...} | h5::gzip{4});
h5::write(fd, tag + "/velocity", b.velocity, h5::current_dims{I,J,K,3}, h5::chunk{...} | h5::gzip{4});
}
For std::array<T,3>:
template<>
hid_t register_struct<std::array<double,3>>() {
hsize_t dims[1] = {3};
return H5Tarray_create2(H5T_IEEE_F64LE, 1, dims);
}
This ensures correct mapping to:
H5T_ARRAY { [3] H5T_IEEE_F64LE }
h5dump -H amr_synthetic.h5
HDF5 "amr_synthetic.h5" {
GROUP "/" {
GROUP "fields" {
GROUP "block_0000" {
DATASET "density" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8 ) / ( 8, 8, 8 ) }
}
DATASET "velocity" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8, 3 ) / ( 8, 8, 8, 3 ) }
}
}
GROUP "block_0001" {
DATASET "density" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8 ) / ( 8, 8, 8 ) }
}
DATASET "velocity" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8, 3 ) / ( 8, 8, 8, 3 ) }
}
}
GROUP "block_0002" {
DATASET "density" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8 ) / ( 8, 8, 8 ) }
}
DATASET "velocity" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 8, 8, 3 ) / ( 8, 8, 8, 3 ) }
}
}
GROUP "block_0003" {
DATASET "density" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 12, 10 ) / ( 8, 12, 10 ) }
}
DATASET "velocity" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 8, 12, 10, 3 ) / ( 8, 12, 10, 3 ) }
}
}
GROUP "block_0004" {
DATASET "density" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 10, 9, 11 ) / ( 10, 9, 11 ) }
}
DATASET "velocity" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 10, 9, 11, 3 ) / ( 10, 9, 11, 3 ) }
}
}
}
GROUP "metadata" {
DATASET "block_dims" {
DATATYPE H5T_ARRAY { [3] H5T_STD_U64LE }
DATASPACE SIMPLE { ( 5 ) / ( 5 ) }
}
DATASET "block_levels" {
DATATYPE H5T_STD_U32LE
DATASPACE SIMPLE { ( 5 ) / ( 5 ) }
}
DATASET "block_origins" {
DATATYPE H5T_ARRAY { [3] H5T_IEEE_F64LE }
DATASPACE SIMPLE { ( 5 ) / ( 5 ) }
}
DATASET "block_spacings" {
DATATYPE H5T_ARRAY { [3] H5T_IEEE_F64LE }
DATASPACE SIMPLE { ( 5 ) / ( 5 ) }
}
}
}
}
Thanks for your reply!
I’ve checked Agentic HDF5, it’s an impressive tool, and from my point of view, Agentic HDF5 will become popular!
Also the Python sketch is quite illuminating!
In the future, I may employ LLM ,LOL.
Best regards!
Thank you for your reply and for sharing!
I’ll take a look at the Conduit + Blueprint libraries, and I’ll be careful about the dimension order you mentioned.
Also, I briefly tried using HyperSlab in XDMF – it’s really difficult and abstract for me, :'D
Finally, thank you!
Best regards
Thanks for your reply!
I checked out your GitHub profile and was amazed that you built such a complete project so quickly – impressive!
Your project is also very insightful!
Thank you, and best regards!