Could portal.hdfgroup.org be made searchable by Googlebot?

[kmccarty]

Hi again

Entering this search string into Google:

site:portal.hdfgroup.org hdf5

turns up no results.

Is there a compelling reason that the HDF5 documentation is not searchable (presumably because Google’s crawler has been banned from this website) ? This really makes it extremely difficult to find HDF5 documentation about specific topics unless one knows in advance exactly where to look. It would be greatly appreciated if this could be fixed!

Thanks for considering,

Kevin

[lori.cooper]

Hi Kevin,

We are unfortunately aware of this problem. While we have identified a few potential causes and solutions, we have not had the resources to devote to this issue. We hope to work on it during this calendar year to get this resolved and get our documentation in a better state for users. As always, we welcome feedback and/or resources from our community to resolve this.

Lori Cooper
Product Marketing Associate

[kmccarty]

Thanks for your reply, Lori!

I noticed since first posting that this issue was already reported an entire year ago:

My google searches always land in support.hdfgroup.org

Whenever I do google searches using a function name from the C API, google only finds results in support.hdfgroup.org, the now no-longer maintained docs. I don’t ever get results in portal.hdfgroup.org. Is anyone else having this problem? Also, with docs in portal.hdfgroup.org, most of the API symbols in the rendered HTML are in ALL CAPS and have an underscore in them that is not actually in the C api function names (e.g. H5P_SET_FILTER instead of H5Pset_filter). I think this is problematic f…

Respectfully, I suggest that prioritizing a fix would benefit both current HDF5 library users and the HDF group. The current lack of searchable docs presents both a challenge to existing users, and a barrier against uptake by prospective users.

I have no insight into the technical issues you face, but maybe a workaround would be to generate a static mirror of the website (daily?) and allow that mirror to be searchable? This mirror could even be maintained by some enterprising person outside the HDF group if permission to create a mirror could be granted. (I am afraid that I cannot be that person since maintaining a mirror site would be far too time intensive for me.)

Best regards,
Kevin

[thomas1]

It looks like the robots.txt file (https://portal.hdfgroup.org/robots.txt ) creates a redirect loop - attempting to fetch it results in a 302 redirect back to the same address. As responsible crawlers use that file to determine what they are allowed to look at, I wonder if they respond to this brokenness by assuming the entire domain is off-limits.