Question to future of HDF5DotNet

This question is aimed at the members of the HDFGroup and also to the community that uses the HDF5DotNet wrappers. I've seen a lot of traffic lately on questions about functionality that does not exist in the wrappers released with the initial release of HDF5 1.8.0. It has been noted that these wrappers were funded by a customer who did not require the full functionality. Therefore, not all of the functions exist. There are several useful ones that are currently not included. Additionally, it is clearly stated in the documentation that these wrappers are a prototype and are not officially supported.

As a side project, I've been doing some work to the HDF5DotNet source code to add some of these functions along with unit tests for the library to make sure things are working. I personally did not like the style in which the wrappers were written because it didn't conform to the .NET standard method of capitalization and other things. So I have updated my version of the library to change these things.

The question to the HDFGroup is this: what does the future hold for the HDF5DotNet wrappers? Is it something that might officially become a product that gets regular updates, or will it always just be the prototype wrappers? The main reason I ask is because I would like to see the ability to have a public source code repository where updates could be made to the wrapper source code. Does the HDFGroup have a Subversion repository that they would be willing to let the community use so that we could develop the HDF5DotNet library further? Or as another option, a public open source project site like Sourceforge could be used for the same purpose. I'd be willing to use my source code as the base because I prefer the styling. However, be aware that because of the naming changes I've made that it wouldn't be a drop in replacement to the current implementation of the HDF5DotNet wrapper.

I'd appreciate any feedback from the community and the HDFGroup on this subject. It seems that there could be some progress made on improving the library from the community, we just need a place for people to contribute and/or access the latest code. I'd vote for creating a Sourceforge project and going down that route, but I don't know if there are any restrictions in place from the HDFGroup.

As a side note, there are some very minor modifications that could be made to future versions of the header files that get released with the HDF5 binaries that would greatly simplify updates to the HDF5DotNet wrappers. It would not affect normal building of the HDF5 library either. It would be really great if that could get included.

Regards,
Jesse

What are the changes to the headers that you need?

···

-----
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Aug 4, 2010, at 20:41, Jesse Lai <jlai.matlab@gmail.com> wrote:

This question is aimed at the members of the HDFGroup and also to the community that uses the HDF5DotNet wrappers. I've seen a lot of traffic lately on questions about functionality that does not exist in the wrappers released with the initial release of HDF5 1.8.0. It has been noted that these wrappers were funded by a customer who did not require the full functionality. Therefore, not all of the functions exist. There are several useful ones that are currently not included. Additionally, it is clearly stated in the documentation that these wrappers are a prototype and are not officially supported.

As a side project, I've been doing some work to the HDF5DotNet source code to add some of these functions along with unit tests for the library to make sure things are working. I personally did not like the style in which the wrappers were written because it didn't conform to the .NET standard method of capitalization and other things. So I have updated my version of the library to change these things.

The question to the HDFGroup is this: what does the future hold for the HDF5DotNet wrappers? Is it something that might officially become a product that gets regular updates, or will it always just be the prototype wrappers? The main reason I ask is because I would like to see the ability to have a public source code repository where updates could be made to the wrapper source code. Does the HDFGroup have a Subversion repository that they would be willing to let the community use so that we could develop the HDF5DotNet library further? Or as another option, a public open source project site like Sourceforge could be used for the same purpose. I'd be willing to use my source code as the base because I prefer the styling. However, be aware that because of the naming changes I've made that it wouldn't be a drop in replacement to the current implementation of the HDF5DotNet wrapper.

I'd appreciate any feedback from the community and the HDFGroup on this subject. It seems that there could be some progress made on improving the library from the community, we just need a place for people to contribute and/or access the latest code. I'd vote for creating a Sourceforge project and going down that route, but I don't know if there are any restrictions in place from the HDFGroup.

As a side note, there are some very minor modifications that could be made to future versions of the header files that get released with the HDF5 binaries that would greatly simplify updates to the HDF5DotNet wrappers. It would not affect normal building of the HDF5 library either. It would be really great if that could get included.

Regards,
Jesse

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Hi,

I'm not a casual user of HDF/HDF.net but I have some elements I concur with:

I personally did not like the style in which the
wrappers were written because it didn't conform to the .NET standard method
of capitalization and other things.

+1, I remember when hacking some code with hdf.net that extension
methods where a short way for me to provide more .net convention
compliant wrapping over the library.

but I also understand that the wrapper library should be just that (to
not compromize performance for say), and a more high level API can sit
on top of it.

One thing to keep in mind is that it would be better to have the base
wrapping work done automatically (from reading the C headers) and add
a "sweet" layer on top of that, I know that mono project is doing
similar things to wrap native APIs in c#

Does the HDFGroup have a Subversion repository that they would
be willing to let the community use so that we could develop the HDF5DotNet
library further?

I would favor a distributed versioning system (hg or git), google code
provides this for free as well as others (sourceforge as well btw)

···

On Wed, Aug 4, 2010 at 17:41, Jesse Lai <jlai.matlab@gmail.com> wrote:

This question is aimed at the members of the HDFGroup and also to the
community that uses the HDF5DotNet wrappers. I've seen a lot of traffic
lately on questions about functionality that does not exist in the wrappers
released with the initial release of HDF5 1.8.0. It has been noted that
these wrappers were funded by a customer who did not require the full
functionality. Therefore, not all of the functions exist. There are
several useful ones that are currently not included. Additionally, it is
clearly stated in the documentation that these wrappers are a prototype and
are not officially supported.

As a side project, I've been doing some work to the HDF5DotNet source code
to add some of these functions along with unit tests for the library to make
sure things are working. I personally did not like the style in which the
wrappers were written because it didn't conform to the .NET standard method
of capitalization and other things. So I have updated my version of the
library to change these things.

The question to the HDFGroup is this: what does the future hold for the
HDF5DotNet wrappers? Is it something that might officially become a product
that gets regular updates, or will it always just be the prototype wrappers?
The main reason I ask is because I would like to see the ability to have a
public source code repository where updates could be made to the wrapper
source code. Does the HDFGroup have a Subversion repository that they would
be willing to let the community use so that we could develop the HDF5DotNet
library further? Or as another option, a public open source project site
like Sourceforge could be used for the same purpose. I'd be willing to use
my source code as the base because I prefer the styling. However, be aware
that because of the naming changes I've made that it wouldn't be a drop in
replacement to the current implementation of the HDF5DotNet wrapper.

I'd appreciate any feedback from the community and the HDFGroup on this
subject. It seems that there could be some progress made on improving the
library from the community, we just need a place for people to contribute
and/or access the latest code. I'd vote for creating a Sourceforge project
and going down that route, but I don't know if there are any restrictions in
place from the HDFGroup.

As a side note, there are some very minor modifications that could be made
to future versions of the header files that get released with the HDF5
binaries that would greatly simplify updates to the HDF5DotNet wrappers. It
would not affect normal building of the HDF5 library either. It would be
really great if that could get included.

Regards,
Jesse

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Jesse and All,

Sorry for the slow response. Many of us are on vacation or travel, and don't have a chance to reply right away.

This question is aimed at the members of the HDFGroup and also to the community that uses the HDF5DotNet wrappers. I've seen a lot of traffic lately on questions about functionality that does not exist in the wrappers released with the initial release of HDF5 1.8.0. It has been noted that these wrappers were funded by a customer who did not require the full functionality. Therefore, not all of the functions exist. There are several useful ones that are currently not included. Additionally, it is clearly stated in the documentation that these wrappers are a prototype and are not officially supported.

As a side project, I've been doing some work to the HDF5DotNet source code to add some of these functions along with unit tests for the library to make sure things are working. I personally did not like the style in which the wrappers were written because it didn't conform to the .NET standard method of capitalization and other things. So I have updated my version of the library to change these things.

The question to the HDFGroup is this: what does the future hold for the HDF5DotNet wrappers?

Good question, and unfortunately at this moment, we do not have a straight answer for you. Said this, we see how valuable .NET APIs are and would like to take several steps (and have been taking already) to provide a better support for the software:

1) .NET will be in SVN (read-only). We will announce it before September 1 after we finish internal testing.
2) We will be happy to accept patches, especially for the new APIs, bug fixes, tests, and programming examples.
3) The most important factor for the .NET support is a regression testing with the HDF5 code under development (both 1.8 branch and trunk). Our group currently
   doesn't have resources to do it and will need help from the HDF5 .NET community. If .NET can be built with CMake, it will be a tremendous help.
4) With the CMake project, we set up an SVN branch with a write access for the developers who worked with us. The check-ins were reviewed, tested and brought
   into the mainstream code that became available with 1.8.5. We think this model worked well and we may try to take a similar approach with .NET.

We (The HDF Group) will need to give more thought to 2-4 before moving forward. Suggestions, comments are more than welcome.

Thank you!

Elena

···

On Aug 4, 2010, at 7:41 PM, Jesse Lai wrote:

Is it something that might officially become a product that gets regular updates, or will it always just be the prototype wrappers? The main reason I ask is because I would like to see the ability to have a public source code repository where updates could be made to the wrapper source code. Does the HDFGroup have a Subversion repository that they would be willing to let the community use so that we could develop the HDF5DotNet library further? Or as another option, a public open source project site like Sourceforge could be used for the same purpose. I'd be willing to use my source code as the base because I prefer the styling. However, be aware that because of the naming changes I've made that it wouldn't be a drop in replacement to the current implementation of the HDF5DotNet wrapper.

I'd appreciate any feedback from the community and the HDFGroup on this subject. It seems that there could be some progress made on improving the library from the community, we just need a place for people to contribute and/or access the latest code. I'd vote for creating a Sourceforge project and going down that route, but I don't know if there are any restrictions in place from the HDFGroup.

As a side note, there are some very minor modifications that could be made to future versions of the header files that get released with the HDF5 binaries that would greatly simplify updates to the HDF5DotNet wrappers. It would not affect normal building of the HDF5 library either. It would be really great if that could get included.

Regards,
Jesse

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Hello Mike,

The changes are very minor. I attached the complete header include directory that I'm using to build HDF5DotNet with HDF5 v1.8.5. There is a file called changes.txt which lists the files that have changes. If you have a diff program like BeyondCompare, you can diff the whole directory to see the updates. The changes are essentially a "#ifndef HDF5_DOT_NET" statement before the public prototypes and function declarations (with matching "#endif" statement afterwards). The reason for it is that they are declared in the .NET wrappers as external calls to the C library, so the compiler thinks there are two definitions.

The update should still allow normal compiling, but would then allow simply dropping in new headers when compiling the HDF5DotNet library. It would be fantastic if that was incorporated in a future release. It would certainly make updating that much easier.

Thanks,
Jesse

include.zip (260 KB)

···

On 8/4/2010 7:01 PM, Michael Jackson wrote:

What are the changes to the headers that you need?

-----
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Aug 4, 2010, at 20:41, Jesse Lai<jlai.matlab@gmail.com> wrote:

This question is aimed at the members of the HDFGroup and also to the community that uses the HDF5DotNet wrappers. I've seen a lot of traffic lately on questions about functionality that does not exist in the wrappers released with the initial release of HDF5 1.8.0. It has been noted that these wrappers were funded by a customer who did not require the full functionality. Therefore, not all of the functions exist. There are several useful ones that are currently not included. Additionally, it is clearly stated in the documentation that these wrappers are a prototype and are not officially supported.

As a side project, I've been doing some work to the HDF5DotNet source code to add some of these functions along with unit tests for the library to make sure things are working. I personally did not like the style in which the wrappers were written because it didn't conform to the .NET standard method of capitalization and other things. So I have updated my version of the library to change these things.

The question to the HDFGroup is this: what does the future hold for the HDF5DotNet wrappers? Is it something that might officially become a product that gets regular updates, or will it always just be the prototype wrappers? The main reason I ask is because I would like to see the ability to have a public source code repository where updates could be made to the wrapper source code. Does the HDFGroup have a Subversion repository that they would be willing to let the community use so that we could develop the HDF5DotNet library further? Or as another option, a public open source project site like Sourceforge could be used for the same purpose. I'd be willing to use my source code as the base because I prefer the styling. However, be aware that because of the naming changes I've made that it wouldn't be a drop in replacement to the current implementation of the HDF5DotNet wrapper.

I'd appreciate any feedback from the community and the HDFGroup on this subject. It seems that there could be some progress made on improving the library from the community, we just need a place for people to contribute and/or access the latest code. I'd vote for creating a Sourceforge project and going down that route, but I don't know if there are any restrictions in place from the HDFGroup.

As a side note, there are some very minor modifications that could be made to future versions of the header files that get released with the HDF5 binaries that would greatly simplify updates to the HDF5DotNet wrappers. It would not affect normal building of the HDF5 library either. It would be really great if that could get included.

Regards,
Jesse

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
     

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Elena,

Thanks for the response. That sounds like a very good approach. The read-only SVN repo will allow people to have access to the latest code base. I don't know anything about CMake, but I will check into it. I did create an automated build script using the MSBuild process for the HDF5DotNet code base I've been working on. I'll look into what it would take (if possible) to build a .NET project with CMake.

I also like the idea of a branch of the code that the community could use for check-ins. I would like to help to the extent that I can and that would allow others to as well. The main benefit is that its hosted with the HDF Group rather than a 3rd party website.

What was the communication method for those developers working on the CMake project? Was there a separate listserv or something similar? I would think that would be a good thing to setup for the .NET wrapper development so developers could communicate without bogging down the whole list.

Thanks,
Jesse

···

On 8/8/2010 11:04 AM, Elena Pourmal wrote:

Good question, and unfortunately at this moment, we do not have a straight answer for you. Said this, we see how valuable .NET APIs are and would like to take several steps (and have been taking already) to provide a better support for the software:

1) .NET will be in SVN (read-only). We will announce it before September 1 after we finish internal testing.
2) We will be happy to accept patches, especially for the new APIs, bug fixes, tests, and programming examples.
3) The most important factor for the .NET support is a regression testing with the HDF5 code under development (both 1.8 branch and trunk). Our group currently
    doesn't have resources to do it and will need help from the HDF5 .NET community. If .NET can be built with CMake, it will be a tremendous help.
4) With the CMake project, we set up an SVN branch with a write access for the developers who worked with us. The check-ins were reviewed, tested and brought
    into the mainstream code that became available with 1.8.5. We think this model worked well and we may try to take a similar approach with .NET.

We (The HDF Group) will need to give more thought to 2-4 before moving forward. Suggestions, comments are more than welcome.

Thank you!

Elena

Jesse,

Good question, and unfortunately at this moment, we do not have a straight answer for you. Said this, we see how valuable .NET APIs are and would like to take several steps (and have been taking already) to provide a better support for the software:

1) .NET will be in SVN (read-only). We will announce it before September 1 after we finish internal testing.
2) We will be happy to accept patches, especially for the new APIs, bug fixes, tests, and programming examples.
3) The most important factor for the .NET support is a regression testing with the HDF5 code under development (both 1.8 branch and trunk). Our group currently
   doesn't have resources to do it and will need help from the HDF5 .NET community. If .NET can be built with CMake, it will be a tremendous help.
4) With the CMake project, we set up an SVN branch with a write access for the developers who worked with us. The check-ins were reviewed, tested and brought
   into the mainstream code that became available with 1.8.5. We think this model worked well and we may try to take a similar approach with .NET.

We (The HDF Group) will need to give more thought to 2-4 before moving forward. Suggestions, comments are more than welcome.

Thank you!

Elena

Elena,

Thanks for the response. That sounds like a very good approach. The read-only SVN repo will allow people to have access to the latest code base. I don't know anything about CMake, but I will check into it. I did create an automated build script using the MSBuild process for the HDF5DotNet code base I've been working on. I'll look into what it would take (if possible) to build a .NET project with CMake.

I also like the idea of a branch of the code that the community could use for check-ins. I would like to help to the extent that I can and that would allow others to as well. The main benefit is that its hosted with the HDF Group rather than a 3rd party website.

Thank you for your comments! We liked that approach too, so I hope it can work for other projects.

What was the communication method for those developers working on the CMake project? Was there a separate listserv or something similar? I would think that would be a good thing to setup for the .NET wrapper development so developers could communicate without bogging down the whole list.

There were only 5-6 people working on CMake, so we didn't use a mailing list. But I think if there are more people involved, setting up the list is a good idea.
Thanks again!

Elena

···

On Aug 12, 2010, at 7:57 PM, Jesse Lai wrote:

On 8/8/2010 11:04 AM, Elena Pourmal wrote:
Thanks,
Jesse

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org