[Esip-documentation] [DIWG] netcdf containers

Wed Jun 11 12:53:08 EDT 2014

Mark,

Sorry about the mistake - moving too quickly here… Thanks for the details…

Ted

[cid:3777702D-45F4-4250-BB1C-8AFBD78174C5]

On Jun 11, 2014, at 6:40 PM, Hedley, Mark <mark.hedley at metoffice.gov.uk<mailto:mark.hedley at metoffice.gov.uk>> wrote:

Hello Ted

I found the discussions today very useful, thank you for all your input.

I don't think that the vectors example (http://marqh.github.io/ncContainers/build/html/examples/vectorfield.html) uses nested groups.  In this case there is one group, which identifies the vector field. The vector components are explicitly allocated a role by the container, such that software may always find, for example, the i_component of vector field.

This example attempts to illustrate that a file which is CF compliant with a set of CF data variables can also provide the information I require to be able to process a vector field effectively and that this information is equally provided in a netCDF4 file, which should be CF compliant, but at the moment it is deemed not to be.

I think that the uncertainty example in my document more clearly demonstrates the way that nested hierarchy may be supported:
http://marqh.github.io/ncContainers/build/html/examples/uncert.html

In this example, the 'group'
sst_stats
contains a list of members who are also groups, containing lists of members

This is encoded using attributes and references in netCDF3 and using actual groups in netCDF4

I am very interested in following up on these topics with any interested parties.  As I stated in the meeting, the goals in my mind are:
 - CF conventions for NetCDF4;
 - capabilities to define roles for members of groups.

Any discussion on steps which may be taken which contribute to these goals is very valuable from my point of view.

many thanks
mark

________________________________
From: Ted Habermann [thabermann at hdfgroup.org<mailto:thabermann at hdfgroup.org>]
Sent: 11 June 2014 16:36
To: Hedley, Mark
Cc: Documentations; esdswg-interoperability at lists.nasa.gov<mailto:esdswg-interoperability at lists.nasa.gov>
Subject: Re: [DIWG] netcdf containers

Mark,

Thanks for the follow-up presentation on netCDF containers (http://marqh.github.io/ncContainers/build/html/intent.html). I am cc’ing two groups in the US that are also interested in this discussion:
the NASA working group I mentioned, and the ESIP Documentation Cluster. The second group is also interested in applying these ideas to the metadata in these files. There is an older (and maybe a bit out-of-date) discussion of some of these ideas at http://wiki.esipfed.org/index.php/NetCDF,_HDF,_and_ISO_Metadata.

The second group is also interested exploring conventional approaches to flattening files with groups. Most of our discussions have focused on concatenating group and variable names with some sort of delimiter(s). The approach you show in the Vector example at http://marqh.github.io/ncContainers/build/html/examples/vectorfield.html with container_members that are containers, might be an approach that avoids this somewhat fragile approach…

Ted

<SignatureSm2.png>

On May 9, 2014, at 4:22 PM, Ted Habermann <thabermann at hdfgroup.org<mailto:thabermann at hdfgroup.org>> wrote:

Mark,

Thanks for keeping me up-tp-date on this. I am sharing it with a NASA working group on interoperability as I know they will be interested...

Ted

On May 9, 2014, at 6:23 AM, Hedley, Mark <mark.hedley at metoffice.gov.uk<mailto:mark.hedley at metoffice.gov.uk>> wrote:

Hello all

I have made some small updates to the content:
http://marqh.github.io/ncContainers/build/html/index.html

based on feedback from a colleague here, which deal with some inconsistencies and clarity; nothing major has changed.

I would hugely value any feedback on the details of this proposal, whether it's aim has merit and potential avenues to pursue.

many thanks
mark

________________________________
From: Hedley, Mark
Sent: 10 April 2014 17:45
To: Jeremy Tandy; Tandy, Jeremy; Lorenzo Bigagli; thabermann at hdfgroup.org<mailto:thabermann at hdfgroup.org>; bendomenico at gmail.com<mailto:bendomenico at gmail.com>
Subject: netcdf containers

Hello

Following on from the discussions at the recent OGC TC on CF NetCDF I have reworked my draft proposal for netCDF containers

I have attempted to illustrate how they may support:

uncertainty
netcdf4 groups in netcdf3-classic
vector fields

I would very much welcome some informal feedback from each of you, as you each expressed interest from different points of view during the OGC meeting

I have prepared my first draft here:
http://marqh.github.io/ncContainers/build/html/index.html

Please feel free to browse the source:
https://github.com/marqh/ncContainers

and provide any comments on my private project issue tracker
https://github.com/marqh/ncContainers/issues

I do not know how this may be able to address some of the issues of the community but it seemed worth pursuing following the Washington discussions

I would like to discuss ways to perhaps take this on, if it is viewed to have merit, potential, not be so full of holes it's not worth further effort.

All input gratefully received

mark

<SignatureSm.png>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140611/279f0f3d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SignatureSm2.png
Type: image/png
Size: 16655 bytes
Desc: SignatureSm2.png
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140611/279f0f3d/attachment-0001.png>