[Esip-documentation] Next weeks meeting

[Ted Habermann]

Bob,

Nice to hear from you… You have always been a great representative of the toolmaker point of view… Would be great to have that point of view represented in the discussion…

I think your suggestion is very similar to the work already done by OPeNDAP and HDF Group with dmr++ and ZARR...

Ted

> On Oct 22, 2020, at 3:07 PM, Bob Simons - NOAA Federal <bob.simons at noaa.gov> wrote:
> 
> I don't think this is a good idea.
> 
> It is specific to S3. By this logic, we would have separate metadata systems for every cloud provider. And it means the poor toolmakers (like me) have to spend the rest of their lives running around after you, adding yet another metadata reader for yet another system. And in this specific case, it means you could only read the metadata by using the S3 tools, whereas you can get the contents of a public bucket (e.g. the data) via a URL, even from outside the system.
> 
> Instead, for files/buckets/whatever (eg .csv files, relational databases) that don't have a native metadata storage system, how about just 1 file-like metadata storage system that could be used by all. And doesn't that already exist with the NCO json (http://nco.sourceforge.net/nco.html#json <http://nco.sourceforge.net/nco.html#json>) (if you leave out the data part and just include the metadata) or ncml?
> 
> 
> 
> On Thu, Oct 22, 2020 at 11:32 AM David Neufeld via Esip-documentation <esip-documentation at lists.esipfed.org <mailto:esip-documentation at lists.esipfed.org>> wrote:
> Right, and it would fit well in an HDF/ZARR context.
> 
> Object 1 file format CSV
> Object 2 file format Zarr
> Object 3 file format NetCDF
> 
> Object metadata properties are standardized across all three!
> 
> 
> 
> On Thu, Oct 22, 2020 at 12:17 PM Ted Habermann <tedhabermann at gmail.com <mailto:tedhabermann at gmail.com>> wrote:
> Dave,
> 
> This sounds good… Contrasts with having the metadata in a separate object, yes?
> 
> We could put this together with Aleksandar talking about HDF/ZARR?
> 
> Ted
> 
>> On Oct 22, 2020, at 12:05 PM, David Neufeld <david.neufeld at noaa.gov <mailto:david.neufeld at noaa.gov>> wrote:
>> 
>> Hi Ted,
>> 
>> How about a discussion on embedding metadata in s3 object metadata? Basically the idea would be to take our ACDD community best practices and do a proof of concept with embedding it in the s3 object metadata. A self describing file format independent of underlying data format might be nice.
>> 
>> Dave
>> 
>> 
>> On Thu, Oct 22, 2020 at 11:57 AM Ted Habermann via Esip-documentation <esip-documentation at lists.esipfed.org <mailto:esip-documentation at lists.esipfed.org>> wrote:
>> Hello all,
>> 
>> Next week we have one agenda item - planning for the winter meeting. This should not take the whole meeting. If anyone has another item they would like to discuss please let the group know…
>> 
>> Stay safe,
>> Ted
>> _______________________________________________
>> Esip-documentation mailing list
>> Esip-documentation at lists.esipfed.org <mailto:Esip-documentation at lists.esipfed.org>
>> https://lists.esipfed.org/mailman/listinfo/esip-documentation <https://lists.esipfed.org/mailman/listinfo/esip-documentation>
> 
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org <mailto:Esip-documentation at lists.esipfed.org>
> https://lists.esipfed.org/mailman/listinfo/esip-documentation <https://lists.esipfed.org/mailman/listinfo/esip-documentation>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.esipfed.org/pipermail/esip-documentation/attachments/20201022/bd117d39/attachment.htm>