[Esip-documentation] updates to ACDD

Tue Oct 12 09:38:58 EDT 2021

Regarding "IOOS had recommended creator_url be ...":
IOOS can recommend whatever they want, but it doesn't change the ACDD 1.3
definition of "creator_url" which is
"The URL of the *person* (or other creator type specified by the
creator_type attribute) principally responsible for creating this data."
[emphasis added]
and which seems to be directly at odds with the IOOS recommendation.

Regarding "at least one major user would not accept any changes to existing
ACDD attributes that would invalidate any use that followed a previous
version" and the desire of some people to make major changes to ACDD:
The person resisting changes to existing attribute names and definitions
was me. I think the reasons for that should be obvious:
This community of NOAA (especially NCEI with its archive), NASA, and
hundreds of other groups, has 100's of thousands of datasets and 100's of
millions of files using the ACDD terms as defined in the various versions
1.0 - 1.3 of the ACDD standard. If you change one of the attribute names or
definitions:

   - At best you introduce a complication (a file's reader has to be aware
   of the difference between ACDD versions and interpret the attribute
   differently according to the stated ACDD version). That means crosswalks to
   other metadata formats need to be more sophisticated in order to deal with
   the differences between ACDD versions. That might not be too bad for the
   changes in one version of ACDD, but what about after 5,6,7 versions of
   ACDD?
   - You introduce uncertainty: Did the file's creator properly understand
   the differences between how the attribute was different versions of ACDD?
   This makes the crosswalks much harder to write.

You can see both of those problems already (although with minor
consequences) with the pointless change in the spelling of the "acknowledgment"
(the US spelling of the word) in pre-1.3 versions of ACDD to
"acknowledgement" (the British spelling) in ACDD 1.3. It is now common to
see files claiming to be ACDD 1.3 compliant with "acknowledgment"
(incorrect) and others using "acknowledgement" (correct).
It should be obvious that if a definition changed (instead of the spelling
of the attribute name), it would be a far more complex situation where the
reader must then guess what the file creator intended.

This situation highlights something else: there is a huge difference
between the perspective of a person creating a new data file for a new
dataset, and a person reading files from various sources. A person creating
a new data file for a new dataset has no prior constraints. All they want
to do is express the metadata content into the file using the standard. But
everyone and every project has different needs. For them, it's easy to get
frustrated with a standard because it doesn't fit their idea of what the
perfect metadata standard would be. Given a blank slate and working alone,
everyone would create a different standard. The hard part of making the
standard (and it was hard) is that we had to reconcile all these different
ideas about what the *perfect* metadata standard would be. So "perfect"
gets thrown out (since it is impossible) and "acceptable compromise" is the
best we can hope for. (Yes, as my wife says, "compromise is when nobody is
happy".) People making ACDD 1.3 had very diverse ideas about what topics
should be addressed, what the attribute names should be, and especially
what the definitions should be. Unfortunately, some people naturally retain
this idea that we should revamp the standard, but they are forgetting about
all the other people who would revamp the standard in a different way. And
they are forgetting about all the consumers of data files who have to deal
with different versions of a standard that have different attribute names
and definitions

As a great example of not changing attribute names or definitions but just
adding new attribute names and definitions, look at CF, which has been
quite stable through 9(?) versions. The result is that a writer of a file
with CF metadata can reliably write attributes to the file as they have
been for years (although periodically adding new attributes to their
vocabulary), and a reader of a file with CF metadata can pretty reliably
ignore the stated CF version and just see what attributes are present. If a
given attribute is present, it's definition is known. Thank goodness!

Ethan Davis had the remarkable luxury of having a blank slate and (I think)
of working alone when he created ACDD 1.0. But now, forever more, new
versions of ACDD will be painfully hashed out by numerous people working
toward an acceptable compromise. On behalf of other consumers of data files
and on behalf of software developers who write software that processes data
files, please, please, please, let's keep existing attribute names and
definitions stable. If you want to add new attribute names and definitions
to address new concepts, go for it.

Here's a compromise (but where everyone is happy) for all of you who really
want to make massive changes to ACDD (essentially starting from a blank
slate): go for it! Create your own metadata standard (just as Ethan did
with ACDD 1.0), but just give it a different name, not "ACDD". After all,
that is what your new metadata standard is: a new metadata standard. If it
is a great standard, as ACDD 1.0 was, it will fill a niche and be widely
adopted and possibly supplant ACDD (if it addresses the same issues). But
effectively killing off the current ACDD 1.x by labelling your new and very
different standard ACDD 2.0, is wrong unless everyone in the ESIP
Documentation Cluster agrees that it is time to kill off ACDD 1.x and go
down that different route (and you probably won't get my vote).  I like
ACDD (1.0 to 1.3) and its stability is incredibly valuable to me. I have a
lot invested in ACDD 1.3.. I know I'm not alone -- think of NASA and NCEI
with millions of archived files with ACDD 1.x metadata and all the software
that writes and reads these files.

Best wishes.

On Mon, Oct 11, 2021 at 4:33 PM John Graybeal via Esip-documentation <
esip-documentation at lists.esipfed.org> wrote:

> Hi Chris,
>
> I believe there have been 3, maybe 4 threads in the past 2-3 years about
> updating ACDD. I wouldn't say any of them were as action-oriented as
> yours—sometimes interest in a particular attribute, other times general
> interest in whether it's being maintained. None has gone so far as to say
> "I want to open a new round of discussion for ACDD." The list archive may
> have some details.
>
> I note that nothing *precludes* your using those identifiers for the
> people or organizations in the contributor attributes, all those
> identifiers can be named via URLs, which is consistent with the ACDD spec.
> What are you trying to do exactly that isn't already possible?
>
> Within the past week, there was a question in the #marinedata Slack
> channel of the ESIP workspace about ORCiDs in netCDF, followed by a long
> discussion about the ACDD 1.3 creator_url. In the course of that discussion
> it was mentioned that IOOS had recommended creator_url be
>
> The URL of the institution that collected the data. Note that this should
> always reference an institution URL, and not a personal URL, even
> if creator_type=person.
>
> I wasn't there so I can't fairly assess that guidance, but it is
> sufficiently, umm, unexpected that it'd be nice to get your needs met by
> ACDD directly.
>
> That said, two considerations about ACDD: (1) At the last update round, at
> least one major user would not accept any changes to existing ACDD
> attributes that would invalidate any use that followed a previous version.
> So our ability to update certain fields the way many members wanted to was
> effectively blocked. (2) With ESIP's Documentation Cluster(Committee?) as
> the current 'standards body' for ACDD, you'd be starting down a path that
> has not been travelled yet, to my knowledge.
>
> I hope that is the right level of information to share in response to your
> query! I think it would be great for ACDD to get another round, especially
> if it was clear that a break from the past was necessary to improve
> metadata quality from what the current standard can support. Obviously that
> would open up quite a number of questions that just might go beyond your
> own interest. ;-)
>
> John
>
> On Oct 11, 2021, at 1:02 PM, Chris Turner via Esip-documentation <
> esip-documentation at lists.esipfed.org> wrote:
>
> Hello all,
>
> I'm curious about any movement or interest to update the ACDD. I know that
> v1.3 is 6 years old, and the ESIP wiki makes it looks like there hasn't
> been interest or discussion in this since 2017. Is there still any intent
> to develop v2.0?
>
> My sudden interest in this comes from the need to include identifiers
> (ORCID, ResearchID, AuthorID, etc) for the person listed in the creator
> attributes and the people listed in the contributor attributes. I'd like to
> do this in a community-vetted way, but but if there isn't an active
> community working on ACDD anymore, we can look at including these
> attributes in one of the netCDF community profiles - probably the the IOOS
> metadata profile
> <https://ioos.github.io/ioos-metadata/ioos-metadata-profile-v1-2.html>.
>
> Thanks for whatever you can tell me.
>
> - Chris
>
>
> --
> Chris Turner
> Data Librarian | Axiom Data Science
> chris at axiomdatascience.com <chris at axiomalaska.com>
>
> _______________________________________________
> Esip-documentation mailing list
> To start a new topic: Esip-documentation at lists.esipfed.org
> To unsubscribe and manage prefs:
> https://lists.esipfed.org/mailman/listinfo/esip-documentation
>
>
> ----------------------
> John Graybeal
> Administrator—ESIP Community Ontology Repository
> jbgraybeal at sonic.net
>
>
>
>
> _______________________________________________
> Esip-documentation mailing list
> To start a new topic: Esip-documentation at lists.esipfed.org
> To unsubscribe and manage prefs:
> https://lists.esipfed.org/mailman/listinfo/esip-documentation
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.esipfed.org/pipermail/esip-documentation/attachments/20211012/01d3f4b1/attachment-0001.htm>