[Esip-documentation] ACDD 2-3 question
Bob Simons - NOAA Federal via Esip-documentation
esip-documentation at lists.esipfed.org
Thu May 22 11:48:20 EDT 2014
Please see my comment below...
On 2014-05-22 7:29 AM, Nan Galbraith via Esip-documentation wrote:
> The assumption is that the *date_content_modified* would always be
> changed when a file is modified (or first written); if the geophysical
> data itself is modified, *date_values_modified* would also need to be
> updated. So, content_modified will never be before values_modified.
>
> Are we agreed that we need a date representing any changes to the
> geophysical data values and another date that's the time the file was
> written (by the data originator, that is)?
>
> If we're at that point, the attribute names might be easier.
>
> - file_date - the date on which this version of the file was created
> or modified, including geophysical values, quality values, metadata,
> or formatting (this is the date of the last time the 'original' copy of
> this file was changed in any way)
>
> - file_geophysical_values_date - the date on which the geophysical
> data in the file was modified
Not all data is geophysical data. E.g., some files have biological data.
>
> My concern is not that content includes data, but that there are 'values'
> including QC variables and other flag variables that would make
> this distinction ... squishy. We should decide (and state) whether
> 'values'
> includes these ancillary variables.
>
> Thanks -
> Nan
>
>
>
> On 5/21/14 8:15 PM, Armstrong, Edward M (398M) via Esip-documentation
> wrote:
>> That is a typo,
>>
>> My point was if you just use *date_content_modified *you are
>> indicating data, metadata and/or format changed. No way to
>> differentiate between the first two.
>>
>> The use case by Nan seemed to indicate changing data is very
>> important. Therefore the user should/must/ use date_values_modified
>> attribute. If they do not you have no idea if it was metadata or
>> data changed in file.
>>
>> Explicitly separating these attributes to describe data vs metadata
>> would avoid this possible confusion.
>>
>> My two cents......
>>
>>
>> On May 21, 2014, at 4:54 PM, John Graybeal <jbgraybeal at mindspring.com
>> <mailto:jbgraybeal at mindspring.com>> wrote:
>>
>>> Is it possible you misread, or we mistyped (or you mistyped)? I
>>> don't think there is a 'data_values_modified' -- I think the two
>>> names offered are as you suggest.
>>>
>>> Or is your concern that date_content_modified /includes/ changes to
>>> the data, and you propose it include just the metadata? We (small
>>> group of we) didn't see a strong use case for this to stand alone,
>>> but that the use case for the last time that *anything* in the file
>>> was changed was very strong (that's the one lots of people want).
>>>
>>> John
>>>
>>>
>>> On May 21, 2014, at 15:42, "Armstrong, Edward M (398M) via
>>> Esip-documentation" <esip-documentation at lists.esipfed.org
>>> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>>>
>>>> Hello all:
>>>> I still have some problems with date_content_modified vs
>>>> data_values_modified because they both contain "data" modified
>>>> references
>>>>
>>>> Would it not be better to explicitly have a one attribute to refer
>>>> to data changes (date_values_modified) and another for metadata
>>>> changes (*date_content_modified )*
>>>>
>>>>
>>>> On May 21, 2014, at 3:30 PM, John Graybeal via Esip-documentation
>>>> <esip-documentation at lists.esipfed.org
>>>> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>>>>
>>>>> Yay! Love all the comments.
>>>>>
>>>>> As we seem to have much attention at this moment, Iet me summarize
>>>>> the status: this version has been under discussion/development for
>>>>> over a year, and there is summary documentation (still painfully
>>>>> detailed) of many of the issues that arose, and how most of them
>>>>> were resolved, at the Talk (Discussion) page of the Working
>>>>> document
>>>>> <http://wiki.esipfed.org/index.php/Talk:Attribute_Convention_for_Data_Discovery_1-2_Working>.
>>>>> No obligation to read; just appreciate that the history is there,
>>>>> for the most part. (If you do read it, you'll find other tricky
>>>>> bits too...)
>>>>>
>>>>>> *Philip*: Which version is currently a candidate for receiving
>>>>>> review comments, and how can one submit comments on that version?
>>>>>> I'd rather not take the time reviewing and submitting comments
>>>>>> for the wrong version. I am looking at the version history table
>>>>>> on this page:
>>>>>> http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery
>>>>>
>>>>> The 'Working' document Anna asked about (the last one in the
>>>>> revision list you cite) is the one that is a candidate for
>>>>> receiving review comments. Comments have been requested on this
>>>>> version on several occasions, so we are all coming to this same
>>>>> point now.
>>>>>
>>>>> The description of the 1.2beta document is a little off, sorry;
>>>>> that document is the 'final form' document that tracks the working
>>>>> document, so you can see what it would look like in final form. I
>>>>> tried to fix the poor descriptions, but my access to the site has
>>>>> again gone south, so it will have to be later.
>>>>>
>>>>>> *Philip*: I didn't realize that some of the date attributes had
>>>>>> been re-named/purposed in 1-2. Is a rationale being captured
>>>>>> somewhere explaining these kinds of changes between versions?
>>>>>
>>>>> It is my understanding that this email list is the principle
>>>>> 'somewhere' for discussion, and my recollection that this question
>>>>> was discussed briefly on the thread, A Long Time Ago. I know it
>>>>> was discussed on a phone call, and there are several comments on
>>>>> it in the previously mentioned Talk page for the Working document
>>>>> (to which people have been pointed in email and the phone calls).
>>>>>
>>>>>> *Ted*: I that the intent of these dates has always been to
>>>>>> describe important dates for the *data* in the file rather than
>>>>>> the metadata in the file.
>>>>>
>>>>> The problem IMHO was that this intent -- to consider the values as
>>>>> data, whereas the metadata is not data -- was not clear from the
>>>>> name or description. ("date_created: The date on which the data
>>>>> was created."). Also, please consider your next sentence below....
>>>>>
>>>>> We went through several clarification rounds on the new
>>>>> definitions -- being unambiguous is hard. Still room for
>>>>> improvement I'm sure, or for rejection of the new approach.
>>>>>
>>>>>> *Ted*: Whether a data provider chooses to alter the date_modified
>>>>>> when they update metadata is up to the provider. If they don't
>>>>>> think it is important to users, they don't change the date_modified.
>>>>>
>>>>> This is explicitly ambiguous, and therefore not useful to those of
>>>>> us who want to know what the date means. It also negates your
>>>>> previous statement. Which helps explain the varied answers, when I
>>>>> asked different people what it meant.
>>>>>
>>>>>> *Ted*: These new names differentiate between content and values.
>>>>>> Does that mean that the data in the file is not content?
>>>>>
>>>>> Content means all file content, values means only that content
>>>>> which specify netCDF file values. I think these terms are
>>>>> unambiguous. I do not know what your term 'data' in your question
>>>>> means, but I think I've given you enough information for you to
>>>>> know the answer?
>>>>>
>>>>> Alternatively, if more people know exactly what 'data' means than
>>>>> know what 'content' and 'values' mean, these definitions are
>>>>> definitely worse.
>>>>>
>>>>>> *Anna*: However, I don't quite understand why
>>>>>> date_product_generated is /suggested /and the other two are
>>>>>> /recommended./
>>>>>
>>>>> My take: It is metadata about the packaging/distribution process,
>>>>> rather than metadata about the content.
>>>>>
>>>>>> *Derrick*: I think we need to just call a vote and tally up the
>>>>>> results.
>>>>>
>>>>> Do you still feel that way given the sudden waxing of this topic?
>>>>>
>>>>> Thanks again everyone for all the input.
>>>>>
>>>>> John
>>>>>
>>>>>> *From: *Philip Jones - NOAA Affiliate via Esip-documentation
>>>>>> <esip-documentation at lists.esipfed.org
>>>>>> <mailto:esip-documentation at lists.esipfed.org>>
>>>>>> *Subject: **Re: [Esip-documentation] ACDD 2-3 question*
>>>>>> *Date: *May 21, 2014 1:30:40 PM PDT
>>>>>> *To: *ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>
>>>>>> *Cc: *esip-documentation at lists.esipfed.org
>>>>>> <mailto:esip-documentation at lists.esipfed.org>
>>>>>> *Reply-To: *Philip Jones - NOAA Affiliate <philip.jones at noaa.gov
>>>>>> <mailto:philip.jones at noaa.gov>>
>>>>>>
>>>>>> All,
>>>>>>
>>>>>> I had raised similar questions as Anna's on ACDD version status
>>>>>> at the March telecon. Which version is currently a candidate for
>>>>>> receiving review comments, and how can one submit comments on
>>>>>> that version? I'd rather not take the time reviewing and
>>>>>> submitting comments for the wrong version. I am looking at the
>>>>>> version history table on this page:
>>>>>> http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery
>>>>>>
>>>>>> Also, I didn't realize that some of the date attributes had been
>>>>>> re-named/purposed in 1-2. Is a rationale being captured somewhere
>>>>>> explaining these kinds of changes between versions?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Phil
>>>>>
>>>>>> *From: *Ted Habermann <thabermann at hdfgroup.org
>>>>>> <mailto:thabermann at hdfgroup.org>>
>>>>>> *Subject: **Re: [Esip-documentation] ACDD 2-3 question*
>>>>>> *Date: *May 21, 2014 1:04:58 PM PDT
>>>>>> *To: *"ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>"
>>>>>> <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>>
>>>>>> *Cc: *"Armstrong, Edward M (398M)"
>>>>>> <Edward.M.Armstrong at jpl.nasa.gov
>>>>>> <mailto:Edward.M.Armstrong at jpl.nasa.gov>>, "John Graybeal"
>>>>>> <jbgraybeal at mindspring.com <mailto:jbgraybeal at mindspring.com>>,
>>>>>> Dave Neufeld <david.neufeld at noaa.gov
>>>>>> <mailto:david.neufeld at noaa.gov>>, Derrick Snowden
>>>>>> <derrick.snowden at noaa.gov <mailto:derrick.snowden at noaa.gov>>,
>>>>>> Anna Milan <anna.milan at noaa.gov <mailto:anna.milan at noaa.gov>>,
>>>>>> Steve Hankin <steven.c.hankin at noaa.gov
>>>>>> <mailto:steven.c.hankin at noaa.gov>>, Aleksandar Jelenak
>>>>>> <aleksandar.jelenak at noaa.gov
>>>>>> <mailto:aleksandar.jelenak at noaa.gov>>, Richard Signell
>>>>>> <rsignell at usgs.gov <mailto:rsignell at usgs.gov>>,
>>>>>> "dblodgett at usgs.gov <mailto:dblodgett at usgs.gov>"
>>>>>> <dblodgett at usgs.gov <mailto:dblodgett at usgs.gov>>, Bob Simons
>>>>>> <bob.simons at noaa.gov <mailto:bob.simons at noaa.gov>>, Ethan Davis
>>>>>> <edavis at unidata.ucar.edu <mailto:edavis at unidata.ucar.edu>>
>>>>>>
>>>>>> Nan,
>>>>>>
>>>>>> I that the intent of these dates has always been to describe
>>>>>> important dates for the *data* in the file rather than the
>>>>>> metadata in the file. Whether a data provider chooses to alter
>>>>>> the date_modified when they update metadata is up to the
>>>>>> provider. If they don't think it is important to users, they
>>>>>> don't change the date_modified. These new names differentiate
>>>>>> between content and values. Does that mean that the data in the
>>>>>> file is not content? Seems like a can of worms to me...
>>>>>>
>>>>>> I think somer of our guiding principles are "cause no confusion"
>>>>>> and "don't fix it if it ain't broken". I'm not convinced that
>>>>>> this is broken enough to fix...
>>>>>>
>>>>>> Ted
>>>>>
>>>>> On May 21, 2014, at 13:08, Anna Milan - NOAA Federal
>>>>> <anna.milan at noaa.gov <mailto:anna.milan at noaa.gov>> wrote:
>>>>>
>>>>>> Although more wordy, I'm comfortable with the
>>>>>> concepts/definitions of the new terms:
>>>>>>
>>>>>> date_content_modified
>>>>>>
>>>>>> The date on which any of the provided content, including
>>>>>> data, metadata, and presented format, was last changed (ISO
>>>>>> 8601 format)
>>>>>>
>>>>>> date_values_modified
>>>>>>
>>>>>> The date on which the provided data values were last changed;
>>>>>> excludes metadata and formatting changes (ISO 8601 format)
>>>>>>
>>>>>> date_product_generated
>>>>>> The date on which this data file or product was
>>>>>> produced/distributed (ISO 8601 format). While this date is
>>>>>> like a file timestamp, the date_content_modified and
>>>>>> date_values_modified should be used to assess the age of the
>>>>>> contents of the file or product.
>>>>>>
>>>>>> However, I don't quite understand why date_product_generated
>>>>>> is /suggested / and the other two are /recommended. /I would
>>>>>> expect it to be the other way around. On the other hand, I
>>>>>> also agree with Ted that 'date_created' and 'date_modified'
>>>>>> are sufficient and waaaay more straightforward.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Anna
>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>> Anna.Milan at noaa.gov <mailto:Anna.Milan at noaa.gov>, 303-497-5099
>>>>> NOAA/NESDIS/NGDC
>>>>>
>>>>> http://www.ngdc.noaa.gov/metadata/emma
>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>>
>>>>>
>>>>> On Wed, May 21, 2014 at 1:53 PM, Nan Galbraith
>>>>> <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>> wrote:
>>>>>
>>>>> Hi Ted -
>>>>>
>>>>> Actually, no, in the context of NetCDF files, I really don't
>>>>> understand
>>>>> date_modified and date_created. That might be because my
>>>>> files are
>>>>> never 'modified', we just write new files whenever we need to
>>>>> change
>>>>> anything. The original date when the first NetCDF file
>>>>> containing the
>>>>> data is not something we hang onto, either.
>>>>>
>>>>> We've had lots of conversations about if/why we need 2 'file
>>>>> dates',
>>>>> and this is what we came up with:
>>>>>
>>>>> The end user may not care about the last time I re-wrote a
>>>>> file if all I was
>>>>> doing was (e.g.) updating the contact info for the
>>>>> instrument's manufacturer,
>>>>> or updating from ACDD-1.1 to ACDD-1.2, but he might want to
>>>>> know if the
>>>>> data he's got on his desktop is exactly the same as what I've
>>>>> got on my server
>>>>> now.
>>>>>
>>>>> Hence, 2 dates. With the terms 'modified' and 'created' he
>>>>> would have no way
>>>>> to know; with 'content_modified' and 'values_modified' it's clear.
>>>>>
>>>>> I think John may have come up with these terms, so I just want
>>>>> to check...
>>>>> a change to the data would cause both date_content_modified and
>>>>> date_values_modified to be updated, and a change to some metadata
>>>>> would require just date_content_modified to be changed. Is
>>>>> that what you
>>>>> had in mind, John? I think I missed that part of the
>>>>> discussion ... or I was
>>>>> at sea and not paying attention.
>>>>>
>>>>> Cheers -
>>>>> Nan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 5/21/14 3:34 PM, Ted Habermann wrote:
>>>>>> All,
>>>>>>
>>>>>> I don't remember a discussion of these new terms (#2) and
>>>>>> feel like they are more, rather than less, confusing.
>>>>>> Everyone understands date_modified and date_created...
>>>>>>
>>>>>> Ted
>>>>>>
>>>>>>
>>>>>> On May 21, 2014, at 1:23 PM, Armstrong, Edward M (398M)
>>>>>> <Edward.M.Armstrong at jpl.nasa.gov
>>>>>> <mailto:Edward.M.Armstrong at jpl.nasa.gov>> wrote:
>>>>>>
>>>>>>> hi all:
>>>>>>>
>>>>>>> That does look like the correct working version site.
>>>>>>>
>>>>>>> I'm familiar with point #2. Guess I have not been paying
>>>>>>> attention. But the fields seem pretty explanatory.
>>>>>>>
>>>>>>> *date_content_modified *by itself would indicate no data
>>>>>>> values**were changed. You would need to add
>>>>>>> date_values_modified in that case too.
>>>>>>>
>>>>>>>
>>>>>>> On May 21, 2014, at 12:15 PM, Nan Galbraith
>>>>>>> <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>> wrote:
>>>>>>>
>>>>>>>> Hi All -
>>>>>>>>
>>>>>>>> Ah, yes, ACDD ... where do we stand with these terms,
>>>>>>>> and with the working version of the document?
>>>>>>>>
>>>>>>>> I don't recall how or when date_content_modified and
>>>>>>>> date_values_modified were arrived at, but they're a little
>>>>>>>> more clear than the old date_modified and date_created,
>>>>>>>> so that seems pretty good.
>>>>>>>>
>>>>>>>> Are we sticking with these new terms? Can we move ahead
>>>>>>>> with finalizing the working version, or should this be asked
>>>>>>>> on the Esip-documentation list, or ... have we given up?
>>>>>>>>
>>>>>>>> Thanks - Nan
>>>>>>>>
>>>>>>>>
>>>>>>>> On 5/21/14 2:06 PM, Anna Milan - NOAA Federal via
>>>>>>>> Esip-documentation wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I'm helping DSCOVR define their netCDF elements and am
>>>>>>>>> using the guidance from this ACDD version:
>>>>>>>>>
>>>>>>>>> http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_%28ACDD%29_Working
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. Is this the best version to be using? (They will NOT be
>>>>>>>>> using groups)
>>>>>>>>>
>>>>>>>>> 2. Are the date_modified, date_created, date_issued fields
>>>>>>>>> deprecated in favor of the following elements
>>>>>>>>> date_content_modified, date_values_modified,
>>>>>>>>> date_product_generated fields?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks in advance for your response!
>>>>>>>>>
>>>>>>>>> Anna
>>>>>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>>>>>> Anna.Milan at noaa.gov <mailto:Anna.Milan at noaa.gov>
>>>>>>>>> <mailto:Anna.Milan at noaa.gov>, 303-497-5099 <tel:303-497-5099>
>>>>>>>>> NOAA/NESDIS/NGDC
>>>>>>>>>
>>>>>>>>> http://www.ngdc.noaa.gov/metadata/emma
>>>>>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *******************************************************
>>>>> * Nan Galbraith Information Systems Specialist *
>>>>> * Upper Ocean Processes Group Mail Stop 29 *
>>>>> * Woods Hole Oceanographic Institution *
>>>>> * Woods Hole, MA 02543(508) 289-2444 <tel:%28508%29%20289-2444> *
>>>>> *******************************************************
>>>>>
>>>>>
>>>>>
>>>>
>>>> John Graybeal
>>>> jbgraybeal at mindspring.com <mailto:jbgraybeal at mindspring.com>
>>>>
>
> --
> *******************************************************
> * Nan Galbraith Information Systems Specialist *
> * Upper Ocean Processes Group Mail Stop 29 *
> * Woods Hole Oceanographic Institution *
> * Woods Hole, MA 02543 (508) 289-2444 *
> *******************************************************
>
>
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> http://www.lists.esipfed.org/mailman/listinfo/esip-documentation
--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
1352 Lighthouse Ave
Pacific Grove, CA 93950-2079
Phone: (831)658-3205
Fax: (831)648-8440
Email: bob.simons at noaa.gov
The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric
Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <>< <><
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140522/6360b50d/attachment-0001.html>
More information about the Esip-documentation
mailing list