[Esip-documentation] on documenting instrument and platform metadata
Jim Biard
jbiard at cicsnc.org
Thu Mar 19 12:11:43 EDT 2015
Sippy-cup! ESIP ACUP.
On 3/19/15 12:00 PM, Ge Peng - NOAA Affiliate wrote:
>
> Actually, I think that Jim’s suggestion of developing ACUP - the
> Attribute Conventions for Usage and Provenance is a great idea.
> Hopefully, it will be picked up by someone and developed by the group
> in not so distant future.
>
>
> --- Peng
>
>
> On Thu, Mar 19, 2015 at 11:25 AM, Jim Biard via Esip-documentation
> <esip-documentation at lists.esipfed.org
> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>
> Hi.
>
> I think that Nan's last comment is well worth considering in
> relation to ACDD. Some metadata are focused more on provenance,
> and some are focused on discovery. The primary purpose of ACDD is
> to capture discovery metadata, not provenance metadata. That
> doesn't mean that provenance metadata is unimportant - it can, in
> fact, be critical to proper use, but that doesn't mean that ACDD
> needs to address it.
>
> Perhaps someone should start developing ACUP - the Attribute
> Conventions for Usage and Provenance. (grin)
>
> Grace and peace,
>
> Jim
>
>
> On 3/17/15 11:30 AM, Nan Galbraith via Esip-documentation wrote:
>> Hi Aaron -
>>
>> Having been involved in the oceansites specification on this, I
>> agree that
>> no one solution always works. I adopted (and try to follow, and
>> promote)
>> the NODC templates as closely as possible. Their development of
>> the ancillary
>> variable 'instrument' was brilliant, in my opinion.
>>
>> My datasets range from met files where all the data is from a single
>> logger but each of 6 or 8 variables might need its sensor
>> described at
>> the variable level, to depth merged temperature files where
>> there's only
>> one data variable but 30 instruments (and 4 or 5 different
>> models). In the met
>> files, variable attributes would work, but I use 'instrument'
>> ancillary variables
>> so I can link data variables recorded by a single sensor module
>> (e.g. Inst_HRH
>> is linked to data variables HRH and ATMP).
>>
>> The depth merged files show the real beauty of the instrument
>> variable, though;
>> since it can have a depth dimension, you can describe the
>> different instruments
>> deployed at different depths. There's no good way to do that
>> using attributes, as
>> far as I've seen - lists break down, especially when data is
>> sub-sampled i.e. in thredds.
>>
>> And, although much of this isn't yet at a stage where software
>> can use it, any and all
>> information that you have about the sensors and platform should
>> still be recorded
>> wherever it seems most appropriate to you, since it might
>> otherwise be lost. Re-formatting
>> this kind of info (if and when a useful standard emerges) will be
>> much less effort
>> than trying to track down something that was not recorded because
>> there didn't
>> seem like a standard way to encode it.
>>
>> If ACDD can't find and understand instrument variables yet, maybe
>> that's something
>> that could be looked at in another rev. Or, maybe instruments
>> aren't a common
>> discovery concept - I really don't know.
>>
>> Cheers -
>> Nan
>>
>>
>> On 3/16/15 6:41 PM, Aaron Sweeney via Esip-documentation wrote:
>>> Thanks very much for the guidance, John. I think I may stick
>>> with the NODC or OceanSites platform and instrument variables to
>>> define the particular instances.
>>>
>>> That being said, for my two use cases, I think I can still
>>> capture the type of platform and instrument in the ACDD global
>>> attributes. This amounts to:
>>>
>>> :platform = "DART"
>>> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version
>>> 8.0"
>>> :instrument = "BOTTOM PRESSURE GAUGES"
>>> :instrument_vocabulary = "NASA/GCMD Instrument Keywords.
>>> Version 8.0"
>>>
>>> and
>>>
>>> :platform = "COASTAL STATIONS"
>>> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version
>>> 8.0"
>>> :instrument = "TIDE GAUGES"
>>> :instrument_vocabulary = "NASA/GCMD Instrument Keywords.
>>> Version 8.0"
>>>
>>> Cordially,
>>> Aaron
>>>
>>> On 03/16/2015 02:43 PM, John Graybeal wrote:
>>>> What great questions! The following thoughts are my own, and
>>>> are unrelated to my previous ACDD activities.
>>>>
>>>> Clearly some data files will have data from multiple platforms
>>>> and instruments; whereas others will get all their variables
>>>> from a single platform/instrument; and some will have data so
>>>> heavily processed as to be barely traceable to the source
>>>> platforms/instruments. Each of the specifications is oriented
>>>> toward somewhat different scenarios, and none of them are yet
>>>> great at representing all the scenarios.
>>>>
>>>> Let's consider ACDD first. Note that this specification
>>>> provides _recommendations_ for attributes, not _requirements_;
>>>> and the platform and instrument attributes are only
>>>> 'Suggested'. These give a broad-brush categorization of the
>>>> platform(s) or instrument(s) that collected a data set; if it
>>>> seems reasonable to specify that information for the whole data
>>>> set, use these attributes to do so. (If it isn't
>>>> useful/applicable, then don't worry about these attributes.)
>>>> The most precise way to give that info is to use a controlled
>>>> vocabulary (see
>>>> https://marinemetadata.org/references?filter0=**ALL**&filter1%5B%5D=129&filter1%5B%5D=141
>>>> <https://marinemetadata.org/references?filter0=**ALL**&filter1[]=129&filter1[]=141> for
>>>> a partial list of vocabularies), and typically I'd expect a
>>>> type vocabulary here (with e.g. "CTD", "Mooring") not a
>>>> specific instance.
>>>>
>>>> If you *are* following ACDD, I don't recommend putting the
>>>> names of variables containing information on platforms and
>>>> instruments in the global attributes 'platform' and
>>>> 'instrument'. Without ACDD describing that as a possibility,
>>>> it's highly unlikely any software will know how to treat that,
>>>> and some people will get confused too. The existence of the
>>>> variables will be pretty self-explanatory, so just leave out
>>>> those ACDD attributes. (I see this conflicts with NODC and
>>>> OceanSITES recommendations, to my surprise; perhaps others will
>>>> offer another view here.)
>>>>
>>>> Whether to put the information in a global instrument variable,
>>>> or attributes of each data variable, or both, seems to me a
>>>> matter of circumstances and taste. Let's stipulate the best
>>>> solution is human-recognizable, computer-parseable, extensible,
>>>> and unambiguous. Thus, global attributes are inadequate.
>>>> Assuming a sophisticated data system 5 years from now would
>>>> look in both variables and data attributes, then the model of
>>>> 'one variable per instrument or platform', and association from
>>>> the data variables to those variables or platforms seems best.
>>>>
>>>> But given that the examples in each of the different profiles
>>>> still seems inconsistent, likely you should follow the guidance
>>>> most close to your work. If you aren't particularly close to
>>>> any of them, I'd do it a bit differently still. But I'll hold
>>>> that thought unless you want to extend the discussion in that
>>>> direction.
>>>>
>>>> John
>>>>
>>>>
>>>> On Mar 16, 2015, at 11:56, Aaron Sweeney via Esip-documentation
>>>> <esip-documentation at lists.esipfed.org
>>>> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>>>>
>>>>> Hi, folks,
>>>>>
>>>>> I’m struggling with how to capture platform and
>>>>> instrument information in the appropriate places in netCDF
>>>>> files. The NODC templates suggest creating separate variables
>>>>> for platforms and instruments, with multiple variable
>>>>> attributes (make, model, serial number, precision, accuracy,
>>>>> etc.), and associating these platform and instrument variables
>>>>> with physical variables via an “ancillary_variable” variable
>>>>> attribute attached to the physical variable. Along these
>>>>> lines, SeaDataNet suggests including variable attributes for
>>>>> instrument (sdn_instrument_urn and sdn_instrument_name), and
>>>>> OceanSites suggests either the use of a separate instrument
>>>>> variable (with make, model, serial number attributes) or
>>>>> capturing make, model and serial number as variable attributes
>>>>> contained directly within the relevant physical variable. But
>>>>> ACDD-1.3 only has single /global/ attributes for platform and
>>>>> instrument (and vocabularies). The NODC templates suggest
>>>>> placing the names of the variables containing information on
>>>>> platforms and instruments in the global attributes: instrument
>>>>> and platform. I like NODC’s and OceanSites’ “associative”
>>>>> approach (i.e. “this instrument goes with this physical
>>>>> variable”), but the use of variable names in global attributes
>>>>> conflicts with ACDD-1.3.
>>>>>
>>>>> It seems that in order to comply with ACDD-1.3, I need to
>>>>> capture instruments and platforms at a global level (dropping
>>>>> their association with physical measurement variables), but in
>>>>> order to comply with OceanSites and others, I need to capture
>>>>> instrument and platform information in separate variables and
>>>>> associate these with physical measurement variables via an
>>>>> ancillary_variable attribute.
>>>>>
>>>>> In order to comply with both, in the interest of enabling
>>>>> interoperability, I seem to need to repeat instrument and
>>>>> platform metadata in two different places.
>>>>>
>>>>> Any thoughts or guidance?
>>>>>
>>>>> References:
>>>>> NODC netCDF timeseries template:
>>>>> http://www.nodc.noaa.gov/data/formats/netcdf/v1.1/timeSeriesOrthogonal.cdl
>>>>> OceanSITES Data Format Reference Manual (See Appendix 2 for
>>>>> sensor variables):
>>>>> http://www.oceansites.org/docs/oceansites_data_format_reference_manual.pdf
>>>>> SeaDataNet Data Transports Manual (See Section 4.2.2
>>>>> Co-ordinate Variables):
>>>>> http://www.seadatanet.org/content/download/16251/106283/file/SDN2_D85_WP8_Datafile_formats.pdf
>>>>>
>>>>> Thanks,
>>>>> Aaron
>>
>> --
>> *******************************************************
>> * Nan Galbraith Information Systems Specialist *
>> * Upper Ocean Processes Group Mail Stop 29 *
>> * Woods Hole Oceanographic Institution *
>> * Woods Hole, MA 02543(508) 289-2444 <tel:%28508%29%20289-2444> *
>> *******************************************************
>>
>>
>>
>>
>> _______________________________________________
>> Esip-documentation mailing list
>> Esip-documentation at lists.esipfed.org <mailto:Esip-documentation at lists.esipfed.org>
>> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
> --
> CICS-NC <http://www.cicsnc.org/> Visit us on
> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC
> <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
> 151 Patton Ave, Asheville, NC 28801
> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
> o: +1 828 271 4900 <tel:%2B1%20828%20271%204900>
>
>
>
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> <mailto:Esip-documentation at lists.esipfed.org>
> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
>
>
>
> --
> *Ge Peng, Ph.D*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
> 151 Patton Ave, Asheville, NC 28801
> ge.peng at noaa.gov <mailto:ge.peng at noaa.gov>
> o: +1 828 257 3009
> f: +1 828 257 3002
>
> Following CICS-NC on Facebook <http://www.facebook.com/cicsnc>
>
>
>
>
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org
o: +1 828 271 4900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150319/7071bb72/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 11847 bytes
Desc: not available
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150319/7071bb72/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: aafhaccc.png
Type: image/png
Size: 11847 bytes
Desc: not available
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150319/7071bb72/attachment-0003.png>
More information about the Esip-documentation
mailing list