Skip to content
Snippets Groups Projects
Commit 0e1fdbf5 authored by Paul Eckhardt's avatar Paul Eckhardt
Browse files

add netcdf tags

parent bba1cd7a
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Examples/Doc/Notebooks/EbasMetadata.ipynb
+ 139
− 142
View file @ 0e1fdbf5
%% Cell type:markdown id: tags:
# Using the EbasMetadata object
%% Cell type:markdown id: tags:
The EbasMetadata object defines the syntax of the ebas.io file objects. It is used internally by the io.file objects for creating the output files and parsing input files.
The user migtht, when creating an output file use the object for retrieving information about how to set a specific attribute in the file object in order to create a specifig metadata element in the output file.
Or vice versa, when reading an input file, know which attribute to access when the contents of a specific metadata element in the file is needed.
Please find below some examples how to list this kind of information interactively.
%% Cell type:markdown id: tags:
## General setup: imports and instanciation
%% Cell type:markdown id: tags:
This part is needed for all the examples below.
Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0.
%% Cell type:code id: tags:
``` python
from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES
meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')
```
%% Cell type:markdown id: tags:
## List all possible metadata elements
%% Cell type:markdown id: tags:
This code sequence lists all possible metadata elements in EBAS. Each line contains the tag (the tag used in the Nasa Ames files) and the key (the key used in the io object's metadata).
This code sequence lists all possible metadata elements in EBAS. Each line contains the Nasa Ames tag (the tag used in the Nasa Ames files), the NetCDF tag (attribute name used in NetCDF files) and the object key (the key used in the io object's metadata).
We only list metadata which are allowed either as global metadata (main) or as variable metadata (vname)
%% Cell type:code id: tags:
``` python
print(f'{"Nasa Ames tag":45s} {"NetCDF attribute":35s} Object key')
print(f'{"-"*45} {"-"*35} ----------')
for elem in [x for x in meta.metadata if x['main'] or x['vname']]:
if not 'renamed_tag' in elem:
print("{:45s} {}".format(elem['tag']+":", elem['key']))
print(f'{elem["tag"]:45s} {elem["nc_tag"]:35s} {elem["key"]}')
```
%% Output
Data definition: datadef
Data license: license
Citation: citation
Set type code: type
Timezone: timezone
Timeref: timeref
File name: filename
Represents DOI: doi
Contains data from DOI: doi_list
File creation: creation_time
Export state: export_state
Export filter: export_filter
Startdate: startdate
Revision date: revdate
Version: revision
Version description: revdesc
Input dataset: input_dataset
Software: software
Statistics: statistics
Data level: datalevel
Period code: period
Resolution code: resolution
Sample duration: duration
Orig. time res.: rescode_sample
Station code: station_code
Platform code: platform_code
Station name: station_name
Station WDCA-ID: station_wdca_id
Station GAW-ID: station_gaw_id
Station GAW-Name: station_gaw_name
Station AIRS-ID: station_airs_id
Station other IDs: station_other_ids
Station state/province: station_state_code
Station land use: station_landuse
Station setting: station_setting
Station GAW type: station_gaw_type
Station WMO region: station_wmo_region
Station latitude: station_latitude
Station longitude: station_longitude
Station altitude: station_altitude
Measurement latitude: mea_latitude
Measurement longitude: mea_longitude
Measurement altitude: mea_altitude
Measurement height: mea_height
Regime: regime
Component: comp_name
Unit: unit
Matrix: matrix
Laboratory code: lab_code
Instrument type: instr_type
Instrument name: instr_name
Instrument manufacturer: instr_manufacturer
Instrument model: instr_model
Instrument serial number: instr_serialno
Sensor type: sensor_type
Analytical laboratory code: ana_lab_code
Analytical measurement technique: ana_technique
Analytical instrument name: ana_instr_name
Analytical instrument manufacturer: ana_instr_manufacturer
Analytical instrument model: ana_instr_model
Analytical instrument serial number: ana_instr_serialno
Ext. lab. code: ext_lab
Method ref: method
Standard method: std_method
Calibration scale: cal_scale
Calibration standard ID: cal_std_id
Secondary standard ID: sec_std_id
Inlet type: inlet_type
Inlet description: inlet_desc
Inlet tube material: inlet_tube_material
Inlet tube outer diameter: inlet_tube_outerD
Inlet tube inner diameter: inlet_tube_innerD
Inlet tube length: inlet_tube_length
Time from entry inlet line to entry of converter: time_inlet_to_converter
Duration of stay in converter or bypass line: time_converter_or_bypass_line
Duration of stay in converter: time_stay_converter
Converter temperature: converter_temp
Maintenance description: maintenance_desc
Flow rate: flow_rate
Filter face velocity: filter_face_velocity
Exposed filter area: filter_area
Filter description: filter_descr
Medium: medium
Coating/Solution: coating_solution
Filter prefiring: filter_prefiring
Filter conditioning: filter_conditioning
Filter type: filter_type
Sample preparation: sample_prep
Blank correction: blank_corr
Artifact correction: artifact_corr
Artifact correction description: artifact_corr_desc
Charring correction: charring_corr
Ozone correction: ozone_corr
Water vapor correction: watervapor_corr
Zero/span check type: zero_span_type
Zero/span check interval: zero_span_interval
Humidity/temperature control: hum_temp_ctrl
Humidity/temperature control description: hum_temp_ctrl_desc
Volume std. temperature: vol_std_temp
Volume std. pressure: vol_std_pressure
Detection limit: detection_limit
Detection limit expl.: detection_limit_desc
Upper range limit: upper_range_limit
Measurement uncertainty: uncertainty
Measurement uncertainty expl.: uncertainty_desc
Zero/negative values code: zero_negative
Zero/negative values: zero_negative_desc
Absorption cross section: abs_cross_section
Mass absorption cross section: mass_abs_cross_section
Multi-scattering correction factor: multi_scattering_corr_fact
Maximum attenuation: max_attenuation
Leakage factor zeta: leakage_factor_zeta
Compensation threshold attenuation 1: comp_thresh_atten1
Compensation threshold attenuation 2: comp_thresh_atten2
Compensation parameter k min: comp_param_kmin
Compensation parameter k max: comp_param_kmax
Coincidence correction: coincidence_corr
Charge type: charge_type
Inlet diffusion loss data: inlet_diffusion_loss_data
CPC default pulse width: cpc_default_pulse_width
QA measure ID: qm_id
QA measure description: qm_desc
QA date: qa_date
QA outcome: qa_outcome
QA bias: qa_bias
QA variability: qa_variability
QA document name: qa_doc_name
QA document date: qa_doc_date
QA document URL: qa_doc_url
Originator: originator
Submitter: submitter
Acknowledgement: acknowledgements
Comment: comment
Nasa Ames tag NetCDF attribute Object key
--------------------------------------------- ----------------------------------- ----------
Data definition data_definition datadef
Data license data_license license
Citation citation citation
Set type code set_type_code type
Timezone timezone timezone
Timeref timeref timeref
File name file_name filename
Represents DOI represents_doi doi
Contains data from DOI contains_doi doi_list
File creation file_creation creation_time
Export state export_state export_state
Export filter export_filter export_filter
Startdate startdate startdate
Revision date revision_date revdate
Version version revision
Version description version_description revdesc
Input dataset input_dataset input_dataset
Software software software
Statistics statistics statistics
Data level data_level datalevel
Period code period_code period
Resolution code resolution_code resolution
Sample duration sample_duration duration
Orig. time res. orig_time_res rescode_sample
Station code station_code station_code
Platform code platform_code platform_code
Station name station_name station_name
Station WDCA-ID station_wdca_id station_wdca_id
Station GAW-ID station_gaw_id station_gaw_id
Station GAW-Name station_gaw_name station_gaw_name
Station AIRS-ID station_airs_id station_airs_id
Station other IDs station_other_ids station_other_ids
Station state/province station_state_province station_state_code
Station land use station_land_use station_landuse
Station setting station_setting station_setting
Station GAW type station_gaw_type station_gaw_type
Station WMO region station_wmo_region station_wmo_region
Station latitude station_latitude station_latitude
Station longitude station_longitude station_longitude
Station altitude station_altitude station_altitude
Measurement latitude measuremenet_latitude mea_latitude
Measurement longitude measurement_longitude mea_longitude
Measurement altitude measurement_altitude mea_altitude
Measurement height measurement_height mea_height
Regime regime regime
Component component comp_name
Unit unit unit
Matrix matrix matrix
Laboratory code laboratory_code lab_code
Instrument type instrument_type instr_type
Instrument name instrument_name instr_name
Instrument manufacturer instrument_manufacturer instr_manufacturer
Instrument model instrument_model instr_model
Instrument serial number instrument_serial_number instr_serialno
Sensor type sensor_type sensor_type
Analytical laboratory code analytical_laboratory_code ana_lab_code
Analytical measurement technique analytical_measurement_technique ana_technique
Analytical instrument name analytical_instrument_name ana_instr_name
Analytical instrument manufacturer analytical_instrument_manufacturer ana_instr_manufacturer
Analytical instrument model analytical_instrument_model ana_instr_model
Analytical instrument serial number analytical_instrument_serial_number ana_instr_serialno
Ext. lab. code ext_lab_code ext_lab
Method ref method_ref method
Standard method standard_method std_method
Calibration scale calibration_scale cal_scale
Calibration standard ID calibration_standard_id cal_std_id
Secondary standard ID secondary_standard_id sec_std_id
Inlet type inlet_type inlet_type
Inlet description inlet_description inlet_desc
Inlet tube material inlet_tube_material inlet_tube_material
Inlet tube outer diameter inlet_tube_outer_diameter inlet_tube_outerD
Inlet tube inner diameter inlet_tube_inner_diameter inlet_tube_innerD
Inlet tube length inlet_tube_length inlet_tube_length
Time from entry inlet line to entry of converter time_from_entry_inlet_line_to_entry_of_converter time_inlet_to_converter
Duration of stay in converter or bypass line duration_of_stay_in_converter_or_bypass_line time_converter_or_bypass_line
Duration of stay in converter duration_of_stay_in_converter time_stay_converter
Converter temperature converter_temperature converter_temp
Maintenance description maintenance_description maintenance_desc
Flow rate flow_rate flow_rate
Filter face velocity filter_face_velocity filter_face_velocity
Exposed filter area exposed_filter_area filter_area
Filter description filter_description filter_descr
Medium medium medium
Coating/Solution coating_solution coating_solution
Filter prefiring filter_prefiring filter_prefiring
Filter conditioning filter_conditioning filter_conditioning
Filter type filter_type filter_type
Sample preparation sample_preparation sample_prep
Blank correction blank_correction blank_corr
Artifact correction artifact_correction artifact_corr
Artifact correction description artifact_correction_description artifact_corr_desc
Charring correction charring_correction charring_corr
Ozone correction ozone_correction ozone_corr
Water vapor correction water_vapor_correction watervapor_corr
Zero/span check type zero_span_check_type zero_span_type
Zero/span check interval zero_span_check_interval zero_span_interval
Humidity/temperature control humidity_temperaure_control hum_temp_ctrl
Humidity/temperature control description humidity_temperaure_control_description hum_temp_ctrl_desc
Volume std. temperature volume_std_temperature vol_std_temp
Volume std. pressure volume_std_pressure vol_std_pressure
Detection limit detection_limit detection_limit
Detection limit expl. detection_limit_expl detection_limit_desc
Upper range limit upper_range_limit upper_range_limit
Measurement uncertainty measurement_uncertainty uncertainty
Measurement uncertainty expl. measurement_uncertainty_expl uncertainty_desc
Zero/negative values code zero_negative_values_code zero_negative
Zero/negative values zero_negative_values zero_negative_desc
Absorption cross section absorption_cross_section abs_cross_section
Mass absorption cross section mass_absorption_cross_section mass_abs_cross_section
Multi-scattering correction factor multi_scattering_correction_factor multi_scattering_corr_fact
Maximum attenuation maximum_attenuation max_attenuation
Leakage factor zeta leakage_factor_zeta leakage_factor_zeta
Compensation threshold attenuation 1 compensation_threshold_attenuation_1 comp_thresh_atten1
Compensation threshold attenuation 2 compensation_threshold_attenuation_2 comp_thresh_atten2
Compensation parameter k min compensation_parameter_k_min comp_param_kmin
Compensation parameter k max compensation_parameter_k_max comp_param_kmax
Coincidence correction coincidence_correction coincidence_corr
Charge type charge_type charge_type
Inlet diffusion loss data inlet_diffusion_loss_data inlet_diffusion_loss_data
CPC default pulse width cpc_default_pulse_width cpc_default_pulse_width
QA measure ID qa_measure_id qm_id
QA measure description qa_measure_description qm_desc
QA date qa_date qa_date
QA outcome qa_outcome qa_outcome
QA bias qa_bias qa_bias
QA variability qa_variability qa_variability
QA document name qa_document_name qa_doc_name
QA document date qa_document_date qa_doc_date
QA document URL qa_document_url qa_doc_url
Originator originator originator
Submitter submitter submitter
Acknowledgement acknowledgement acknowledgements
Comment comment comment
%% Cell type:markdown id: tags:
## List only metadata which are mandatory when writing files
Here we restrict to _global metadata_ (```main```) which are _mandatory when writing_ (```& 2```) a file:
%% Cell type:code id: tags:
``` python
for elem in [x for x in meta.metadata if x['main'] & 2]:
if not 'renamed_tag' in elem:
print("{:45s} {}".format(elem['tag']+":", elem['key']))
```
%% Output
Data definition: datadef
Set type code: type
Timezone: timezone
File name: filename
File creation: creation_time
Startdate: startdate
Revision date: revdate
Data level: datalevel
Period code: period
Resolution code: resolution
Station code: station_code
Platform code: platform_code
Regime: regime
Component: comp_name
Matrix: matrix
Laboratory code: lab_code
Instrument type: instr_type
Instrument name: instr_name
Method ref: method
Originator: originator
Submitter: submitter
%% Cell type:markdown id: tags:
#### specifications for cardinality
- main: specifies the cardinality of the element in the main header (NNCOM lines)
Bitfield:
0 not allowed
1 allowed but not mandatory
2 mandatory on export
4 mandatory on import
8 critical on import (if missing, exit after reading header)
- vname: specifies the cardinality of the element in the vname line.
Bitfield:
0 not allowed
1 allowed but not mandatory
2 mandatory on export (not used)
4 mandatory on import
8 critical on import (if missing, exit after reading header)
%% Cell type:markdown id: tags:
## Finding the key to a tag
%% Cell type:markdown id: tags:
If you see a metadata element in the NASA Ames file and want to access it after reading the file using the ebas.io module, you need to find the element's *key* when knowing the *tag*:
%% Cell type:code id: tags:
``` python
meta.metadata_tags['Instrument manufacturer']['key']
```
%% Output
'instr_manufacturer'
%% Cell type:code id: tags:
``` python
meta.metadata_tags['Orig. time res.']['key']
```
%% Output
'rescode_sample'
%% Cell type:markdown id: tags:
## Finding the tag to a key
%% Cell type:markdown id: tags:
The other way around, you might want to find the *tag* to a known *key*:
%% Cell type:code id: tags:
``` python
meta.metadata_keys['instr_manufacturer']['tag']
```
%% Output
'Instrument manufacturer'
%% Cell type:code id: tags:
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment