Commit 7893f0f5 authored by Test Paul's avatar Test Paul
Browse files

release V.3.01.01

parent 82295313
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using the EbasMetadata object"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The EbasMetadata object defines the syntax of the ebas.io file objects. It is used internally by the io.file objects for creating the output files and parsing input files.\n",
"\n",
"The user migtht, when creating an output file use the object for retrieving information about how to set a specific attribute in the file object in order to create a specifig metadata element in the output file.\n",
"\n",
"Or vice versa, when reading an input file, know which attribute to access when the contents of a specific metadata element in the file is needed.\n",
"\n",
"Please find below some examples how to list this kind of information interactively."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## General setup: imports and instanciation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This part is needed for all the examples below.\n",
"\n",
"Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES\n",
"meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## List all possible metadata elements"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This code sequence lists all possible metadata elements in EBAS. Each line contains the tag (the tag used in the Nasa Ames files) and the key (the key used in the io object's metadata)."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Data definition: datadef\n",
"Set type code: type\n",
"Timezone: timezone\n",
"Timeref: timeref\n",
"File name: filename\n",
"File creation: creation_time\n",
"Startdate: startdate\n",
"Dataset key: setkey\n",
"Revision date: revdate\n",
"Version: revision\n",
"Version description: revdesc\n",
"Statistics: statistics\n",
"Data level: datalevel\n",
"Period code: period\n",
"Resolution code: resolution\n",
"Sample duration: duration\n",
"Orig. time res.: rescode_sample\n",
"Station code: station_code\n",
"Platform code: platform_code\n",
"Station name: station_name\n",
"Station WDCA-ID: station_wdca_id\n",
"Station GAW-ID: station_gaw_id\n",
......@@ -18,15 +18,17 @@
%% Cell type:markdown id: tags:
This part is needed for all the examples below.
Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0.
%% Cell type:code id: tags:
``` python
from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES
meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES)
meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')
```
%% Cell type:markdown id: tags:
## List all possible metadata elements
......@@ -86,5 +88,11 @@
```
%%%% Output: execute_result
'Instrument manufacturer'
%% Cell type:code id: tags:
``` python
```
......
......@@ -208,17 +208,18 @@ div#notebook {
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This part is needed for all the examples below.</p>
<p>Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">ebas.io.ebasmetadata</span> <span class="kn">import</span> <span class="n">EbasMetadata</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span>
<span class="n">meta</span> <span class="o">=</span> <span class="n">EbasMetadata</span><span class="p">(</span><span class="s1">&#39;EBAS_1.1&#39;</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span><span class="p">)</span>
<span class="n">meta</span> <span class="o">=</span> <span class="n">EbasMetadata</span><span class="p">(</span><span class="s1">&#39;EBAS_1.1&#39;</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span><span class="p">,</span> <span class="n">data_level</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">)</span>
</pre></div>
</div>
......@@ -247,7 +248,7 @@ div#notebook {
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">elem</span> <span class="ow">in</span> <span class="n">meta</span><span class="o">.</span><span class="n">metadata</span><span class="p">:</span>
......@@ -306,14 +307,15 @@ Regime: regime
Component: comp_name
Unit: unit
Matrix: matrix
Instrument type: instr_type
Laboratory code: lab_code
Instrument type: instr_type
Instrument name: instr_name
Instrument manufacturer: instr_manufacturer
Instrument model: instr_model
Instrument serial number: instr_serialno
Analytical measurement technique: ana_technique
Sensor type: sensor_type
Analytical laboratory code: ana_lab_code
Analytical measurement technique: ana_technique
Analytical instrument name: ana_instr_name
Analytical instrument manufacturer: ana_instr_manufacturer
Analytical instrument model: ana_instr_model
......@@ -330,20 +332,27 @@ Inlet tube material: inlet_tube_material
Inlet tube outer diameter: inlet_tube_outerD
Inlet tube inner diameter: inlet_tube_innerD
Inlet tube length: inlet_tube_length
Time from entry inlet line to entry of converter: time_inlet_to_converter
Duration of stay in converter or bypass line: time_converter_or_bypass_line
Duration of stay in converter: time_stay_converter
Converter temperature: converter_temp
Maintenance description: maintenance_desc
Flow rate: flow_rate
Filter face velocity: filter_face_velocity
Exposed filter area: filter_area
Filter description: filter_descr
Medium: medium
Coating/Solution: coating_solution
Filter prefiring: filter_prefiring
Filter conditioning: filter_conditioning
Coating/Solution: coating_solution
Filter type: filter_type
Sample preparation: sample_prep
Blank correction: blank_corr
Artifact correction: artifact_corr
Artifact correction description: artifact_corr_desc
Charring correction: charring_corr
Ozone correction: ozone_corr
Water vapor correction: watervapor_corr
Zero/span check type: zero_span_type
Zero/span check interval: zero_span_interval
Humidity/temperature control: hum_temp_ctrl
......@@ -352,16 +361,22 @@ Volume std. temperature: vol_std_temp
Volume std. pressure: vol_std_pressure
Detection limit: detection_limit
Detection limit expl.: detection_limit_desc
Upper range limit: upper_range_limit
Measurement uncertainty: uncertainty
Measurement uncertainty expl.: uncertainty_desc
Zero/negative values code: zero_negative
Zero/negative values: zero_negative_desc
Absorption cross section: abs_cross_section
Mass absorption cross section: mass_abs_cross_section
Multi-scattering correction factor: multi_scattering_corr_fact
Maximum attenuation: max_attenuation
Leakage factor zeta: leakage_factor_zeta
Compensation threshold attenuation 1: comp_thresh_atten1
Compensation threshold attenuation 2: comp_thresh_atten2
Compensation parameter k min: comp_param_kmin
Compensation parameter k max: comp_param_kmax
QA measure ID: qm_id
QA measure description: qm_desc
QA measure document name: qm_doc_name
QA measure document date: qm_doc_date
QA measure document URL: qm_doc_url
QA date: qa_date
QA outcome: qa_outcome
QA bias: qa_bias
......@@ -404,7 +419,7 @@ Comment: comment
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">meta</span><span class="o">.</span><span class="n">metadata_tags</span><span class="p">[</span><span class="s1">&#39;Instrument manufacturer&#39;</span><span class="p">][</span><span class="s1">&#39;key&#39;</span><span class="p">]</span>
......@@ -418,7 +433,7 @@ Comment: comment
<div class="output">
<div class="output_area"><div class="prompt output_prompt">Out[8]:</div>
<div class="output_area"><div class="prompt output_prompt">Out[5]:</div>
<div class="output_text output_subarea output_execute_result">
......@@ -433,7 +448,7 @@ Comment: comment
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">meta</span><span class="o">.</span><span class="n">metadata_tags</span><span class="p">[</span><span class="s1">&#39;Orig. time res.&#39;</span><span class="p">][</span><span class="s1">&#39;key&#39;</span><span class="p">]</span>
......@@ -447,7 +462,7 @@ Comment: comment
<div class="output">
<div class="output_area"><div class="prompt output_prompt">Out[9]:</div>
<div class="output_area"><div class="prompt output_prompt">Out[6]:</div>
<div class="output_text output_subarea output_execute_result">
......@@ -481,7 +496,7 @@ Comment: comment
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">meta</span><span class="o">.</span><span class="n">metadata_keys</span><span class="p">[</span><span class="s1">&#39;instr_manufacturer&#39;</span><span class="p">][</span><span class="s1">&#39;tag&#39;</span><span class="p">]</span>
......@@ -495,7 +510,7 @@ Comment: comment
<div class="output">
<div class="output_area"><div class="prompt output_prompt">Out[11]:</div>
<div class="output_area"><div class="prompt output_prompt">Out[7]:</div>
<div class="output_text output_subarea output_execute_result">
......@@ -507,6 +522,19 @@ Comment: comment
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span>
</pre></div>
</div>
</div>
</div>
</div>
</div>
</div>
......
......@@ -112,7 +112,7 @@ def add_private_args(parser, cmdline): # pylint: disable=W0613
parser_output_group = parser.add_argument_group('output options')
parser_output_group.add_argument(
'--format', type=_parse_format_arg, default=EbasNasaAmes,
'--format', type=_parse_format_arg, default=EBAS_IOFORMAT_NASA_AMES,
help='file format for output: (NasaAmes, CSV, XML, NetCDF, OPeNDAP), '
'(default: NasaAmes)')
parser_target_group = parser_output_group.add_mutually_exclusive_group()
......@@ -201,6 +201,7 @@ def ebas_convert(cmdline):
exitcode = 0 # number of failed files (max 255)
indexdb = open_indexdb(args.fileindex, logger)
print args.format
fileset = EbasIOResultSet(
outformat=args.format, createfiles=args.createfiles,
destdir=args.destdir, xmlwrap=args.xmlwrap, flags=args.flags,
......@@ -258,4 +259,4 @@ EbasCommandline(
custom_args=['CONFIG', 'LOGGING', 'TIME_CRIT'],
private_args=add_private_args,
help_description='%(prog)s example for converting a NasaAmes datafile to '
'different dataformats', version=__version__)
'different dataformats', version=__version__).run()
......@@ -157,4 +157,4 @@ EbasCommandline(
custom_args=['CONFIG', 'LOGGING', 'TIME_CRIT'],
private_args=add_private_args,
help_description='%(prog)s example for reading a NasaAmes datafile.',
version=__version__)
version=__version__).run()
......@@ -60,7 +60,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='NILU', PS_ORG_UNIT='Atmosphere and Climate Department',
PS_ADDR_LINE1='Instituttveien 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='2007', PS_ADDR_CITY='Kjeller',
PS_ADDR_COUNTRY='Norway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
nas.metadata.originator.append(
DataObject(
PS_LAST_NAME=u'Tørseth', PS_FIRST_NAME='Kjetil', PS_EMAIL='kt@nilu.no',
......@@ -68,7 +70,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='NILU', PS_ORG_UNIT='Atmosphere and Climate Department',
PS_ADDR_LINE1='Instituttveien 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='2007', PS_ADDR_CITY='Kjeller',
PS_ADDR_COUNTRY='Norway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
# Data Submitters (contact for data technical issues)
nas.metadata.submitter = []
......@@ -79,7 +83,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='NILU', PS_ORG_UNIT='Atmosphere and Climate Department',
PS_ADDR_LINE1='Instituttveien 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='2007', PS_ADDR_CITY='Kjeller',
PS_ADDR_COUNTRY='Norway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
# Station metadata
nas.metadata.station_code = 'NO0042G'
......
......@@ -54,7 +54,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='NILU', PS_ORG_UNIT='Atmosphere and Climate Department',
PS_ADDR_LINE1='Instituttveien 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='2007', PS_ADDR_CITY='Kjeller',
PS_ADDR_COUNTRY='Norway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
nas.metadata.originator.append(
DataObject(
PS_LAST_NAME=u'Someone', PS_FIRST_NAME='Else',
......@@ -63,7 +65,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='WOW', PS_ORG_UNIT='Super interesting division',
PS_ADDR_LINE1='Street 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='X-9999', PS_ADDR_CITY='Paradise',
PS_ADDR_COUNTRY='Noway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
# Data Submitters (contact for data technical issues)
nas.metadata.submitter = []
......@@ -75,7 +79,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='NILU', PS_ORG_UNIT='Atmosphere and Climate Department',
PS_ADDR_LINE1='Instituttveien 18', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='2007', PS_ADDR_CITY='Kjeller',
PS_ADDR_COUNTRY='Norway'))
PS_ADDR_COUNTRY='Norway',
PS_ORCID=None,
))
# Station metadata
nas.metadata.station_code = 'NO0001R'
......
......@@ -149,7 +149,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
nas.metadata.originator.append(
DataObject(
PS_LAST_NAME=u'Vratolis', PS_FIRST_NAME='Stergios', PS_EMAIL='vratolis@ipta.demokritos.gr',
......@@ -157,7 +159,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
# Data Submitters (contact for data technical issues)
nas.metadata.submitter = []
......@@ -168,7 +172,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
#nas.metadata.qm_id=''
......
......@@ -148,7 +148,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
nas.metadata.originator.append(
DataObject(
PS_LAST_NAME=u'Vratolis', PS_FIRST_NAME='Stergios', PS_EMAIL='vratolis@ipta.demokritos.gr',
......@@ -156,7 +158,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
# Data Submitters (contact for data technical issues)
nas.metadata.submitter = []
......@@ -167,7 +171,9 @@ def set_fileglobal_metadata(nas):
PS_ORG_ACR='DEM', PS_ORG_UNIT='ERL',
PS_ADDR_LINE1='Ag.Paraskevi', PS_ADDR_LINE2=None,
PS_ADDR_ZIP='15310', PS_ADDR_CITY='Athens',
PS_ADDR_COUNTRY='Greece'))
PS_ADDR_COUNTRY='Greece',
PS_ORCID=None,
))
#nas.metadata.qm_id=''
......
......@@ -138,4 +138,4 @@ EbasCommandline(
custom_args=['CONFIG', 'LOGGING', 'TIME_CRIT'],
private_args=add_private_args,
help_description='%(prog)s example for reading a NasaAmes datafile.',
version=__version__)
version=__version__).run()
......@@ -81,4 +81,4 @@ EbasCommandline(
custom_args=['CONFIG', 'LOGGING', 'TIME_CRIT'],
private_args=add_private_args,
help_description='%(prog)s example for reformatting a NasaAmes datafile.',
version=__version__)
version=__version__).run()
# EBAS IO
ebas-io is a python package for reading/writing NASA-Ames 1001 files with EBAS extensions (EBAS NASA-Ames).
> For more information, please vistsi the [Wiki pages](../../wikis/home).
> For more information, please vistsi the [Wiki pages](../../../wikis/home).
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment