Commit 7893f0f5 authored by Test Paul's avatar Test Paul

release V.3.01.01

parent 82295313
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using the EbasMetadata object"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The EbasMetadata object defines the syntax of the ebas.io file objects. It is used internally by the io.file objects for creating the output files and parsing input files.\n",
"\n",
"The user migtht, when creating an output file use the object for retrieving information about how to set a specific attribute in the file object in order to create a specifig metadata element in the output file.\n",
"\n",
"Or vice versa, when reading an input file, know which attribute to access when the contents of a specific metadata element in the file is needed.\n",
"\n",
"Please find below some examples how to list this kind of information interactively."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## General setup: imports and instanciation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This part is needed for all the examples below.\n",
"\n",
"Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES\n",
"meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## List all possible metadata elements"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This code sequence lists all possible metadata elements in EBAS. Each line contains the tag (the tag used in the Nasa Ames files) and the key (the key used in the io object's metadata)."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Data definition: datadef\n",
"Set type code: type\n",
"Timezone: timezone\n",
"Timeref: timeref\n",
"File name: filename\n",
"File creation: creation_time\n",
"Startdate: startdate\n",
"Dataset key: setkey\n",
"Revision date: revdate\n",
"Version: revision\n",
"Version description: revdesc\n",
"Statistics: statistics\n",
"Data level: datalevel\n",
"Period code: period\n",
"Resolution code: resolution\n",
"Sample duration: duration\n",
"Orig. time res.: rescode_sample\n",
"Station code: station_code\n",
"Platform code: platform_code\n",
"Station name: station_name\n",
"Station WDCA-ID: station_wdca_id\n",
"Station GAW-ID: station_gaw_id\n",
"Station GAW-Name: station_gaw_name\n",
"Station AIRS-ID: station_airs_id\n",
"Station other IDs: station_other_ids\n",
"Station state/province: station_state_code\n",
"Station land use: station_landuse\n",
"Station setting: station_setting\n",
"Station GAW type: station_gaw_type\n",
"Station WMO region: station_wmo_region\n",
"Station latitude: station_latitude\n",
"Station longitude: station_longitude\n",
"Station altitude: station_altitude\n",
"Measurement latitude: mea_latitude\n",
"Measurement longitude: mea_longitude\n",
"Measurement altitude: mea_altitude\n",
"Measurement height: mea_height\n",
"Regime: regime\n",
"Component: comp_name\n",
"Unit: unit\n",
"Matrix: matrix\n",
"Laboratory code: lab_code\n",
"Instrument type: instr_type\n",
"Instrument name: instr_name\n",
"Instrument manufacturer: instr_manufacturer\n",
"Instrument model: instr_model\n",
"Instrument serial number: instr_serialno\n",
"Sensor type: sensor_type\n",
"Analytical laboratory code: ana_lab_code\n",
"Analytical measurement technique: ana_technique\n",
"Analytical instrument name: ana_instr_name\n",
"Analytical instrument manufacturer: ana_instr_manufacturer\n",
"Analytical instrument model: ana_instr_model\n",
"Analytical instrument serial number: ana_instr_serialno\n",
"Ext. lab. code: ext_lab\n",
"Method ref: method\n",
"Standard method: std_method\n",
"Calibration scale: cal_scale\n",
"Calibration standard ID: cal_std_id\n",
"Secondary standard ID: sec_std_id\n",
"Inlet type: inlet_type\n",
"Inlet description: inlet_desc\n",
"Inlet tube material: inlet_tube_material\n",
"Inlet tube outer diameter: inlet_tube_outerD\n",
"Inlet tube inner diameter: inlet_tube_innerD\n",
"Inlet tube length: inlet_tube_length\n",
"Time from entry inlet line to entry of converter: time_inlet_to_converter\n",
"Duration of stay in converter or bypass line: time_converter_or_bypass_line\n",
"Duration of stay in converter: time_stay_converter\n",
"Converter temperature: converter_temp\n",
"Maintenance description: maintenance_desc\n",
"Flow rate: flow_rate\n",
"Filter face velocity: filter_face_velocity\n",
"Exposed filter area: filter_area\n",
"Filter description: filter_descr\n",
"Medium: medium\n",
"Coating/Solution: coating_solution\n",
"Filter prefiring: filter_prefiring\n",
"Filter conditioning: filter_conditioning\n",
"Filter type: filter_type\n",
"Sample preparation: sample_prep\n",
"Blank correction: blank_corr\n",
"Artifact correction: artifact_corr\n",
"Artifact correction description: artifact_corr_desc\n",
"Charring correction: charring_corr\n",
"Ozone correction: ozone_corr\n",
"Water vapor correction: watervapor_corr\n",
"Zero/span check type: zero_span_type\n",
"Zero/span check interval: zero_span_interval\n",
"Humidity/temperature control: hum_temp_ctrl\n",
"Humidity/temperature control description: hum_temp_ctrl_desc\n",
"Volume std. temperature: vol_std_temp\n",
"Volume std. pressure: vol_std_pressure\n",
"Detection limit: detection_limit\n",
"Detection limit expl.: detection_limit_desc\n",
"Upper range limit: upper_range_limit\n",
"Measurement uncertainty: uncertainty\n",
"Measurement uncertainty expl.: uncertainty_desc\n",
"Zero/negative values code: zero_negative\n",
"Zero/negative values: zero_negative_desc\n",
"Absorption cross section: abs_cross_section\n",
"Mass absorption cross section: mass_abs_cross_section\n",
"Multi-scattering correction factor: multi_scattering_corr_fact\n",
"Maximum attenuation: max_attenuation\n",
"Leakage factor zeta: leakage_factor_zeta\n",
"Compensation threshold attenuation 1: comp_thresh_atten1\n",
"Compensation threshold attenuation 2: comp_thresh_atten2\n",
"Compensation parameter k min: comp_param_kmin\n",
"Compensation parameter k max: comp_param_kmax\n",
"QA measure ID: qm_id\n",
"QA measure description: qm_desc\n",
"QA date: qa_date\n",
"QA outcome: qa_outcome\n",
"QA bias: qa_bias\n",
"QA variability: qa_variability\n",
"QA document name: qa_doc_name\n",
"QA document date: qa_doc_date\n",
"QA document URL: qa_doc_url\n",
"Organization: org\n",
"Frameworks: projects\n",
"Originator: originator\n",
"Submitter: submitter\n",
"Acknowledgement: acknowledgements\n",
"Comment: comment\n"
]
}
],
"source": [
"for elem in meta.metadata:\n",
" if not 'renamed_tag' in elem:\n",
" print \"{:45s} {}\".format(elem['tag']+\":\", elem['key'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Finding the key to a tag"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you see a metadata element in the NASA Ames file and want to access it after reading the file using the ebas.io module, you need to find the element's *key* when knowing the *tag*:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"'instr_manufacturer'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"meta.metadata_tags['Instrument manufacturer']['key']"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"'rescode_sample'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"meta.metadata_tags['Orig. time res.']['key']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Finding the tag to a key"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The other way around, you might want to find the *tag* to a known *key*:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"'Instrument manufacturer'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"meta.metadata_keys['instr_manufacturer']['tag']"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.13"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
......@@ -31,19 +31,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This part is needed for all the examples below."
"This part is needed for all the examples below.\n",
"\n",
"Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0."
]
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES\n",
"meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES)"
"meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')"
]
},
{
......@@ -62,7 +64,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": false
......@@ -113,14 +115,15 @@
"Component: comp_name\n",
"Unit: unit\n",
"Matrix: matrix\n",
"Instrument type: instr_type\n",
"Laboratory code: lab_code\n",
"Instrument type: instr_type\n",
"Instrument name: instr_name\n",
"Instrument manufacturer: instr_manufacturer\n",
"Instrument model: instr_model\n",
"Instrument serial number: instr_serialno\n",
"Analytical measurement technique: ana_technique\n",
"Sensor type: sensor_type\n",
"Analytical laboratory code: ana_lab_code\n",
"Analytical measurement technique: ana_technique\n",
"Analytical instrument name: ana_instr_name\n",
"Analytical instrument manufacturer: ana_instr_manufacturer\n",
"Analytical instrument model: ana_instr_model\n",
......@@ -137,20 +140,27 @@
"Inlet tube outer diameter: inlet_tube_outerD\n",
"Inlet tube inner diameter: inlet_tube_innerD\n",
"Inlet tube length: inlet_tube_length\n",
"Time from entry inlet line to entry of converter: time_inlet_to_converter\n",
"Duration of stay in converter or bypass line: time_converter_or_bypass_line\n",
"Duration of stay in converter: time_stay_converter\n",
"Converter temperature: converter_temp\n",
"Maintenance description: maintenance_desc\n",
"Flow rate: flow_rate\n",
"Filter face velocity: filter_face_velocity\n",
"Exposed filter area: filter_area\n",
"Filter description: filter_descr\n",
"Medium: medium\n",
"Coating/Solution: coating_solution\n",
"Filter prefiring: filter_prefiring\n",
"Filter conditioning: filter_conditioning\n",
"Coating/Solution: coating_solution\n",
"Filter type: filter_type\n",
"Sample preparation: sample_prep\n",
"Blank correction: blank_corr\n",
"Artifact correction: artifact_corr\n",
"Artifact correction description: artifact_corr_desc\n",
"Charring correction: charring_corr\n",
"Ozone correction: ozone_corr\n",
"Water vapor correction: watervapor_corr\n",
"Zero/span check type: zero_span_type\n",
"Zero/span check interval: zero_span_interval\n",
"Humidity/temperature control: hum_temp_ctrl\n",
......@@ -159,16 +169,22 @@
"Volume std. pressure: vol_std_pressure\n",
"Detection limit: detection_limit\n",
"Detection limit expl.: detection_limit_desc\n",
"Upper range limit: upper_range_limit\n",
"Measurement uncertainty: uncertainty\n",
"Measurement uncertainty expl.: uncertainty_desc\n",
"Zero/negative values code: zero_negative\n",
"Zero/negative values: zero_negative_desc\n",
"Absorption cross section: abs_cross_section\n",
"Mass absorption cross section: mass_abs_cross_section\n",
"Multi-scattering correction factor: multi_scattering_corr_fact\n",
"Maximum attenuation: max_attenuation\n",
"Leakage factor zeta: leakage_factor_zeta\n",
"Compensation threshold attenuation 1: comp_thresh_atten1\n",
"Compensation threshold attenuation 2: comp_thresh_atten2\n",
"Compensation parameter k min: comp_param_kmin\n",
"Compensation parameter k max: comp_param_kmax\n",
"QA measure ID: qm_id\n",
"QA measure description: qm_desc\n",
"QA measure document name: qm_doc_name\n",
"QA measure document date: qm_doc_date\n",
"QA measure document URL: qm_doc_url\n",
"QA date: qa_date\n",
"QA outcome: qa_outcome\n",
"QA bias: qa_bias\n",
......@@ -207,7 +223,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 5,
"metadata": {
"collapsed": false
},
......@@ -218,7 +234,7 @@
"'instr_manufacturer'"
]
},
"execution_count": 8,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
......@@ -229,7 +245,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 6,
"metadata": {
"collapsed": false,
"scrolled": true
......@@ -241,7 +257,7 @@
"'rescode_sample'"
]
},
"execution_count": 9,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
......@@ -266,7 +282,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 7,
"metadata": {
"collapsed": false
},
......@@ -277,7 +293,7 @@
"'Instrument manufacturer'"
]
},
"execution_count": 11,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
......@@ -285,6 +301,15 @@
"source": [
"meta.metadata_keys['instr_manufacturer']['tag']"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
......@@ -303,7 +328,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.11"
"version": "2.7.13"
}
},
"nbformat": 4,
......
......@@ -208,17 +208,18 @@ div#notebook {
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This part is needed for all the examples below.</p>
<p>Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">ebas.io.ebasmetadata</span> <span class="kn">import</span> <span class="n">EbasMetadata</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span>
<span class="n">meta</span> <span class="o">=</span> <span class="n">EbasMetadata</span><span class="p">(</span><span class="s1">&#39;EBAS_1.1&#39;</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span><span class="p">)</span>
<span class="n">meta</span> <span class="o">=</span> <span class="n">EbasMetadata</span><span class="p">(</span><span class="s1">&#39;EBAS_1.1&#39;</span><span class="p">,</span> <span class="n">EBAS_IOFORMAT_NASA_AMES</span><span class="p">,</span> <span class="n">data_level</span><span class="o">=</span><span class="s1">&#39;0&#39;</span><span class="p">)</span>
</pre></div>
</div>
......@@ -247,7 +248,7 @@ div#notebook {
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">elem</span> <span class="ow">in</span> <span class="n">meta</span><span class="o">.</span><span class="n">metadata</span><span class="p">:</span>
......@@ -306,14 +307,15 @@ Regime: regime
Component: comp_name
Unit: unit
Matrix: matrix
Instrument type: instr_type
Laboratory code: lab_code
Instrument type: instr_type
Instrument name: instr_name
Instrument manufacturer: instr_manufacturer
Instrument model: instr_model
Instrument serial number: instr_serialno
Analytical measurement technique: ana_technique
Sensor type: sensor_type
Analytical laboratory code: ana_lab_code
Analytical measurement technique: ana_technique
Analytical instrument name: ana_instr_name
Analytical instrument manufacturer: ana_instr_manufacturer
Analytical instrument model: ana_instr_model
......@@ -330,20 +332,27 @@ Inlet tube material: inlet_tube_material
Inlet tube outer diameter: inlet_tube_outerD
Inlet tube inner diameter: inlet_tube_innerD
Inlet tube length: inlet_tube_length
Time from entry inlet line to entry of converter: time_inlet_to_converter
Duration of stay in converter or bypass line: time_converter_or_bypass_line
Duration of stay in converter: time_stay_converter
Converter temperature: converter_temp
Maintenance description: maintenance_desc
Flow rate: flow_rate
Filter face velocity: filter_face_velocity
Exposed filter area: filter_area
Filter description: filter_descr
Medium: medium
Coating/Solution: coating_solution
Filter prefiring: filter_prefiring
Filter conditioning: filter_conditioning
Coating/Solution: coating_solution
Filter type: filter_type
Sample preparation: sample_prep
Blank correction: blank_corr
Artifact correction: artifact_corr
Artifact correction description: artifact_corr_desc
Charring correction: charring_corr
Ozone correction: ozone_corr
Water vapor correction: watervapor_corr
Zero/span check type: zero_span_type
Zero/span check interval: zero_span_interval
Humidity/temperature control: hum_temp_ctrl
......@@ -352,16 +361,22 @@ Volume std. temperature: vol_std_temp
Volume std. pressure: vol_std_pressure
Detection limit: detection_limit
Detection limit expl.: detection_limit_desc
Upper range limit: upper_range_limit
Measurement uncertainty: uncertainty
Measurement uncertainty expl.: uncertainty_desc
Zero/negative values code: zero_negative
Zero/negative values: zero_negative_desc
Absorption cross section: abs_cross_section
Mass absorption cross section: mass_abs_cross_section
Multi-scattering correction factor: multi_scattering_corr_fact
Maximum attenuation: max_attenuation
Leakage factor zeta: leakage_factor_zeta
Compensation threshold attenuation 1: comp_thresh_atten1
Compensation threshold attenuation 2: comp_thresh_atten2
Compensation parameter k min: comp_param_kmin
Compensation parameter k max: comp_param_kmax
QA measure ID: qm_id
QA measure description: qm_desc
QA measure document name: qm_doc_name
QA measure document date: qm_doc_date
QA measure document URL: qm_doc_url
QA date: qa_date
QA outcome: qa_outcome
QA bias: qa_bias
......@@ -404,7 +419,7 @@ Comment: comment
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="prompt input_prompt">In&nbsp;[5]:</div>