diff --git a/Examples/Doc/Notebooks/.ipynb_checkpoints/Dataset Characteristics-checkpoint.ipynb b/Examples/Doc/Notebooks/.ipynb_checkpoints/Dataset Characteristics-checkpoint.ipynb
deleted file mode 100644
index d0c6ed2dee1c3d64e9a8e27817468a32b93eb2a1..0000000000000000000000000000000000000000
--- a/Examples/Doc/Notebooks/.ipynb_checkpoints/Dataset Characteristics-checkpoint.ipynb
+++ /dev/null
@@ -1,1251 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# TOC\n",
- "\n",
- "* __[Dataset Characteristics](#Dataset-Characteristics)__\n",
- "* __[How to handle dataset characteristics in ebas-io](#How-to-handle-dataset-characteristics-in-ebas-io)__\n",
- " * __[Writing files with variables using dataset characteristics](#Writing-files-with-variables-using-dataset-characteristics)__\n",
- " * __[Preparations](#Preparations:)__\n",
- " * __[Add variables with characteristics](#Add-variables-with-characteristics)__\n",
- " * __[Putting it all together](#Putting-it-all-together:)__\n",
- " * __[Output as a real multidimensional variable](#Output-as-a-real-multidimensional-variable:)__\n",
- " * __[Reading files with variables using dataset characteristics](#Reading-files-with-variables-using-dataset-characteristics)__\n",
- " * __[Inspecting the characteristics for each variable](#Inspecting-the-characteristics-for-each-variable:)__\n",
- " * __[Convenience methods](#Convenience-methods)__\n",
- " * __[Methods of the DatasetCharacteristicList class](#Methods-of-the-DatasetCharacteristicList-class)__\n",
- " * __[as_dict()](#as_dict())__\n",
- " * __[as_str()](#as_str())__\n",
- " * __[dc_by_ct_type()](#dc_by_ct_type())__\n",
- " * __[Methods of the DatasetCharacteristic class](#Methods-of-the-DatasetCharacteristic-class)__\n",
- " * __[tuple()](#tuple())__\n",
- " * __[value and unit](#value-and-unit)__\n",
- " * __[value_string()](#value_string())__"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Dataset Characteristics\n",
- "\n",
- "Dataset characteristics are a way to add an additional dimension to variables (in addition to the time dimension used for time series). For example the different wavelengths for measured for aerosol_light scattering can be seen as an additional dimension for those measurements. Other examples are the particle diameter fro particle_number_size_distribution, or the different inlet heights for tower measurements.\n",
- "\n",
- "Nasa Ames 1001 (the base for EBAS Nasa Ames) does not directly support additional dimensions. In case of writing the data to Nasa Ames, the additional dimensions are written to as single variables (as in the data model), each with a special metadata element encoding the respective dimension extent.\n",
- "\n",
- "Other formats like NetCDF or the OPeNDP data model supports multidimensional variables. So when such data are written to NetCDF for exaple, the variables with different values for such a dimension are all written to a single variable with an additional dimension.\n",
- "\n",
- "Multiple dimensions can be added, as for example with cloud condensation nucleus measurements when measuring CCN concentration as function of supersaturation and particle diameter (see https://ebas-submit.nilu.no/Submit-Data/Data-Reporting/Templates/Category/Aerosol/Cloud-Condensation-Nucleus-Counter/level2/CCN-concentration-as-function-of-supersaturation-and-particle-diameter)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# How to handle dataset characteristics in ebas-io\n",
- "When creating files with ebas-io the characteristics must of course be created. In the same way for reading files, the characteristics must be accessed and used to interpret the data."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Writing files with variables using dataset characteristics"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Preparations:\n",
- "Here we define a method for setting up the file object for writing. Define all needed global metadata and set up the time axis. ***Don't pay much attention to this, it's necessary, but out of scope for this exercise.***"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [],
- "source": [
- "import datetime\n",
- "from ebas.io.ebasmetadata import DatasetCharacteristicList\n",
- "from nilutility.datetime_helper import DatetimeInterval\n",
- "from nilutility.datatypes import DataObject\n",
- "\n",
- "def setup_global_metadata(outfile):\n",
- " outfile.metadata.revdate = datetime.datetime.utcnow()\n",
- " outfile.metadata.datalevel = '2'\n",
- " outfile.metadata.station_code ='NO0002R'\n",
- " outfile.metadata.station_name = 'Birkenes II'\n",
- " outfile.metadata.matrix = 'pm10'\n",
- " outfile.metadata.lab_code = 'NO01L'\n",
- " outfile.metadata.instr_type = 'filter_absorption_photometer'\n",
- " outfile.metadata.instr_name = 'my_instrument'\n",
- " outfile.metadata.method = 'NO01L_my_method'\n",
- " outfile.metadata.reference_date = datetime.datetime(2020, 1, 1)\n",
- " outfile.metadata.resolution = '1h'\n",
- " outfile.metadata.projects = ['ACTRIS']\n",
- " outfile.metadata.org = DataObject(\n",
- " OR_CODE='NO01L',\n",
- " OR_NAME='Norwegian Institute for Air Research',\n",
- " OR_ACRONYM='NILU', OR_UNIT='Atmosphere and Climate Department',\n",
- " OR_ADDR_LINE1='Instituttveien 18', OR_ADDR_LINE2=None,\n",
- " OR_ADDR_ZIP='2007', OR_ADDR_CITY='Kjeller', OR_ADDR_COUNTRY='Norway'\n",
- " )\n",
- " outfile.metadata.originator.append(DataObject(\n",
- " PS_LAST_NAME=u'Someone', PS_FIRST_NAME='Else',\n",
- " PS_EMAIL='Someone@somewhere.no',\n",
- " PS_ORG_NAME='Some nice Institute',\n",
- " PS_ORG_ACR='WOW', PS_ORG_UNIT='Super interesting division',\n",
- " PS_ADDR_LINE1='Street 18', PS_ADDR_LINE2=None,\n",
- " PS_ADDR_ZIP='X-9999', PS_ADDR_CITY='Paradise',\n",
- " PS_ADDR_COUNTRY='Norway',\n",
- " PS_ORCID=None,\n",
- " ))\n",
- " outfile.metadata.submitter.append(DataObject(\n",
- " PS_LAST_NAME=u'Someone', PS_FIRST_NAME='Else',\n",
- " PS_EMAIL='Someone@somewhere.no',\n",
- " PS_ORG_NAME='Some nice Institute',\n",
- " PS_ORG_ACR='WOW', PS_ORG_UNIT='Super interesting division',\n",
- " PS_ADDR_LINE1='Street 18', PS_ADDR_LINE2=None,\n",
- " PS_ADDR_ZIP='X-9999', PS_ADDR_CITY='Paradise',\n",
- " PS_ADDR_COUNTRY='Norway',\n",
- " PS_ORCID=None,\n",
- " ))\n",
- " outfile.sample_times = [\n",
- " DatetimeInterval(datetime.datetime(2020, 1, 1, 0, 0), datetime.datetime(2020, 1, 1, 1, 0)),\n",
- " DatetimeInterval(datetime.datetime(2020, 1, 1, 1, 0), datetime.datetime(2020, 1, 1, 2, 0))\n",
- " ]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Add variables with characteristics\n",
- "***Here comes the interesting part:*** We set up some variables with characteristics (additional dimension). Specifically we add aerosol_absorption_coefficient in three different wavelengths.\n",
- "\n",
- "***Pay attention to the parts commented with ```### ADD CHARACTERISTICS```***"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [],
- "source": [
- "def setup_variables(outfile):\n",
- " # variable 1: aerosol_absorption_coefficient, 470 nm\n",
- " values = [0.5566, None] # missing value is None!\n",
- " flags = [[], [999]]\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'aerosol_absorption_coefficient'\n",
- " metadata.unit = '1/Mm'\n",
- " metadata.title = 'abs470'\n",
- " metadata.uncertainty = (6, '%')\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- " ### ADD CHARACTERISTICS\n",
- " outfile.add_var_characteristics(-1, 'Wavelength', 450)\n",
- "\n",
- "\n",
- " # variable 2: aerosol_absorption_coefficient, 520 nm\n",
- " values = [0.3196, None] # missing value is None!\n",
- " flags = [[], [999]]\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'aerosol_absorption_coefficient'\n",
- " metadata.unit = '1/Mm'\n",
- " metadata.title = 'abs520'\n",
- " metadata.uncertainty = (6, '%')\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- " ### ADD CHARACTERISTICS\n",
- " outfile.add_var_characteristics(-1, 'Wavelength', 520)\n",
- "\n",
- " # variable 3: aerosol_absorption_coefficient, 660 nm\n",
- " values = [0.3956, None] # missing value is None!\n",
- " flags = [[], [999]]\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'aerosol_absorption_coefficient'\n",
- " metadata.unit = '1/Mm'\n",
- " metadata.title = 'abs660'\n",
- " metadata.uncertainty = (6, '%')\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- " ### ADD CHARACTERISTICS\n",
- " outfile.add_var_characteristics(-1, 'Wavelength', 660)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Putting it all together:\n",
- "Create an output file object, add the global metadata, the time dimension and add the variables (aerosol_absorption_coefficient in three wavelenghts):"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [],
- "source": [
- "from ebas.io.file.nasa_ames import EbasNasaAmes\n",
- "nas = EbasNasaAmes()\n",
- "setup_global_metadata(nas)\n",
- "setup_variables(nas)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "When we write the Nasa Ames file, we see the three variables, each with the ```Wavelength``` specified in the VNAME lines (lines 14-16):"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "45 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "ACTRIS\n",
- "1 1\n",
- "2020 01 01 2022 11 17\n",
- "0.041667\n",
- "days from file reference point\n",
- "5\n",
- "1 1 1 1 1\n",
- "9.999999 9.9999 9.9999 9.9999 9.999\n",
- "end_time of measurement, days from the file reference point\n",
- "aerosol_absorption_coefficient, 1/Mm, Wavelength=450.0 nm\n",
- "aerosol_absorption_coefficient, 1/Mm, Wavelength=520.0 nm\n",
- "aerosol_absorption_coefficient, 1/Mm, Wavelength=660.0 nm\n",
- "numflag, no unit\n",
- "0\n",
- "26\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117101850.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nas\n",
- "File creation: 20221117101852898364\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20221117101850\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 2h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: aerosol_absorption_coefficient\n",
- "Unit: 1/Mm\n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_absorption_photometer\n",
- "Instrument name: my_instrument\n",
- "Method ref: NO01L_my_method\n",
- "Measurement uncertainty: 6 %\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime abs470 abs520 abs660 flag\n",
- "0.000000 0.041667 0.5566 0.3196 0.3956 0.000\n",
- "0.041667 0.083333 9.9999 9.9999 9.9999 0.999\n"
- ]
- }
- ],
- "source": [
- "nas.write()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Output as a real multidimensional variable:\n",
- "When we do the same with an EbasNetcf object, we see that the variable\n",
- "\n",
- " float64 aerosol_absorption_coefficient(u'Wavelength', u'time')\n",
- "is in fact a 2D variable (with dimensions Wavelength and time).\n",
- "\n",
- "Remark: The following cell will only run successfully if netCDF4 is installed on your system. Although netCDF4 is necessary for creating NetCDF output, it is not a dependecy when installing the ebas-io package (for most users this would be an unecessary and quite heavy dependency; users who usually work with NetCDF will have the module installed anyway).\n",
- "\n",
- "Remark: Writing an EbasNetcdf object to stdout just dumps the header for convinience. When writing to an actual output file (```ncf.write(createfiles=True)```), the actual NetCDF file would be written in full."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "<class 'netCDF4._netCDF4.Dataset'>\n",
- "root group (NETCDF4 data model, file format HDF5):\n",
- " Conventions: CF-1.8, ACDD-1.3\n",
- " featureType: timeSeries\n",
- " title: Ground based in situ observations of aerosol_absorption_coefficient at Birkenes II (NO0002R) using filter_absorption_photometer\n",
- " keywords: ACTRIS, pm10, NO0002R, Birkenes II, aerosol_absorption_coefficient\n",
- " id: NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
- " naming_authority: no.nilu.ebas\n",
- " project: ACTRIS\n",
- " acknowledgement: []\n",
- " license: ACTRIS: http://actris.nilu.no/Content/Documents/DataPolicy.pdf\n",
- " summary: Ground based in situ observations of aerosol_absorption_coefficient at Birkenes II (NO0002R) using filter_absorption_photometer. These measurements are gathered as a part of the following projects ACTRIS and they are stored in the EBAS database (http://ebas.nilu.no/). Parameters measured are: aerosol_absorption_coefficient in pm10\n",
- " source: surface observation\n",
- " institution: NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, 2007, Kjeller, Norway\n",
- " processing_level: 2: final (Physical parameters, aggregated (if needed), info on variability recommended, quality assured by human inspection. This is the typical EBAS data level meant for dissemination and long term preservation.)\n",
- " date_created: 2022-11-17T10:19:00 UTC\n",
- " date_metadata_modified: 2022-11-17T10:19:00 UTC\n",
- " creator_name: Else Someone\n",
- " creator_type: person\n",
- " creator_email: Someone@somewhere.no\n",
- " creator_institution: \"Some nice Institute, Super interesting division, WOW\"\n",
- " contributor_name: Else Someone\n",
- " contributor_role: data submitter\n",
- " publisher_type: institution\n",
- " publisher_name: NILU - Norwegian Institute for Air Research, ATMOS, EBAS\n",
- " publisher_institution: NILU - Norwegian Institute for Air Research, ATMOS, EBAS\n",
- " publisher_email: ebas@nilu.no\n",
- " publisher_url: http://ebas.nilu.no/\n",
- " geospatial_bounds: POINT Z EMPTY\n",
- " geospatial_bounds_crs: EPSG:4979\n",
- " geospatial_vertical_positive: up\n",
- " time_coverage_start: 2020-01-01T00:00:00 UTC\n",
- " time_coverage_end: 2020-01-01T02:00:00 UTC\n",
- " time_coverage_duration: P0000-00-00T02:00:00\n",
- " time_coverage_resolution: P0000-00-00T01:00:00\n",
- " timezone: UTC\n",
- " ebas_data_definition: EBAS_1.1\n",
- " ebas_set_type_code: TU\n",
- " ebas_timezone: UTC\n",
- " ebas_file_name: NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
- " ebas_file_creation: 2022-11-17T10:19:00.981306 UTC\n",
- " ebas_startdate: 20200101000000\n",
- " ebas_revision_date: 20221117101900\n",
- " ebas_statistics: arithmetic mean\n",
- " ebas_data_level: 2\n",
- " ebas_period_code: 2h\n",
- " ebas_resolution_code: 1h\n",
- " ebas_station_code: NO0002R\n",
- " ebas_platform_code: NO0002S\n",
- " ebas_station_name: Birkenes II\n",
- " ebas_regime: IMG\n",
- " ebas_component: aerosol_absorption_coefficient\n",
- " ebas_unit: 1/Mm\n",
- " ebas_matrix: pm10\n",
- " ebas_laboratory_code: NO01L\n",
- " ebas_instrument_type: filter_absorption_photometer\n",
- " ebas_instrument_name: my_instrument\n",
- " ebas_method_ref: NO01L_my_method\n",
- " ebas_measurement_uncertainty: 6 %\n",
- " ebas_organization: NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- " ebas_framework_acronym: ACTRIS\n",
- " ebas_originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- " ebas_submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- " ebas_metadata: {\n",
- " \"Data definition\": \"EBAS_1.1\",\n",
- " \"Set type code\": \"TU\",\n",
- " \"Timezone\": \"UTC\",\n",
- " \"File name\": \"NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\",\n",
- " \"File creation\": \"2022-11-17T10:19:00.981306 UTC\",\n",
- " \"Startdate\": \"20200101000000\",\n",
- " \"Revision date\": \"20221117101900\",\n",
- " \"Statistics\": \"arithmetic mean\",\n",
- " \"Data level\": \"2\",\n",
- " \"Period code\": \"2h\",\n",
- " \"Resolution code\": \"1h\",\n",
- " \"Station code\": \"NO0002R\",\n",
- " \"Platform code\": \"NO0002S\",\n",
- " \"Station name\": \"Birkenes II\",\n",
- " \"Regime\": \"IMG\",\n",
- " \"Component\": \"aerosol_absorption_coefficient\",\n",
- " \"Unit\": \"1/Mm\",\n",
- " \"Matrix\": \"pm10\",\n",
- " \"Laboratory code\": \"NO01L\",\n",
- " \"Instrument type\": \"filter_absorption_photometer\",\n",
- " \"Instrument name\": \"my_instrument\",\n",
- " \"Method ref\": \"NO01L_my_method\",\n",
- " \"Measurement uncertainty\": \"6 %\",\n",
- " \"Organization\": \"NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\",\n",
- " \"Framework acronym\": \"ACTRIS\",\n",
- " \"Originator\": \"Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\",\n",
- " \"Submitter\": \"Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\"\n",
- "}\n",
- " dimensions(sizes): time(2), metadata_time(1), tbnds(2), Wavelength(3), aerosol_absorption_coefficient_qc_flags(1)\n",
- " variables(dimensions): float64 time(time), float64 time_bnds(time, tbnds), float64 metadata_time(metadata_time), float64 metadata_time_bnds(metadata_time, tbnds), float64 Wavelength(Wavelength), float64 aerosol_absorption_coefficient(Wavelength, time), int32 aerosol_absorption_coefficient_qc(Wavelength, aerosol_absorption_coefficient_qc_flags, time), <class 'str'> aerosol_absorption_coefficient_ebasmetadata(Wavelength, metadata_time)\n",
- " groups: "
- ]
- }
- ],
- "source": [
- "from ebas.io.file.netcdf import EbasNetcdf\n",
- "ncf = EbasNetcdf()\n",
- "setup_global_metadata(ncf)\n",
- "setup_variables(ncf)\n",
- "ncf.write()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Reading files with variables using dataset characteristics\n",
- "We read a test file containing cloud condenstation nucleus concentration as function of supersaturation and particle diameter, which means two additoional dimensions (particle diameter and super saturation)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [],
- "source": [
- "import logging\n",
- "import pprint\n",
- "logging.basicConfig(level=logging.ERROR)\n",
- "from ebas.io.file.nasa_ames import EbasNasaAmes\n",
- "nas = EbasNasaAmes()\n",
- "nas.read('test_ccn.nas', ignore_valuecheck=True)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Inspecting the characteristics for each variable:\n",
- "Like other any other metadata we can also find out the characteristics of each variable. The characteristics are implemented as a list of characteristics in each variables metadata. The single characteristics ar dictionaries containing the information about a single characteristic.\n",
- "\n",
- "Each characteristic contains the following elements:\n",
- "* ```CT_TYPE```: the type of the characteristic, like ```Wavelength``` or ```Inlet towr Height```\n",
- "* ```CT_DATATYPE```: the data type of the characteristic: ```DBL``` for float, ```INT``` for integer and ```CHR``` for string\n",
- "* ```DC_VAL_DBL```: the actual value for the chatacteristic\n",
- "* ```CO_COMP_NAME``` and ```FT_TYPE```: are stored internally in order to control the validity of the characteristic for a specific component and instrument type"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, arithmetic mean\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.2,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.3,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.5,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 10.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 15.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 30.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 50.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n",
- "pm10, cloud_condensation_nuclei_number_size_distribution, uncertainty\n",
- "[{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 1.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'},\n",
- " {'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'D',\n",
- " 'DC_VAL_DBL': 100.0,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}]\n"
- ]
- }
- ],
- "source": [
- "for i, var in enumerate(nas.variables):\n",
- " # iterate through the variables of the file and\n",
- " # find out which matrix, component name and statistics code it is:\n",
- " matrix = nas.get_meta_for_var(i, 'matrix')\n",
- " comp_name = nas.get_meta_for_var(i, 'comp_name')\n",
- " statistics = nas.get_meta_for_var(i, 'statistics') \n",
- " print(\", \".join([matrix, comp_name, statistics]))\n",
- " \n",
- " # And this is how the characteristics look like:\n",
- " pprint.pprint(var.metadata.characteristics)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Convenience methods\n",
- "As we see below, the list of of characteristics is of type DatasetCharacteristicList and the single characteristics are of type DatasetCharacteristic. Thus we can expect some additional functionality."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "<class 'ebas.io.ebasmetadata.DatasetCharacteristicList'>\n",
- "<class 'ebas.io.ebasmetadata.DatasetCharacteristic'>\n"
- ]
- }
- ],
- "source": [
- "print(type(nas.variables[0].metadata.characteristics))\n",
- "print(type(nas.variables[0].metadata.characteristics[0]))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Methods of the DatasetCharacteristicList class\n",
- "\n",
- "The DatasetCharacteristicList provides some methods for handlin all characteristics for one variable.\n",
- "\n",
- "##### as_dict()\n",
- "Probably the most useful method when reading characteristics from a file, is the as_dict() method, which gives us __one single__ dictionary from __all__ characteristics of a variable."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 9,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'SS': 0.1, 'D': 15.0}\n"
- ]
- }
- ],
- "source": [
- "print(nas.variables[1].metadata.characteristics.as_dict())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "##### sorted()\n",
- "The _sorted()_ yields all characteristics in the defined sort order. It can be used for serializing the characteristics, e.g. i a string like this:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "D=15.0 nm, SS=0.1 %\n"
- ]
- }
- ],
- "source": [
- "print(\", \".join([\"{}={}\".format(char.CT_TYPE, char.value_string())\n",
- " for char in nas.variables[1].metadata.characteristics.sorted()]))\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "##### dc_by_ct_type()\n",
- "Get a single characteristics element by the characteristic type."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 11,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'CO_COMP_NAME': 'cloud_condensation_nuclei_number_size_distribution',\n",
- " 'CT_DATATYPE': 'DBL',\n",
- " 'CT_TYPE': 'SS',\n",
- " 'DC_VAL_DBL': 0.1,\n",
- " 'FT_TYPE': 'DMPS-CCNC'}\n",
- "\n",
- "In case of not set characteristic type: None\n"
- ]
- }
- ],
- "source": [
- "pprint.pprint(nas.variables[1].metadata.characteristics.dc_by_ct_type('SS'))\n",
- "\n",
- "# There is no Wavelength for CCNC...\n",
- "print(\"\\nIn case of not set characteristic type: {}\".format(\n",
- " nas.variables[1].metadata.characteristics.dc_by_ct_type('Wavelength')))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### Methods of the DatasetCharacteristic class\n",
- "##### tuple()\n",
- "A single DatasetCharacteristic provides the method tuple() which returns the characteristic in the form (type, value unit)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 12,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "SS tuple: ('SS', 0.1, '%')\n",
- "all tuples:\n",
- " ('SS', 0.1, '%')\n",
- " ('D', 50.0, 'nm')\n"
- ]
- }
- ],
- "source": [
- "# let's say we want to get the tuple for the SS characteristic of the first variable:\n",
- "cha = nas.variables[3].metadata.characteristics.dc_by_ct_type('SS')\n",
- "print('SS tuple:', cha.tuple())\n",
- "\n",
- "# or we want to iterate through all characteristics of the first variable and get the tuple:\n",
- "print(\"all tuples:\")\n",
- "for cha in nas.variables[3].metadata.characteristics:\n",
- " print(\" \", cha.tuple())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "##### value and unit\n",
- "For accessing just the value or just the unit of a characteristic.\n",
- "\n",
- "(Value and unit are property methods and look like attibutes)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 13,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "0.1\n",
- "<class 'float'>\n",
- "%\n"
- ]
- }
- ],
- "source": [
- "cha = nas.variables[3].metadata.characteristics.dc_by_ct_type('SS')\n",
- "\n",
- "print(cha.value)\n",
- "print(type(cha.value))\n",
- "print(cha.unit)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### value_string()\n",
- "Generate the thring representation of value and unit for a characteristic."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 14,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "50.0 nm\n"
- ]
- }
- ],
- "source": [
- "cha = nas.variables[3].metadata.characteristics.dc_by_ct_type('D')\n",
- "\n",
- "print(cha.value_string())"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.6"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/Examples/Doc/Notebooks/.ipynb_checkpoints/EbasMetadata-checkpoint.ipynb b/Examples/Doc/Notebooks/.ipynb_checkpoints/EbasMetadata-checkpoint.ipynb
deleted file mode 100644
index 25dceeb285d61573939fa899e0249b414ad97dc3..0000000000000000000000000000000000000000
--- a/Examples/Doc/Notebooks/.ipynb_checkpoints/EbasMetadata-checkpoint.ipynb
+++ /dev/null
@@ -1,395 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Using the EbasMetadata object"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The EbasMetadata object defines the syntax of the ebas.io file objects. It is used internally by the io.file objects for creating the output files and parsing input files.\n",
- "\n",
- "The user migtht, when creating an output file use the object for retrieving information about how to set a specific attribute in the file object in order to create a specifig metadata element in the output file.\n",
- "\n",
- "Or vice versa, when reading an input file, know which attribute to access when the contents of a specific metadata element in the file is needed.\n",
- "\n",
- "Please find below some examples how to list this kind of information interactively."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## General setup: imports and instanciation"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This part is needed for all the examples below.\n",
- "\n",
- "Be aware to specify the datalevel if you want to see metadata that are only allowed for level 0. This is important if you want to write a script for level 0 data submissions. For some templates (e.g. mmps, nephelometer and filter_absorption_potometer), additional metadata are used which are only valid for data level 0."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [],
- "source": [
- "from ebas.io.ebasmetadata import EbasMetadata, EBAS_IOFORMAT_NASA_AMES\n",
- "meta = EbasMetadata('EBAS_1.1', EBAS_IOFORMAT_NASA_AMES, data_level='0')"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## List all possible metadata elements"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This code sequence lists all possible metadata elements in EBAS. Each line contains the tag (the tag used in the Nasa Ames files) and the key (the key used in the io object's metadata).\n",
- "We only list metadata which are allowed either as global metadata (main) or as variable metadata (vname)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {
- "scrolled": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Data definition: datadef\n",
- "Set type code: type\n",
- "Timezone: timezone\n",
- "Timeref: timeref\n",
- "File name: filename\n",
- "File creation: creation_time\n",
- "Export state: export_state\n",
- "Export filter: export_filter\n",
- "Startdate: startdate\n",
- "Revision date: revdate\n",
- "Version: revision\n",
- "Version description: revdesc\n",
- "Statistics: statistics\n",
- "Data level: datalevel\n",
- "Period code: period\n",
- "Resolution code: resolution\n",
- "Sample duration: duration\n",
- "Orig. time res.: rescode_sample\n",
- "Station code: station_code\n",
- "Platform code: platform_code\n",
- "Station name: station_name\n",
- "Station WDCA-ID: station_wdca_id\n",
- "Station GAW-ID: station_gaw_id\n",
- "Station GAW-Name: station_gaw_name\n",
- "Station AIRS-ID: station_airs_id\n",
- "Station other IDs: station_other_ids\n",
- "Station state/province: station_state_code\n",
- "Station land use: station_landuse\n",
- "Station setting: station_setting\n",
- "Station GAW type: station_gaw_type\n",
- "Station WMO region: station_wmo_region\n",
- "Station latitude: station_latitude\n",
- "Station longitude: station_longitude\n",
- "Station altitude: station_altitude\n",
- "Measurement latitude: mea_latitude\n",
- "Measurement longitude: mea_longitude\n",
- "Measurement altitude: mea_altitude\n",
- "Measurement height: mea_height\n",
- "Regime: regime\n",
- "Component: comp_name\n",
- "Unit: unit\n",
- "Matrix: matrix\n",
- "Laboratory code: lab_code\n",
- "Instrument type: instr_type\n",
- "Instrument name: instr_name\n",
- "Instrument manufacturer: instr_manufacturer\n",
- "Instrument model: instr_model\n",
- "Instrument serial number: instr_serialno\n",
- "Sensor type: sensor_type\n",
- "Analytical laboratory code: ana_lab_code\n",
- "Analytical measurement technique: ana_technique\n",
- "Analytical instrument name: ana_instr_name\n",
- "Analytical instrument manufacturer: ana_instr_manufacturer\n",
- "Analytical instrument model: ana_instr_model\n",
- "Analytical instrument serial number: ana_instr_serialno\n",
- "Ext. lab. code: ext_lab\n",
- "Method ref: method\n",
- "Standard method: std_method\n",
- "Calibration scale: cal_scale\n",
- "Calibration standard ID: cal_std_id\n",
- "Secondary standard ID: sec_std_id\n",
- "Inlet type: inlet_type\n",
- "Inlet description: inlet_desc\n",
- "Inlet tube material: inlet_tube_material\n",
- "Inlet tube outer diameter: inlet_tube_outerD\n",
- "Inlet tube inner diameter: inlet_tube_innerD\n",
- "Inlet tube length: inlet_tube_length\n",
- "Time from entry inlet line to entry of converter: time_inlet_to_converter\n",
- "Duration of stay in converter or bypass line: time_converter_or_bypass_line\n",
- "Duration of stay in converter: time_stay_converter\n",
- "Converter temperature: converter_temp\n",
- "Maintenance description: maintenance_desc\n",
- "Flow rate: flow_rate\n",
- "Filter face velocity: filter_face_velocity\n",
- "Exposed filter area: filter_area\n",
- "Filter description: filter_descr\n",
- "Medium: medium\n",
- "Coating/Solution: coating_solution\n",
- "Filter prefiring: filter_prefiring\n",
- "Filter conditioning: filter_conditioning\n",
- "Filter type: filter_type\n",
- "Sample preparation: sample_prep\n",
- "Blank correction: blank_corr\n",
- "Artifact correction: artifact_corr\n",
- "Artifact correction description: artifact_corr_desc\n",
- "Charring correction: charring_corr\n",
- "Ozone correction: ozone_corr\n",
- "Water vapor correction: watervapor_corr\n",
- "Zero/span check type: zero_span_type\n",
- "Zero/span check interval: zero_span_interval\n",
- "Humidity/temperature control: hum_temp_ctrl\n",
- "Humidity/temperature control description: hum_temp_ctrl_desc\n",
- "Volume std. temperature: vol_std_temp\n",
- "Volume std. pressure: vol_std_pressure\n",
- "Detection limit: detection_limit\n",
- "Detection limit expl.: detection_limit_desc\n",
- "Upper range limit: upper_range_limit\n",
- "Measurement uncertainty: uncertainty\n",
- "Measurement uncertainty expl.: uncertainty_desc\n",
- "Zero/negative values code: zero_negative\n",
- "Zero/negative values: zero_negative_desc\n",
- "Absorption cross section: abs_cross_section\n",
- "Mass absorption cross section: mass_abs_cross_section\n",
- "Multi-scattering correction factor: multi_scattering_corr_fact\n",
- "Maximum attenuation: max_attenuation\n",
- "Leakage factor zeta: leakage_factor_zeta\n",
- "Compensation threshold attenuation 1: comp_thresh_atten1\n",
- "Compensation threshold attenuation 2: comp_thresh_atten2\n",
- "Compensation parameter k min: comp_param_kmin\n",
- "Compensation parameter k max: comp_param_kmax\n",
- "Coincidence correction: coincidence_corr\n",
- "Charge type: charge_type\n",
- "Inlet diffusion loss data: inlet_diffusion_loss_data\n",
- "CPC default pulse width: cpc_default_pulse_width\n",
- "QA measure ID: qm_id\n",
- "QA measure description: qm_desc\n",
- "QA date: qa_date\n",
- "QA outcome: qa_outcome\n",
- "QA bias: qa_bias\n",
- "QA variability: qa_variability\n",
- "QA document name: qa_doc_name\n",
- "QA document date: qa_doc_date\n",
- "QA document URL: qa_doc_url\n",
- "Originator: originator\n",
- "Submitter: submitter\n",
- "Acknowledgement: acknowledgements\n",
- "Comment: comment\n"
- ]
- }
- ],
- "source": [
- "for elem in [x for x in meta.metadata if x['main'] or x['vname']]:\n",
- " if not 'renamed_tag' in elem:\n",
- " print(\"{:45s} {}\".format(elem['tag']+\":\", elem['key']))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## List only metadata which are mandatory when writing files\n",
- "Here we restrict to _global metadata_ (```main```) which are _mandatory when writing_ (```& 2```) a file:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Data definition: datadef\n",
- "Set type code: type\n",
- "Timezone: timezone\n",
- "File name: filename\n",
- "File creation: creation_time\n",
- "Startdate: startdate\n",
- "Revision date: revdate\n",
- "Data level: datalevel\n",
- "Period code: period\n",
- "Resolution code: resolution\n",
- "Station code: station_code\n",
- "Platform code: platform_code\n",
- "Regime: regime\n",
- "Component: comp_name\n",
- "Matrix: matrix\n",
- "Laboratory code: lab_code\n",
- "Instrument type: instr_type\n",
- "Instrument name: instr_name\n",
- "Method ref: method\n",
- "Originator: originator\n",
- "Submitter: submitter\n"
- ]
- }
- ],
- "source": [
- "for elem in [x for x in meta.metadata if x['main'] & 2]:\n",
- " if not 'renamed_tag' in elem:\n",
- " print(\"{:45s} {}\".format(elem['tag']+\":\", elem['key']))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "#### specifications for cardinality\n",
- "\n",
- "- main: specifies the cardinality of the element in the main header (NNCOM lines)\n",
- "\n",
- " Bitfield: \n",
- " 0 not allowed \n",
- " 1 allowed but not mandatory \n",
- " 2 mandatory on export \n",
- " 4 mandatory on import \n",
- " 8 critical on import (if missing, exit after reading header) \n",
- " \n",
- "- vname: specifies the cardinality of the element in the vname line. \n",
- " \n",
- " Bitfield: \n",
- " 0 not allowed \n",
- " 1 allowed but not mandatory \n",
- " 2 mandatory on export (not used) \n",
- " 4 mandatory on import \n",
- " 8 critical on import (if missing, exit after reading header) \n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Finding the key to a tag"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "If you see a metadata element in the NASA Ames file and want to access it after reading the file using the ebas.io module, you need to find the element's *key* when knowing the *tag*:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "'instr_manufacturer'"
- ]
- },
- "execution_count": 4,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "meta.metadata_tags['Instrument manufacturer']['key']"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {
- "scrolled": true
- },
- "outputs": [
- {
- "data": {
- "text/plain": [
- "'rescode_sample'"
- ]
- },
- "execution_count": 5,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "meta.metadata_tags['Orig. time res.']['key']"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Finding the tag to a key"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The other way around, you might want to find the *tag* to a known *key*:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "'Instrument manufacturer'"
- ]
- },
- "execution_count": 6,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "meta.metadata_keys['instr_manufacturer']['tag']"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.6"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 1
-}
diff --git a/Examples/Doc/Notebooks/.ipynb_checkpoints/WriteNasaAmes-checkpoint.ipynb b/Examples/Doc/Notebooks/.ipynb_checkpoints/WriteNasaAmes-checkpoint.ipynb
deleted file mode 100644
index 0cd027fd6352d994d835e23450b1a4fba74330dd..0000000000000000000000000000000000000000
--- a/Examples/Doc/Notebooks/.ipynb_checkpoints/WriteNasaAmes-checkpoint.ipynb
+++ /dev/null
@@ -1,811 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# TOC\n",
- "\n",
- "* __[Writing EBAS (Nasa Ames) files](#Writing-EBAS-(Nasa-Ames)-files)__\n",
- " * __[Create an empty IO object](#Create-an-empty-IO-object)__\n",
- " * __[Add Global Metadata](#Add-Global-Metadata)__\n",
- " * __[Add Sample times](#Add-Sample-times)__\n",
- " * __[Add Variables and Variable Metadata](#Add-Variables-and-Variable-Metadata)__\n",
- " * __[Write the file](#Write-the-file)__\n",
- " * __[What happens behind the scenes?](#What-happens-behind-the-scenes?)__\n",
- "* __[Tweaking the output format](#Tweaking-the-output-format)__\n",
- " * __[Suppress the sorting of variables](#Suppress-the-sorting-of-variables)__\n",
- " * __[Suppress moving metadata between global and variable specific](#Suppress-moving-metadata-between-global-and-variable-specific)__\n",
- " * __[Customize the missing values](#Customize-the-missing-values)__\n",
- " * __[Example for tweaking the output format](#Example-for-tweaking-the-output-format)__\n",
- " * __[Flag columns](#Flag-columns)__\n",
- " * __[Example for controlling the flag options](#Example-for-controlling-the-flag-options)__"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Writing EBAS (Nasa Ames) files\n",
- "This tutorial is about writing EBAS datafiles. Although the procedure is the same for all file formats (Nasa Ames, NetCDF, CSV), the tutorial focusses on Nasa Ames. This is most relevant from a user perspective: Most users are interested in generating files for submission to EBAS."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Create an empty IO object\n",
- "First we need to set up an IO object (in this case of class EbasNasaAmes, if you want to create other file types, use an alternative class; all the rest will be the same for all file formats)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [],
- "source": [
- "from ebas.io.file.nasa_ames import EbasNasaAmes\n",
- "nas = EbasNasaAmes()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now we have an object (nas) which represents the file we want to write."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Add Global Metadata\n",
- "Next we need to add some global metadata to the file. Here the most basic metadata are shown."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [],
- "source": [
- "import datetime\n",
- "from nilutility.datatypes import DataObject\n",
- "\n",
- "def setup_global_metadata(outfile):\n",
- " outfile.metadata.revdate = datetime.datetime.utcnow()\n",
- " outfile.metadata.datalevel = '2'\n",
- " outfile.metadata.station_code ='NO0002R'\n",
- " outfile.metadata.station_name = 'Birkenes II'\n",
- " outfile.metadata.matrix = 'pm10'\n",
- " outfile.metadata.lab_code = 'NO01L'\n",
- " outfile.metadata.instr_type = 'filter_3pack'\n",
- " outfile.metadata.instr_name = 'NILU_f3p_d_0001'\n",
- " outfile.metadata.ana_lab_code = 'NO01L'\n",
- " outfile.metadata.ana_technique = 'IC'\n",
- " outfile.metadata.ana_instr_name = 'NILU_IC_03'\n",
- " outfile.metadata.ana_instr_manufacturer = 'Dionex'\n",
- " outfile.metadata.ana_instr_model = 'ICS-3000'\n",
- " outfile.metadata.ana_instr_serialno = '12345'\n",
- " outfile.metadata.reference_date = datetime.datetime(2020, 1, 1)\n",
- " outfile.metadata.resolution = '1h'\n",
- " outfile.metadata.projects = ['CAMP', 'EMEP']\n",
- " outfile.metadata.org = DataObject(\n",
- " OR_CODE='NO01L',\n",
- " OR_NAME='Norwegian Institute for Air Research',\n",
- " OR_ACRONYM='NILU', OR_UNIT='Atmosphere and Climate Department',\n",
- " OR_ADDR_LINE1='Instituttveien 18', OR_ADDR_LINE2=None,\n",
- " OR_ADDR_ZIP='2007', OR_ADDR_CITY='Kjeller', OR_ADDR_COUNTRY='Norway'\n",
- " )\n",
- " outfile.metadata.originator.append(DataObject(\n",
- " PS_LAST_NAME=u'Someone', PS_FIRST_NAME='Else',\n",
- " PS_EMAIL='Someone@somewhere.no',\n",
- " PS_ORG_NAME='Some nice Institute',\n",
- " PS_ORG_ACR='WOW', PS_ORG_UNIT='Super interesting division',\n",
- " PS_ADDR_LINE1='Street 18', PS_ADDR_LINE2=None,\n",
- " PS_ADDR_ZIP='X-9999', PS_ADDR_CITY='Paradise',\n",
- " PS_ADDR_COUNTRY='Norway',\n",
- " PS_ORCID=None,\n",
- " ))\n",
- " outfile.metadata.submitter.append(DataObject(\n",
- " PS_LAST_NAME=u'Someone', PS_FIRST_NAME='Else',\n",
- " PS_EMAIL='Someone@somewhere.no',\n",
- " PS_ORG_NAME='Some nice Institute',\n",
- " PS_ORG_ACR='WOW', PS_ORG_UNIT='Super interesting division',\n",
- " PS_ADDR_LINE1='Street 18', PS_ADDR_LINE2=None,\n",
- " PS_ADDR_ZIP='X-9999', PS_ADDR_CITY='Paradise',\n",
- " PS_ADDR_COUNTRY='Norway',\n",
- " PS_ORCID=None,\n",
- " ))\n",
- "\n",
- "setup_global_metadata(nas)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Add Sample times\n",
- "Add the sample time intervals. Here we add only three samples for demonstration."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [],
- "source": [
- "from nilutility.datetime_helper import DatetimeInterval\n",
- "\n",
- "def add_sample_times(outfile):\n",
- " outfile.sample_times = [\n",
- " DatetimeInterval(datetime.datetime(2020, 1, 1, 0, 0), datetime.datetime(2020, 1, 1, 1, 0)),\n",
- " DatetimeInterval(datetime.datetime(2020, 1, 1, 1, 0), datetime.datetime(2020, 1, 1, 2, 0)),\n",
- " DatetimeInterval(datetime.datetime(2020, 1, 1, 2, 0), datetime.datetime(2020, 1, 1, 3, 0))\n",
- " ]\n",
- "\n",
- "add_sample_times(nas)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Add Variables and Variable Metadata"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [],
- "source": [
- "def setup_variables(outfile):\n",
- " # variable 1: sodium\n",
- " values = [0.06, 0.0000022, None] # values for the three samples; missing value is None!\n",
- " flags = [[], [], [999]] # flags for the three samples (the last sample is flagged as missing)\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'sodium'\n",
- " metadata.matrix = 'pm10'\n",
- " metadata.unit = 'ug/m3'\n",
- " metadata.title = 'Na'\n",
- " metadata.inlet_type = None\n",
- " metadata.flow_rate = 10\n",
- " metadata.qa = [\n",
- " DataObject({\n",
- " 'qa_number': 1,\n",
- " 'qm_id': 'EMEP31',\n",
- " 'qa_date': datetime.datetime(2013, 10, 16),\n",
- " 'qa_outcome': True, # pass\n",
- " }),\n",
- " DataObject({\n",
- " 'qa_number': 2,\n",
- " 'qm_id': 'EMEP32',\n",
- " 'qa_date': datetime.datetime(2014, 10, 22),\n",
- " 'qa_outcome': True, # pass\n",
- " })\n",
- " ]\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- "\n",
- " # variable 2: magnesium\n",
- " values = [0.556, 1.22, None] # values for the three samples; missing value is None!\n",
- " flags = [[], [], [999]] # flags for the three samples (the last sample is flagged as missing)\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'magnesium'\n",
- " metadata.matrix = 'pm10'\n",
- " metadata.unit = 'ug/m3'\n",
- " metadata.title = 'Mg'\n",
- " metadata.inlet_type = None\n",
- " metadata.flow_rate = 10\n",
- " metadata.qa = [\n",
- " DataObject({\n",
- " 'qa_number': 1,\n",
- " 'qm_id': 'EMEP31',\n",
- " 'qa_date': datetime.datetime(2013, 10, 16),\n",
- " 'qa_outcome': True, # pass\n",
- " }),\n",
- " DataObject({\n",
- " 'qa_number': 2,\n",
- " 'qm_id': 'EMEP32',\n",
- " 'qa_date': datetime.datetime(2014, 10, 22),\n",
- " 'qa_outcome': True, # pass\n",
- " })\n",
- " ]\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- "\n",
- " # variable 2: calcium\n",
- " values = [0.556, 1.22, None] # values for the three samples; missing value is None!\n",
- " flags = [[], [], [999]] # flags for the three samples (the last sample is flagged as missing)\n",
- " metadata = DataObject()\n",
- " metadata.comp_name = 'calcium'\n",
- " metadata.matrix = 'pm10'\n",
- " metadata.unit = 'ug/m3'\n",
- " metadata.title = 'Ca'\n",
- " metadata.inlet_type = None\n",
- " metadata.flow_rate = 10\n",
- " metadata.qa = [\n",
- " DataObject({\n",
- " 'qa_number': 1,\n",
- " 'qm_id': 'EMEP31',\n",
- " 'qa_date': datetime.datetime(2013, 10, 16),\n",
- " 'qa_outcome': False, # no pass\n",
- " }),\n",
- " DataObject({\n",
- " 'qa_number': 2,\n",
- " 'qm_id': 'EMEP32',\n",
- " 'qa_date': datetime.datetime(2014, 10, 22),\n",
- " 'qa_outcome': False, # no pass\n",
- " })\n",
- " ]\n",
- " # add the variable\n",
- " outfile.variables.append(DataObject(values_=values, flags=flags, flagcol=True,\n",
- " metadata=metadata))\n",
- "\n",
- "setup_variables(nas)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Write the file\n",
- "Now the file is ready to be written (without specifying anything special, it will be printet to stdout)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "automatically generating 'Method ref' from analytical metadata: NO01L_NILU_IC_03\n"
- ]
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "57 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "CAMP EMEP\n",
- "1 1\n",
- "2020 01 01 2022 11 17\n",
- "0.041667\n",
- "days from file reference point\n",
- "5\n",
- "1 1 1 1 1\n",
- "9.999999 99.999 99.999 9.9E+99 9.999\n",
- "end_time of measurement, days from the file reference point\n",
- "calcium, ug/m3, QA1 outcome=no pass, QA2 outcome=no pass\n",
- "magnesium, ug/m3\n",
- "sodium, ug/m3\n",
- "numflag, no unit\n",
- "0\n",
- "38\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102200.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102211423076\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20221117102200\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 3h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: \n",
- "Unit: ug/m3\n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_3pack\n",
- "Instrument name: NILU_f3p_d_0001\n",
- "Analytical laboratory code: NO01L\n",
- "Analytical measurement technique: IC\n",
- "Analytical instrument name: NILU_IC_03\n",
- "Analytical instrument manufacturer: Dionex\n",
- "Analytical instrument model: ICS-3000\n",
- "Analytical instrument serial number: 12345\n",
- "Method ref: NO01L_NILU_IC_03\n",
- "Flow rate: 10 l/min\n",
- "QA1 measure ID: EMEP31\n",
- "QA1 date: 20131016\n",
- "QA1 outcome: pass\n",
- "QA2 measure ID: EMEP32\n",
- "QA2 date: 20141022\n",
- "QA2 outcome: pass\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime Ca Mg Na flag\n",
- "0.000000 0.041667 0.556 0.556 6.0E-02 0.000\n",
- "0.041667 0.083333 1.220 1.220 2.2E-06 0.000\n",
- "0.083333 0.125000 99.999 99.999 9.9E+99 0.999\n"
- ]
- }
- ],
- "source": [
- "nas.write()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### What happens behind the scenes?\n",
- "Here is an annotdated version of the output from above.\n",
- "\n",
- "```\n",
- "57 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "CAMP EMEP\n",
- "1 1\n",
- "2020 01 01 2020 11 19\n",
- "0.041667\n",
- "days from file reference point\n",
- "5\n",
- "1 1 1 1 1\n",
- "```\n",
- ">Missing values automatically chosen for each variable. Usually it's all nines, one order of magnitude higher than the higest value in the data. Number of decimals is also chosed according to the data.\n",
- "For variables, where the scientific representation has a clear advantage, it is automatically used (see sodium).\n",
- "Missing value for the flag column has beed adjusted to the maximum number of flags used at the same time.\n",
- "\n",
- "```\n",
- "9.999999 99.999 99.999 9.9E+99 9.999\n",
- "end_time of measurement, days from the file reference point\n",
- "```\n",
- "> The variables are automatically sorted according to the standard sort order (impoortant to generate reproducable files).\n",
- "Metadata are automatically moved between global metadata and file global metadata:\n",
- " * Inlet type is completely dropped (it was only set in the global metadata, and all variables override it with None).\n",
- " * Metadata elment flow_rate turns global (all variables use 10 l/min)\n",
- " * QA metadata: most QA metadata are the same in all variables, but outcome is 'no pass' for calcium. Thus, all QA metadata turn global, but outcome is overwritten for calcium (see remaining variable specific metadata).\n",
- "\n",
- "```\n",
- "calcium, ug/m3, QA1 outcome=no pass, QA2 outcome=no pass\n",
- "magnesium, ug/m3\n",
- "sodium, ug/m3\n",
- "numflag, no unit\n",
- "0\n",
- "38\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20201119190121.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20201119190124\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20201119190121\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 3h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: \n",
- "Unit: ug/m3\n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_3pack\n",
- "Instrument name: NILU_f3p_d_0001\n",
- "Analytical laboratory code: NO01L\n",
- "Analytical measurement technique: IC\n",
- "Analytical instrument name: NILU_IC_03\n",
- "Analytical instrument manufacturer: Dionex\n",
- "Analytical instrument model: ICS-3000\n",
- "Analytical instrument serial number: 12345\n",
- "Method ref: NO01L_NILU_IC_03\n",
- "```\n",
- "> Flow rate turned global:\n",
- "\n",
- "```\n",
- "Flow rate: 10 l/min\n",
- "\n",
- "```\n",
- "> QA metadata turned global, but calcium has an overwritten 'no pass'\n",
- "\n",
- "```\n",
- "QA1 measure ID: EMEP31\n",
- "QA1 date: 20131016\n",
- "QA1 outcome: pass\n",
- "QA2 measure ID: EMEP32\n",
- "QA2 date: 20141022\n",
- "QA2 outcome: pass\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime Ca Mg Na flag\n",
- "```\n",
- "> Here we see again, the number representations have been optimized in number of digits and scientific representation in case of sodium.\n",
- ">\n",
- "> The number format for the flag column is also adjusted to the maximum number of flags used at the same time.\n",
- "\n",
- "```\n",
- "0.000000 0.041667 0.556 0.556 6.0E-02 0.000\n",
- "0.041667 0.083333 1.220 1.220 2.2E-06 0.000\n",
- "0.083333 0.125000 99.999 99.999 9.9E+99 0.999\n",
- "```"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Tweaking the output format\n",
- "In order to be more flexible in generating customized output, some of the standardisations and automisations shown above can be turned off. This might be useful for reproducing an output file in a certain format (according to a template). However, this has ___no___ effect on the data content. It's just a different representation of the same data!\n",
- "\n",
- "## Suppress the sorting of variables\n",
- "We can force the variables to appear in the same sequence as specified when adding them to the file object. This is done by passing a keyword argument to the write method: ```suppress=SUPPRESS_SORT_VARIABLES```.\n",
- "\n",
- "## Suppress moving metadata between global and variable specific\n",
- "As we've seen the occurence of all metadata elements is optimized before the object is written. If most variables use the same value for a metadata element, it is turned into a global element, thus needing explicit values per variable only for exceptional variables.\n",
- "\n",
- "However, sometimes it might be convenient to suppress this automatic behaviour and rather print the metadata exactly how they were defined when creating the object. Variable specific metadata stay variable specific, and global stay global.\n",
- "This can also be achieved by passing SUPPRESS_METADATA_OCCURRENCE to the suppress parameter: ```suppress=SUPPRESS_METADATA_OCCURRENCE```\n",
- "\n",
- "Suppress options can be combined with a bit or, eg: ```suppress=SUPPRESS_SORT_VARIABLES|SUPPRESS_METADATA_OCCURRENCE``` for suppressing both options.\n",
- "\n",
- "## Customize the missing values\n",
- "Somtimes one would like to statically choose the MISSING VALUE to be used for a variable. This might be for example useful when constantly generating near realtime files with just a short time interval of data in each file. The actual data for a short periode might sometimes only contain low values, sometimes higher values. Thus the MISSING VALUE will probably change from file to file (which is perfectly correct, but aestetic annoyance).\n",
- "\n",
- "In order to provide a MISSING VALUE, each variable definition in the file objects ```variables``` can have an element ```vmiss``` which contains the missing value which should be used.\n",
- "\n",
- "> Atention!\n",
- ">\n",
- ">If the missing value provided is not sufficient for the actual data to be written, the missing value is NOT used. However the notation of the specified missing value is still used.\n",
- ">\n",
- ">Example:\n",
- ">\n",
- ">A variable would automatically get a missing value ```99.999```, but we explicitly specify a missing value of ```9.99E+99``` which is NOT sufficient to represent the values in the variable (the data need 5 digits precision).\n",
- "The provided missing value is ___not___ used in this case. However the fact that scientific format should be used is still respected. Ebas-io will calculate a new missing value, but use scientific notation\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Example for tweaking the output format\n",
- "Now we start from scratch, build the same I/O object, but tweak some details about how the file should be written. First, rebuild the same object:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [],
- "source": [
- "nas = EbasNasaAmes()\n",
- "setup_global_metadata(nas)\n",
- "add_sample_times(nas)\n",
- "setup_variables(nas)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "But this time, we want to control the missing values used in the output file:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "automatically generating 'Method ref' from analytical metadata: NO01L_NILU_IC_03\n"
- ]
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "49 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "CAMP EMEP\n",
- "1 1\n",
- "2020 01 01 2022 11 17\n",
- "0.041667\n",
- "days from file reference point\n",
- "5\n",
- "1 1 1 1 1\n",
- "9.999999 99.9999999 999.999 9.99E+99 9.999999\n",
- "end_time of measurement, days from the file reference point\n",
- "sodium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "magnesium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "calcium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=no pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=no pass\n",
- "numflag, no unit\n",
- "0\n",
- "30\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 3h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: \n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_3pack\n",
- "Instrument name: NILU_f3p_d_0001\n",
- "Analytical laboratory code: NO01L\n",
- "Analytical measurement technique: IC\n",
- "Analytical instrument name: NILU_IC_03\n",
- "Analytical instrument manufacturer: Dionex\n",
- "Analytical instrument model: ICS-3000\n",
- "Analytical instrument serial number: 12345\n",
- "Method ref: NO01L_NILU_IC_03\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime Na Mg Ca flag\n",
- "0.000000 0.041667 0.0600000 0.556 5.56E-01 0.000000\n",
- "0.041667 0.083333 0.0000022 1.220 1.22E+00 0.000000\n",
- "0.083333 0.125000 99.9999999 999.999 9.99E+99 0.999000\n"
- ]
- }
- ],
- "source": [
- "from ebas.io.file import SUPPRESS_SORT_VARIABLES, SUPPRESS_METADATA_OCCURRENCE\n",
- "\n",
- "# Variable 0: sodium\n",
- "# We used values between 0.0000022 and 0.06, which would automatically lead to scientific\n",
- "# notation for this variable. This time, we want to force decimal representation.\n",
- "# We use ```99.9999999``` as missing value.\n",
- "nas.variables[0].vmiss = '99.9999999'\n",
- "# If a flag column is written for this variable, use the format 9.999999 instaead of 9.999\n",
- "nas.variables[0].flag_vmiss = '9.999999'\n",
- "\n",
- "# Variable 1: magnesium\n",
- "# We used values between 0.556 and 1.22, which would automatically lead to 99.999 as missing value.\n",
- "# This time we want to use 999.999 as missing value.\n",
- "nas.variables[1].vmiss = '999.999'\n",
- "# If a flag column is written for this variable, use the format 9.999999 instaead of 9.999\n",
- "nas.variables[1].flag_vmiss = '9.999999'\n",
- "\n",
- "\n",
- "# Variable 2: calcium\n",
- "# We used values between 0.556 and 1.22, which would automatically lead to 99.999 as missing value.\n",
- "# This time we want to use scientific notation for this variable, although it's not an advantadge\n",
- "# given the range of actual values.\n",
- "# But we fail in actually using a correct missing value format. '9.9E+99' is not sufficient for\n",
- "# the data in the variable (at least 3 digits mantissa would be necessary). Ebas-io will ignore our\n",
- "# provided missing value, but still obey our wish to use scientific notation. The calculated missing\n",
- "# value will be 9.99E+99\n",
- "nas.variables[2].vmiss = '9.9E+99'\n",
- "# If a flag column is written for this variable, use the format 9.999999 instaead of 9.999\n",
- "nas.variables[2].flag_vmiss = '9.999999'\n",
- "\n",
- "\n",
- "# Additionally we want to prevent the automatic sorting of variables. The order should be\n",
- "# sodium, magnesium, calcium, as we specified. We need to specify SUPPRESS_SORT_VARIABLES\n",
- "# in the suppress parameter.\n",
- "\n",
- "# Finally, we want to prevent that any metadata get move between variable specific and global.\n",
- "# All metadata should be written to the file as they are specified in the object. To achieve this,\n",
- "# we add SUPPRESS_METADATA_OCCURRENCE to the suppress parameter.\n",
- "\n",
- "nas.write(suppress=SUPPRESS_SORT_VARIABLES|SUPPRESS_METADATA_OCCURRENCE)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Flag columns\n",
- "By default ebas-io has a sophisticated way of generating flag columns which is called FLAGS_ONE_OR_ALL (see below). Possible options for generating flag columns are:\n",
- "\n",
- " * FLAGS_ONE_OR_ALL: generates either one single flag column (last column) which applies to ___all___ variables __if all variables contain the exactly same flag information for each measurement__ (there are some exceptions for auxiliary data). Otherwise one column for each variable is created. This flag option is a trade off between space efficiency and easiness to understand and process.\n",
- " * FLAGS_ONE_OR_ALL\n",
- " * FLAGS_COMPRESS: similar, consecutive variables which use the same flag information share one flag variable (in the end of the group of variables. This is the most efficient way to store flag information (however it can be confusing for users not familiar with the ebas domain logic).\n",
- " * FLAGS_ALL: always use a dedicated flag variable for each data variable. Easy to understand and to process, but the most inefficient way to store flag information.\n",
- " * FLAGS_NONE: Write no flag information at all. The file will just contain data variables. All values with invalid flags will be set to missing. This is the easiest option for understanding and processing data. However, detail information in the flags is lost. Be aware that files without flags will fail to read with ebas-io!\n",
- "\n",
- "## Example for controlling the flag options: Add a flag column for each data variable"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "51 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "CAMP EMEP\n",
- "1 1\n",
- "2020 01 01 2022 11 17\n",
- "0.041667\n",
- "days from file reference point\n",
- "7\n",
- "1 1 1 1 1 1 1\n",
- "9.999999 99.9999999 9.999999 999.999 9.999999 9.99E+99 9.999999\n",
- "end_time of measurement, days from the file reference point\n",
- "sodium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "numflag sodium, no unit, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "magnesium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "numflag magnesium, no unit, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "calcium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=no pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=no pass\n",
- "numflag calcium, no unit, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=no pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=no pass\n",
- "0\n",
- "30\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 3h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: \n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_3pack\n",
- "Instrument name: NILU_f3p_d_0001\n",
- "Analytical laboratory code: NO01L\n",
- "Analytical measurement technique: IC\n",
- "Analytical instrument name: NILU_IC_03\n",
- "Analytical instrument manufacturer: Dionex\n",
- "Analytical instrument model: ICS-3000\n",
- "Analytical instrument serial number: 12345\n",
- "Method ref: NO01L_NILU_IC_03\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime Na flag_Na Mg flag_Mg Ca flag_Ca\n",
- "0.000000 0.041667 0.0600000 0.000000 0.556 0.000000 5.56E-01 0.000000\n",
- "0.041667 0.083333 0.0000022 0.000000 1.220 0.000000 1.22E+00 0.000000\n",
- "0.083333 0.125000 99.9999999 0.999000 999.999 0.999000 9.99E+99 0.999000\n"
- ]
- }
- ],
- "source": [
- "from ebas.io.file import FLAGS_ALL\n",
- "nas.write(flags=FLAGS_ALL, suppress=SUPPRESS_SORT_VARIABLES|SUPPRESS_METADATA_OCCURRENCE)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Example for controlling the flag options: Completly manual control\n",
- "When using `flags=FLAGS_AS_IS`, one can fully control which variables should get flag columns. EBAS-IO checks if the flag columns are legal and do not change the content. Examples:\n",
- " * the last variable must always be accompanied by a flag column\n",
- " * if the sequence of flags is different for two consecutive variables, they **can not** share a flag column\n",
- "Whenever such a problem is detected, EBAS-IO will simply add flag columns as needed.\n",
- "\n",
- "Keep in mind that very likely at least `suppress=SUPPRESS_SORT_VARIABLES` needs to be set additionally (Hard to know manually which variables should have flag columns, if the variables are re sorted automatically). Otherwise one will end up with one of the problems listed above."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 9,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "50 1001\n",
- "Someone, Else\n",
- "NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, , 2007, Kjeller, Norway\n",
- "Someone, Else\n",
- "CAMP EMEP\n",
- "1 1\n",
- "2020 01 01 2022 11 17\n",
- "0.041667\n",
- "days from file reference point\n",
- "6\n",
- "1 1 1 1 1 1\n",
- "9.999999 99.9999999 999.999 9.999999 9.99E+99 9.999999\n",
- "end_time of measurement, days from the file reference point\n",
- "sodium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "magnesium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=pass\n",
- "numflag, no unit\n",
- "calcium, ug/m3, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=no pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=no pass\n",
- "numflag calcium, no unit, Matrix=pm10, Inlet type=, Flow rate=10 l/min, QA1 measure ID=EMEP31, QA1 date=20131016, QA1 outcome=no pass, QA2 measure ID=EMEP32, QA2 date=20141022, QA2 outcome=no pass\n",
- "0\n",
- "30\n",
- "Data definition: EBAS_1.1\n",
- "Set type code: TU\n",
- "Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
- "Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
- "Statistics: arithmetic mean\n",
- "Data level: 2\n",
- "Period code: 3h\n",
- "Resolution code: 1h\n",
- "Station code: NO0002R\n",
- "Platform code: NO0002S\n",
- "Station name: Birkenes II\n",
- "Regime: IMG\n",
- "Component: \n",
- "Matrix: pm10\n",
- "Laboratory code: NO01L\n",
- "Instrument type: filter_3pack\n",
- "Instrument name: NILU_f3p_d_0001\n",
- "Analytical laboratory code: NO01L\n",
- "Analytical measurement technique: IC\n",
- "Analytical instrument name: NILU_IC_03\n",
- "Analytical instrument manufacturer: Dionex\n",
- "Analytical instrument model: ICS-3000\n",
- "Analytical instrument serial number: 12345\n",
- "Method ref: NO01L_NILU_IC_03\n",
- "Originator: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "Submitter: Someone, Else, Someone@somewhere.no, Some nice Institute, WOW, Super interesting division, Street 18, , X-9999, Paradise, Norway\n",
- "starttime endtime Na Mg flag Ca flag_Ca\n",
- "0.000000 0.041667 0.0600000 0.556 0.000000 5.56E-01 0.000000\n",
- "0.041667 0.083333 0.0000022 1.220 0.000000 1.22E+00 0.000000\n",
- "0.083333 0.125000 99.9999999 999.999 0.999000 9.99E+99 0.999000\n"
- ]
- }
- ],
- "source": [
- "nas.variables[0].flagcol = False\n",
- "nas.variables[1].flagcol = True\n",
- "nas.variables[2].flagcol = True\n",
- "from ebas.io.file import FLAGS_AS_IS\n",
- "nas.write(flags=FLAGS_AS_IS, suppress=SUPPRESS_SORT_VARIABLES|SUPPRESS_METADATA_OCCURRENCE)"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.6"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/Examples/Doc/Notebooks/Dataset Characteristics.ipynb b/Examples/Doc/Notebooks/Dataset Characteristics.ipynb
index d0c6ed2dee1c3d64e9a8e27817468a32b93eb2a1..7322bb971145c9eb95bc4a6215217b00966ad1b2 100644
--- a/Examples/Doc/Notebooks/Dataset Characteristics.ipynb
+++ b/Examples/Doc/Notebooks/Dataset Characteristics.ipynb
@@ -224,7 +224,7 @@
"Someone, Else\n",
"ACTRIS\n",
"1 1\n",
- "2020 01 01 2022 11 17\n",
+ "2020 01 01 2024 05 31\n",
"0.041667\n",
"days from file reference point\n",
"5\n",
@@ -240,10 +240,10 @@
"Data definition: EBAS_1.1\n",
"Set type code: TU\n",
"Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117101850.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nas\n",
- "File creation: 20221117101852898364\n",
+ "File name: NO0002R.20200101000000.20240531133333.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nas\n",
+ "File creation: 20240531133336838507\n",
"Startdate: 20200101000000\n",
- "Revision date: 20221117101850\n",
+ "Revision date: 20240531133333\n",
"Statistics: arithmetic mean\n",
"Data level: 2\n",
"Period code: 2h\n",
@@ -300,19 +300,20 @@
"root group (NETCDF4 data model, file format HDF5):\n",
" Conventions: CF-1.8, ACDD-1.3\n",
" featureType: timeSeries\n",
- " title: Ground based in situ observations of aerosol_absorption_coefficient at Birkenes II (NO0002R) using filter_absorption_photometer\n",
- " keywords: ACTRIS, pm10, NO0002R, Birkenes II, aerosol_absorption_coefficient\n",
- " id: NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
+ " title: Aerosol_absorption_coefficient at Birkenes II\n",
+ " keywords: aerosol_absorption_coefficient, NO0002R, ACTRIS, Birkenes II, pm10\n",
+ " id: NO0002R.20200101000000.20240531133355.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
" naming_authority: no.nilu.ebas\n",
" project: ACTRIS\n",
" acknowledgement: []\n",
- " license: ACTRIS: http://actris.nilu.no/Content/Documents/DataPolicy.pdf\n",
- " summary: Ground based in situ observations of aerosol_absorption_coefficient at Birkenes II (NO0002R) using filter_absorption_photometer. These measurements are gathered as a part of the following projects ACTRIS and they are stored in the EBAS database (http://ebas.nilu.no/). Parameters measured are: aerosol_absorption_coefficient in pm10\n",
+ " doi: []\n",
+ " license: https://creativecommons.org/licenses/by/4.0/\n",
+ " summary: Aerosol_absorption_coefficient at Birkenes II. These measurements are gathered as a part of the following projects ACTRIS and they are stored in the EBAS database (http://ebas.nilu.no/). Parameters measured are: aerosol_absorption_coefficient in pm10\n",
" source: surface observation\n",
" institution: NO01L, Norwegian Institute for Air Research, NILU, Atmosphere and Climate Department, Instituttveien 18, 2007, Kjeller, Norway\n",
" processing_level: 2: final (Physical parameters, aggregated (if needed), info on variability recommended, quality assured by human inspection. This is the typical EBAS data level meant for dissemination and long term preservation.)\n",
- " date_created: 2022-11-17T10:19:00 UTC\n",
- " date_metadata_modified: 2022-11-17T10:19:00 UTC\n",
+ " date_created: 2024-05-31T13:33:55 UTC\n",
+ " date_metadata_modified: 2024-05-31T13:33:55 UTC\n",
" creator_name: Else Someone\n",
" creator_type: person\n",
" creator_email: Someone@somewhere.no\n",
@@ -335,10 +336,10 @@
" ebas_data_definition: EBAS_1.1\n",
" ebas_set_type_code: TU\n",
" ebas_timezone: UTC\n",
- " ebas_file_name: NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
- " ebas_file_creation: 2022-11-17T10:19:00.981306 UTC\n",
+ " ebas_file_name: NO0002R.20200101000000.20240531133355.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\n",
+ " ebas_file_creation: 2024-05-31T13:33:55.517809 UTC\n",
" ebas_startdate: 20200101000000\n",
- " ebas_revision_date: 20221117101900\n",
+ " ebas_revision_date: 20240531133355\n",
" ebas_statistics: arithmetic mean\n",
" ebas_data_level: 2\n",
" ebas_period_code: 2h\n",
@@ -363,10 +364,10 @@
" \"Data definition\": \"EBAS_1.1\",\n",
" \"Set type code\": \"TU\",\n",
" \"Timezone\": \"UTC\",\n",
- " \"File name\": \"NO0002R.20200101000000.20221117101900.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\",\n",
- " \"File creation\": \"2022-11-17T10:19:00.981306 UTC\",\n",
+ " \"File name\": \"NO0002R.20200101000000.20240531133355.filter_absorption_photometer.aerosol_absorption_coefficient.pm10.2h.1h.NO01L_my_instrument.NO01L_my_method.lev2.nc\",\n",
+ " \"File creation\": \"2024-05-31T13:33:55.517809 UTC\",\n",
" \"Startdate\": \"20200101000000\",\n",
- " \"Revision date\": \"20221117101900\",\n",
+ " \"Revision date\": \"20240531133355\",\n",
" \"Statistics\": \"arithmetic mean\",\n",
" \"Data level\": \"2\",\n",
" \"Period code\": \"2h\",\n",
@@ -1225,6 +1226,13 @@
"\n",
"print(cha.value_string())"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
diff --git a/Examples/Doc/Notebooks/EbasMetadata.ipynb b/Examples/Doc/Notebooks/EbasMetadata.ipynb
index 25dceeb285d61573939fa899e0249b414ad97dc3..12178816791f15ed8df7ea8c7e5fe2cba44652f2 100644
--- a/Examples/Doc/Notebooks/EbasMetadata.ipynb
+++ b/Examples/Doc/Notebooks/EbasMetadata.ipynb
@@ -73,10 +73,14 @@
"output_type": "stream",
"text": [
"Data definition: datadef\n",
+ "Data license: license\n",
+ "Citation: citation\n",
"Set type code: type\n",
"Timezone: timezone\n",
"Timeref: timeref\n",
"File name: filename\n",
+ "Represents DOI: doi\n",
+ "Contains data from DOI: doi_list\n",
"File creation: creation_time\n",
"Export state: export_state\n",
"Export filter: export_filter\n",
@@ -84,6 +88,8 @@
"Revision date: revdate\n",
"Version: revision\n",
"Version description: revdesc\n",
+ "Input dataset: input_dataset\n",
+ "Software: software\n",
"Statistics: statistics\n",
"Data level: datalevel\n",
"Period code: period\n",
@@ -369,6 +375,13 @@
"source": [
"meta.metadata_keys['instr_manufacturer']['tag']"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
diff --git a/Examples/Doc/Notebooks/WriteNasaAmes.ipynb b/Examples/Doc/Notebooks/WriteNasaAmes.ipynb
index 0cd027fd6352d994d835e23450b1a4fba74330dd..ced524e3fa25e3875c3cba1d743911afc2c53708 100644
--- a/Examples/Doc/Notebooks/WriteNasaAmes.ipynb
+++ b/Examples/Doc/Notebooks/WriteNasaAmes.ipynb
@@ -5,28 +5,24 @@
"metadata": {},
"source": [
"# TOC\n",
- "\n",
- "* __[Writing EBAS (Nasa Ames) files](#Writing-EBAS-(Nasa-Ames)-files)__\n",
- " * __[Create an empty IO object](#Create-an-empty-IO-object)__\n",
- " * __[Add Global Metadata](#Add-Global-Metadata)__\n",
- " * __[Add Sample times](#Add-Sample-times)__\n",
- " * __[Add Variables and Variable Metadata](#Add-Variables-and-Variable-Metadata)__\n",
- " * __[Write the file](#Write-the-file)__\n",
- " * __[What happens behind the scenes?](#What-happens-behind-the-scenes?)__\n",
- "* __[Tweaking the output format](#Tweaking-the-output-format)__\n",
- " * __[Suppress the sorting of variables](#Suppress-the-sorting-of-variables)__\n",
- " * __[Suppress moving metadata between global and variable specific](#Suppress-moving-metadata-between-global-and-variable-specific)__\n",
- " * __[Customize the missing values](#Customize-the-missing-values)__\n",
- " * __[Example for tweaking the output format](#Example-for-tweaking-the-output-format)__\n",
- " * __[Flag columns](#Flag-columns)__\n",
- " * __[Example for controlling the flag options](#Example-for-controlling-the-flag-options)__"
+ " * [Writing EBAS (Nasa Ames) files](#writing_ebas_(nasa)\n",
+ "\t * [Create an empty IO object](#create_an_empty)\n",
+ "\t * [Add Global Metadata](#add_global_metadata)\n",
+ "\t * [Add Sample times](#add_sample_times)\n",
+ "\t * [Add Variables and Variable Metadata](#add_variables_and)\n",
+ "\t * [Write the file](#write_the_file)\n",
+ "\t\t * [What happens behind the scenes?](#what_happens_behind)\n",
+ " * [Tweaking the output format](#tweaking_the_output)\n",
+ "\t * [Example for tweaking the output format](#example_for_tweaking)\n",
+ "\t * [Flag columns](#flag_columns)\n",
+ "\t * [Example for controlling the flag options: Completly manual control](#example_for_controlling)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Writing EBAS (Nasa Ames) files\n",
+ "# Writing EBAS (Nasa Ames) files <a class=\"anchor\" id=\"writing_ebas_(nasa\"></a>\n",
"This tutorial is about writing EBAS datafiles. Although the procedure is the same for all file formats (Nasa Ames, NetCDF, CSV), the tutorial focusses on Nasa Ames. This is most relevant from a user perspective: Most users are interested in generating files for submission to EBAS."
]
},
@@ -34,7 +30,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Create an empty IO object\n",
+ "## Create an empty IO object <a class=\"anchor\" id=\"create_an_empty\"></a>\n",
"First we need to set up an IO object (in this case of class EbasNasaAmes, if you want to create other file types, use an alternative class; all the rest will be the same for all file formats)."
]
},
@@ -59,7 +55,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Add Global Metadata\n",
+ "## Add Global Metadata <a class=\"anchor\" id=\"add_global_metadata\"></a>\n",
"Next we need to add some global metadata to the file. Here the most basic metadata are shown."
]
},
@@ -125,7 +121,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Add Sample times\n",
+ "## Add Sample times <a class=\"anchor\" id=\"add_sample_times\"></a>\n",
"Add the sample time intervals. Here we add only three samples for demonstration."
]
},
@@ -151,7 +147,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Add Variables and Variable Metadata"
+ "## Add Variables and Variable Metadata <a class=\"anchor\" id=\"add_variables_and\"></a>\n"
]
},
{
@@ -252,22 +248,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Write the file\n",
+ "## Write the file <a class=\"anchor\" id=\"write_the_file\"></a>\n",
"Now the file is ready to be written (without specifying anything special, it will be printet to stdout)."
]
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 26,
"metadata": {},
"outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "automatically generating 'Method ref' from analytical metadata: NO01L_NILU_IC_03\n"
- ]
- },
{
"name": "stdout",
"output_type": "stream",
@@ -278,7 +267,7 @@
"Someone, Else\n",
"CAMP EMEP\n",
"1 1\n",
- "2020 01 01 2022 11 17\n",
+ "2020 01 01 2024 06 03\n",
"0.041667\n",
"days from file reference point\n",
"5\n",
@@ -294,10 +283,10 @@
"Data definition: EBAS_1.1\n",
"Set type code: TU\n",
"Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102200.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102211423076\n",
+ "File name: NO0002R.20200101000000.20240603142447.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
+ "File creation: 20240603142741380345\n",
"Startdate: 20200101000000\n",
- "Revision date: 20221117102200\n",
+ "Revision date: 20240603142447\n",
"Statistics: arithmetic mean\n",
"Data level: 2\n",
"Period code: 3h\n",
@@ -343,7 +332,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### What happens behind the scenes?\n",
+ "### What happens behind the scenes? <a class=\"anchor\" id=\"what_happens_behind\"></a>\n",
"Here is an annotdated version of the output from above.\n",
"\n",
"```\n",
@@ -443,7 +432,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "# Tweaking the output format\n",
+ "# Tweaking the output format <a class=\"anchor\" id=\"tweaking_the_output\"></a>\n",
"In order to be more flexible in generating customized output, some of the standardisations and automisations shown above can be turned off. This might be useful for reproducing an output file in a certain format (according to a template). However, this has ___no___ effect on the data content. It's just a different representation of the same data!\n",
"\n",
"## Suppress the sorting of variables\n",
@@ -476,7 +465,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Example for tweaking the output format\n",
+ "## Example for tweaking the output format <a class=\"anchor\" id=\"example_for_tweaking\"></a>\n",
"Now we start from scratch, build the same I/O object, but tweak some details about how the file should be written. First, rebuild the same object:"
]
},
@@ -521,7 +510,7 @@
"Someone, Else\n",
"CAMP EMEP\n",
"1 1\n",
- "2020 01 01 2022 11 17\n",
+ "2020 01 01 2024 06 03\n",
"0.041667\n",
"days from file reference point\n",
"5\n",
@@ -537,10 +526,10 @@
"Data definition: EBAS_1.1\n",
"Set type code: TU\n",
"Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
+ "File name: NO0002R.20200101000000.20240603142125.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
+ "File creation: 20240603142126406168\n",
"Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
+ "Revision date: 20240603142125\n",
"Statistics: arithmetic mean\n",
"Data level: 2\n",
"Period code: 3h\n",
@@ -617,7 +606,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Flag columns\n",
+ "## Flag columns <a class=\"anchor\" id=\"flag_columns\"></a>\n",
"By default ebas-io has a sophisticated way of generating flag columns which is called FLAGS_ONE_OR_ALL (see below). Possible options for generating flag columns are:\n",
"\n",
" * FLAGS_ONE_OR_ALL: generates either one single flag column (last column) which applies to ___all___ variables __if all variables contain the exactly same flag information for each measurement__ (there are some exceptions for auxiliary data). Otherwise one column for each variable is created. This flag option is a trade off between space efficiency and easiness to understand and process.\n",
@@ -625,6 +614,7 @@
" * FLAGS_COMPRESS: similar, consecutive variables which use the same flag information share one flag variable (in the end of the group of variables. This is the most efficient way to store flag information (however it can be confusing for users not familiar with the ebas domain logic).\n",
" * FLAGS_ALL: always use a dedicated flag variable for each data variable. Easy to understand and to process, but the most inefficient way to store flag information.\n",
" * FLAGS_NONE: Write no flag information at all. The file will just contain data variables. All values with invalid flags will be set to missing. This is the easiest option for understanding and processing data. However, detail information in the flags is lost. Be aware that files without flags will fail to read with ebas-io!\n",
+ " * FLAGS_AS_IS: Allows full control which variables should get flag columns. EBAS-IO checks if the flag columns are legal and do not change the content. Keep in mind that very likely at least `suppress=SUPPRESS_SORT_VARIABLES` needs to be set additionally. For more details, see the next example below.\n",
"\n",
"## Example for controlling the flag options: Add a flag column for each data variable"
]
@@ -644,7 +634,7 @@
"Someone, Else\n",
"CAMP EMEP\n",
"1 1\n",
- "2020 01 01 2022 11 17\n",
+ "2020 01 01 2024 06 03\n",
"0.041667\n",
"days from file reference point\n",
"7\n",
@@ -662,10 +652,10 @@
"Data definition: EBAS_1.1\n",
"Set type code: TU\n",
"Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
+ "File name: NO0002R.20200101000000.20240603142125.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
+ "File creation: 20240603142126406168\n",
"Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
+ "Revision date: 20240603142125\n",
"Statistics: arithmetic mean\n",
"Data level: 2\n",
"Period code: 3h\n",
@@ -704,7 +694,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Example for controlling the flag options: Completly manual control\n",
+ "## Example for controlling the flag options: Completly manual control <a class=\"anchor\" id=\"example_for_controlling\"></a>\n",
"When using `flags=FLAGS_AS_IS`, one can fully control which variables should get flag columns. EBAS-IO checks if the flag columns are legal and do not change the content. Examples:\n",
" * the last variable must always be accompanied by a flag column\n",
" * if the sequence of flags is different for two consecutive variables, they **can not** share a flag column\n",
@@ -728,7 +718,7 @@
"Someone, Else\n",
"CAMP EMEP\n",
"1 1\n",
- "2020 01 01 2022 11 17\n",
+ "2020 01 01 2024 06 03\n",
"0.041667\n",
"days from file reference point\n",
"6\n",
@@ -745,10 +735,10 @@
"Data definition: EBAS_1.1\n",
"Set type code: TU\n",
"Timezone: UTC\n",
- "File name: NO0002R.20200101000000.20221117102218.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
- "File creation: 20221117102220071506\n",
+ "File name: NO0002R.20200101000000.20240603142125.filter_3pack..pm10.3h.1h.NO01L_NILU_f3p_d_0001.NO01L_NILU_IC_03.lev2.nas\n",
+ "File creation: 20240603142126406168\n",
"Startdate: 20200101000000\n",
- "Revision date: 20221117102218\n",
+ "Revision date: 20240603142125\n",
"Statistics: arithmetic mean\n",
"Data level: 2\n",
"Period code: 3h\n",
@@ -785,6 +775,33 @@
"from ebas.io.file import FLAGS_AS_IS\n",
"nas.write(flags=FLAGS_AS_IS, suppress=SUPPRESS_SORT_VARIABLES|SUPPRESS_METADATA_OCCURRENCE)"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Variable 1: Regime/Matrix/Component combination 'IMG'/'pm10'/'sodium' is not allowed for instrument type filter_3pack\n",
+ "Variable 2: Regime/Matrix/Component combination 'IMG'/'pm10'/'magnesium' is not allowed for instrument type filter_3pack\n",
+ "Variable 3: Regime/Matrix/Component combination 'IMG'/'pm10'/'calcium' is not allowed for instrument type filter_3pack\n",
+ "No applicable template found\n"
+ ]
+ }
+ ],
+ "source": [
+ "nas.check()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
diff --git a/binder/requirements.txt b/binder/requirements.txt
index 815433953db7165bd71906f93dcd7602a092c6a3..2a9a1f383f9a986edf0149ddef130d2d64f52739 100644
--- a/binder/requirements.txt
+++ b/binder/requirements.txt
@@ -1,2 +1,2 @@
-./dist/ebas_io-4.3.4-py3-none-any.whl
+./dist/ebas_io-4.3.8-py3-none-any.whl
netCDF4
diff --git a/dist/ebas_io-4.3.4-py2-none-any.whl b/dist/ebas_io-4.3.4-py2-none-any.whl
deleted file mode 100644
index 6b1456801e37e7bafe9d8e15a6b66df82d8ab5a1..0000000000000000000000000000000000000000
Binary files a/dist/ebas_io-4.3.4-py2-none-any.whl and /dev/null differ
diff --git a/dist/ebas_io-4.3.4-py3-none-any.whl b/dist/ebas_io-4.3.4-py3-none-any.whl
deleted file mode 100644
index ee69ef2a80b3e44dc4f892760cb88138c9c216e1..0000000000000000000000000000000000000000
Binary files a/dist/ebas_io-4.3.4-py3-none-any.whl and /dev/null differ
diff --git a/dist/ebas_io-4.3.8-py2-none-any.whl b/dist/ebas_io-4.3.8-py2-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..e7b62ea9ebdd0e820fb23512c480935b42095451
Binary files /dev/null and b/dist/ebas_io-4.3.8-py2-none-any.whl differ
diff --git a/dist/ebas_io-4.3.8-py3-none-any.whl b/dist/ebas_io-4.3.8-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..88317502cf254b2eb1ff155404e9f0de4c4b9cfa
Binary files /dev/null and b/dist/ebas_io-4.3.8-py3-none-any.whl differ