README.md

# ACTRIS ACSM converter

Tool to convert PTRMS raw data to ACTRIS NASA Ames files. Supports regular annual lev0/1/2 products and NRT lev0b automatically qualified products

## Informations

* dev repository: https://git.icare.univ-lille1.fr/dev/actris_ptrms_converter
* version: 0.0.0

## Copyright (©) 2024

* Université de Lille
* Centre national de la recherche scientifique (CNRS)

## Authors

* Scientific development : Valerie Gros, Lorna Foliot (LSCE/IPSL)
* Convertion algorithm development : Marc-Antoine Drouin, Christophe Boitel (LMD/IPSL)
* Software development : Nicolas Pascal, Aurélien Chauvigné (AERIS/ICARE)

## License

This project is licensed under the GNU AFFERO General Public License v3 - see the [LICENSE](LICENSE) file for details

## Prerequisites

- Python3
- numpy
- pandas
- scipy
- pysftp-extension
- click
- matplotlib
- pycurl
- toml
- tomli

## Install

### Environment

Code should be working with any python >= 3.8

- pip

    ```bash
    pip install -r requirements.txt
    ```

- conda

    ```bash
    conda create --name ptrms -c conda-forge --file requirements.txt python
    ```

### Configuration

Each station/instrument setup needs to be configured in instrument_configuration/<STATION>_PTRMS.json file.


## Running

### NRT submissions

#### raw -> lev0: runs at stations

```
* All under the hood

Usage: python3 src/uploader/uploader.py [options]

Options:
  -h, --help            show this help message and exit
  -a, --all-days        Reprocess the whole time serie
  -p PROD_ID, --prod-id=PROD_ID
                        Process only this product. All by default
  -v, --verbose         Print out processing informations
  -x, --overwrite       Overwrite existing files

* For a given hour

Usage: python3 src/uploader/hourly_uploader.py [options] <prod-id> <hour-id>

Args:
  <prod-id> : ID of the product to process, see product.cfg file for supported ones
  <hour-id> : hour to process, using the format YYYY-MM-DDThh

Options:
  -h, --help            show this help message and exit
  -o OUT_DIR, --out_dir=OUT_DIR
                        force a specific output directory
  -v, --verbose         Print out processing informations
  -x, --overwrite       Overwrite existing files

* For a a list of files

Usage: python3 src/converter/PtrmsConverter.py [options] <infiles>

Args:
  <infiles> : list of files to process

Options:
  -h, --help            show this help message and exit
  -p PROD_ID, --prod-id=PROD_ID
                        Process only this product. All by default
  -o OUT_DIR, --out_dir=OUT_DIR
                        force a specific output directory

```

### Input/Output format

* Inputs

PTR-MS measurements in CSV format with column names as shown in tests/SIRTA/inputs/

Valve status in CSV format with column names as shown in tests/SIRTA/inputs/

* Outputs

NASA Ames level 0 files as shown in tests/SIRTA/results

### NRT submissions

To enable NASA Ames file submission to NILU's server, an access to NILU's ftp is required.

User has to be defined in upload.cfg and corresponding private key added to .ssh/id_rsa.

File submission can be enable by setting enable_upload to 1.

#### RAW -> lev0

* Inputs

A lev0 ACTRIS NASA-AMES files, as shown in tests/SIRTA/results/

* Outputs

A lev0b ACTRIS NASA-AMES files, as shown in tests/SIRTA/results/