Step 2: Receive your central working area

Who’s involved

NCAS Data Manager/NCAS IT

Present Action

This will trigger a storage area to be set up in the NCAS Obs GWS on JASMIN and associated back-ups storage. Also get a github repo as a place to store processing code that can have tagged releases out into Zenodo for later citation.

Introduction

A standard pipeline for data provides a manageable approach, with greater support from the NCAS data scientist and NCAS IT team. This can assist with managing back-ups and data flows through to the archive. Following these structures will also help with the onward flow of data to the archive and publication as we’ll capture the deployment details.

Following instrument registration a series of directories will be established to provide a framework for the data pipeline and associated workflows. These consist of a structured set of directories in the NCAS group workspace and associated github repositories to capture processing code etc. Having the latter will also enable specific instances of code etc. to be captured and become referenceable to aid wider traceability and openness on the data production pipelines, crucial for building transparency and trust in NCAS data.

Most of this work is undertaken automatically or on behalf of the instrument scientist by the NCAS Data Manager and/or the NCAS IT team who will then provide details of the resulting central working areas to the instrument scientist with accompanying instructions and support on how to use them.

Workflow

Find NCAS name and observatory of new instrument
Create three new public GitHub repositories under the “ncasuk” organisation, called <instrument-name>-software, <instrument-name>-tools, <instrument-name>-additionaldata
On JASMIN, use the script in the relevant observatory folder (where “amf” replaces “mobile”), which should be “/gws/pw/j07/ncas_obs_vol{1,2}/\/GWS_scripts/create_new_<OBSERVATORY-NAME>_instrument”
Go into the newly create folders for the instrument in software, tools and additionaldata to pull in the (empty) GitHub repositories, e.g.:

cd /gws/pw/j07/ncas_obs_vol{1,2}/<observatory-name>/software/<instrument-name>
git clone git@github.com:ncasuk/<instrument-name>-software .

Note the “.” at the end of the above command

Future Plan

None

Further Details

None