Step 2: Receive your central working area

Who’s involved

NCAS Data Manager/NCAS IT

Introduction

A standard pipeline for data provides a manageable approach, with greater support from the NCAS data scientist and NCAS IT team. This can assist with managing back-ups and data flows through to the archive. Following these structures will also help with the onward flow of data to the archive and publication as we’ll capture the details of the deployment.

What happens

Following instrument registration a series of directories will be established to provide a framework for the data pipeline and associated workflows. These consist of a structured set of directories in the NCAS Group Work Space (GWS) and associated github repositories to capture processing code etc. Having the latter will also enable specific instances of code etc. to be captured and become referenceable to aid wider traceability and openness about the data production pipelines; crucial for building transparency and trust for NCAS data.

Most of this work is undertaken automatically or on behalf of the instrument scientist by the NCAS Data Manager and/or the NCAS IT team who will then provide details of the resulting central working areas to the instrument scientist with accompanying instructions and support on how to use them.

Workflow (NCAS Data Manager/NCAS IT)

Find the NCAS instrument name (its short identifier) and observatory of new instrument (this might be 'mobile' as opposed to wao, cvao, bttao etc)
Create three new public GitHub repositories under the “ncasuk” github organisation, called:
<instrument-name>-software
<instrumnt-name>-tools
<instrument-name>-software
On JASMIN, use the script in the relevant observatory folder (where “amf” replaces “mobile”), which should be “/gws/pw/j07/ncas_obs_vol{1,2}/<observatory-name>/GWS_scripts/create_new_<OBSERVATORY-NAME>_instrument”
Go into the newly create folders for the instrument in software, tools and additionaldata to pull in the (empty) GitHub repositories, e.g.:

cd /gws/pw/j07/ncas_obs_vol{1,2}/<observatory-name>/software/<instrument-name>
git clone git@github.com:ncasuk/<instrument-name>-software .

Note the “.” at the end of the above command

Once the NCAS Data Manager/NCAS IT has set up the area they will be in touch to let you know that these are ready to use and provide further details on accessing them.

Future Plan

None

Further Details