Step 3: Deployment Data Flow Preparation

Who’s involved

Instrument scientist, CEDA data scientist

Introduction

Each time an instrument is set up for measurement - whether that is for a specific campaign or once for a long-term deployment - it’s important to ensure that the related data are clearly captured as a coherent data stream, distinct from other deployments (either by the same instrument for different purposes and/or from other instruments on the same campaign). This step also provides a point at which the specific data products for the given deployment are established - i.e. a reference back to the science proposal for the deployment* . This may be some or all of the possible products identified in Step 01: Registering your instrument with NCAS; it could also mean that new data products need to be listed for the instrument too.

*i.e. what's the funding line paying for the observations... this is important distinction as it determines the data policy which applies for things like access control, licencing and how the data are archived. This doesn't preclude the data being used by other co-located projects at the time or subsequent re-use. [If those details are important, raise these with your CEDA archive contact]

Workflow

The overall workflow for each deployment is the same regardless of the operation type of the instrument (i.e. long-term vs campaign mode). Though, for long-term instruments this is likely only needed to be done once.

1. Capture the basic information for the deployment:
   1. Start date and time for the instrument’s deployment. Note, this is the first date that instrument will be deployed, it may not be the same date as when the first measurement is taken. This has been chosen for consistency of approach and to aid full instrument history to be easily identified.
   2. Platform/location details that may be needed later to convey the geospatial context for the data. Details of the site/platform will also be needed later too.
2. Create required ‘deployment’ directories within the NCAS Group Work Space for raw data, intermediate processing areas and final NCAS data products. These should be named following the deployment directory naming convention. See: Quick Guide F: Directory and Filenaming and will have the form:

<deployment start date>_<deployment name>

Overall, there are 4 modes in which instruments are likely to be operated. The deployment directory names for these follow the same overall naming convention outlined above. However, to clarify:

1. Campaign only instruments will use a short name for the campaign in the deployment name (e.g. project acronym, not location).
2. Longterm only instruments will use ‘longterm’ for the deployment name
3. Dual mode instruments - i.e. those that switch between campaign and long-term deployments will follow option 1. above for campaign deployment and the site-name when re-deployed back to long-term measurements at its ‘home’ site [note, the site-name should be one already in use, e.g. an NCAS observatory such as cvao , wao , bttao , iao ]
4. Interleaved mode instruments (e.g. Chilbolton Atmospheric Observatory radars) are those where data collection will switch between one project and another over a period of time. For these instruments one deployment directory is created for each project (i.e. following the campaign/long-term modes above), with the start date of the first deployment for that given deployment. Measurements for each deployment are then grouped together within the same deployment directory.

Future Plan

None

Further Details

See Quick Guide: NCAS Obs GWS on JASMIN and Quick Guide F: Directory and Filenaming for fuller details on the structure of the NCAS Group Work Space and overall directory naming conventions used (links below)