Data fra E39-prosjekt (Data collection from E39-project)

MET has a collection of survey data provided by the Norwegian road administration. This data is primarily sampled from a selection of fjords in Sunnmøre in regards to the 'ferry-less' E39 road plan.
The data is stored in a common network common dataform (netCDF) format, which enables multidimensional variables. It is a widely used data format for scientific data. Every .nc file has a myriad of variables, all of which has descriptive meta-data.

Since it is difficult to publish multi-dimensional data through MQTT, some compromises have been made. For example, the measurement of salinity is stored and accessed with regards to both time and depth. I.e to get a value for salinity, depth and time must be provided. Time can be provided through timestamp, but depth has to be set to a static variable. In this case the depth is set to be always 5 meters.

The script called XarrayBuouData.py fetches every .nc file on MET buoy data section of E39 and parses through all variables and stores their keys in an array. A separate function handles connection and publishing through MQTT. Below is a code-snippet which shows how to select and publish a variable.

json_data = json.dumps(kwargs, indent=2)

if 'adcp' in locationname:
                publishMqtt(json_data,'acdcp')
            elif 'aquadopp' in locationname:
                publishMqtt(json_data,'aquadopp')
            elif 'ctd' in locationname:
                publishMqtt(json_data, 'ctd')
            elif 'raw_wind' in locationname:
                publishMqtt(json_data, 'raw')
            elif 'wave' in locationname:
                publishMqtt(json_data, 'wave')
            elif 'wind' in locationname:
                publishMqtt(json_data, 'wind')

The 'adcp', 'aquadopp' and other tags makes sure the data gets sent to the correct data converter on Thingsboard.  A sample json package sent by the script to the server might look something like this 

E39_F_Vartdalsfjorden_wind
Json Data:  {
  "time": 1553426576000.0,
  "latitude": 62.22162247,
  "longitude": 5.89998245,
  "WindSpeed": 4.511720180511475,
  "WindGust": 7.96875,
  "WindDirection": 147.3046875,
  "deviceName": "E39_F_Vartdalsfjorden_wind"
}

The catalog

To a

'http://thredds.met.no/thredds/catalog/obs/buoy-svv-e39/' + year + '/' + month + '/' + 'catalog.html'