Cron

To fetch data from different data providers several Python scripts were created to download the data and publish it to the MQTT-broker. In order to consistently fetch new data from different sources cron is utilized to make sure that the scripts are executed and disposed of properly. Each script is a cron job that is executed at a specific time and date. For example, the weather data scripts run every 5 mintues. A crontab holds every task and when it is scheduled to be executed.

Managing the cronjobs

To see the current cronjobs configured you use the following bash-command.

$ crontab -l

To edit or add cronjobs, the following bash-command is used.

$ crontab -e

Checking the status of running cronjobs can be done by the following bash-command.

$ sudo /etc/init.d/cron status

Or,

$ sudo grep CRON /var/log/syslog

Configuring cronjobs

Configuration file

The current configuration of the cronjobs are shown below.

# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/aalesund.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/trondheim.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/hareid.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/festoy.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/solavaagen.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/magerholm.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/sykkylven.py
*/5 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/open-weather-map-scripts/sulesund.py
*/15 * * * * /usr/bin/timeout -s SIGTERM 600 /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/buoy-data/XarrayBuoyData.py >> /home/administrator1/Logs.txt 2>&1
12 * * * * /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/simulations/dummy-ferry/ferry-sim.py

Time scheduling

Scripts such as the weather-scripts are executed every 5 minutes. This is denoted by ,

*/5 * * * *

Where */5 tells cron that the should be executed every fifth minute. The following, * * * *, means it should be executed any hour, day, month, and day of the week.

Properly disposing script and log output

In some cases such as with the XarrayBuoyData.py the script won't properly dispose of itself. Resulting in the script still taking up RAM-memory. If this isn't handled correctly this will lead to the Thingsboard-instance to crashing, due to lack of memory. This is solved by adding a timeout to the script. If the script is still taking up memory after 10 minutes it will be terminated. The entire configuration for executing XarrayBuoyData.py is showcased below.

*/15 * * * * /usr/bin/timeout -s SIGTERM 600 /usr/bin/python3.7 /home/administrator1/scripts/mi-iot/buoy-data/XarrayBuoyData.py >> /home/administrator1/Logs.txt 2>&1

After 600 seconds, cron sends a SIGTERM signal which terminates the process. The timeout is created by,

/usr/bin/timeout -s SIGTERM 600

The script is executed by,

/usr/bin/python3.7 /home/administrator1/scripts/mi-iot/buoy-data/XarrayBuoyData.py

Finally, both the scripts output and error messages is logged to a text file by the following command.

>> /home/administrator1/Logs.txt 2>&1

Additional documentation

For more documentation regarding Cron, we encourage you to check out their

Wiki here.