Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Basic rules

  • always check the location where you read/write files and space availability. See df command.
  • AVOID using nfs file systems/mounts for your simulations, especially when running on multiple nodes.
    This will double sometimes your execution time since for every read/write operation an extra IOwait time is needed to transfer information over the network.
    Use a location mounted on a local disk, usually /export/work/export-b/work/ or /scratch (on HPC nodes) and move your data after finishing in a desired location.

To learn

Commands

Local Linux Cluster

  • info about the cluster status using ganglia (works only internally).  If you choose then "cluster linux" , you see in "choose a node" the available computers.
  • from outside, you have first to connect via ssh  to login.stud.ntnu.no (or login.ansatt.ntnu.no). From there you can connect in the next step to a local computer.
  • use nice/renice to lower the priority of your processes
  • you can ran processes in parallel using MPI. Multithread libraries in C++, such as pthread, are also available.

...

ssh -XYC username@login.stud.ntnu.no
ssh -XYC computername.phys.ntnu.no
# Open browser
firefox -no-remote -no-xshm http://bird/ganglia/
# links http://bird/ganglia/
# lynx http://bird/ganglia/


SoftwareAll centralized astro-software (binaries and source) is placed under /cluster/apps/software/, which should be exported as the environmental variable $SOFTWARE.
To use it, run first "ml

...

Note: currently only bash shell is supported (SHELL=/bin/bash).

Some python packages are installed via anaconda (https://anaconda.cloud/package-categories/scientific). To load it type:

$ ml anaconda3
Then run:
$ init-conda
or
$ eval "$(conda shell.bash hook)"

Then to see the available environments type:

$ conda env list

# conda environments:
#
#base                  *  /cluster/apps/software/anaconda3/2022.05
#astroconda               /cluster/apps/software/anaconda3/2022.05/envs/astroconda
#astrosource              /cluster/apps/software/anaconda3/2022.05/envs/astrosource
#ciao                     /cluster/apps/software/anaconda3/2022.05/envs/ciao
#fermi                    /cluster/apps/software/anaconda3/2022.05/envs/fermi
#iraf                     /cluster/apps/software/anaconda3/2022.05/envs/iraf
#python-2.7               /cluster/apps/software/anaconda3/2022.05/envs/python-2.7

More details on all the available astro-software:

  • ciao: Chandra Interactive Analysis of Observations (https://cxc.cfa.harvard.edu/ciao/)
    • Description: CIAO is the software package developed by the Chandra X-Ray Center for analysing data from the Chandra X-ray Telescope. It can also be used with data from other Astronomical observatories, whether ground or space based.
    • To load it:
    • Version installed:
    • Acknowledgement:
  • starlink: Starlink Software Collection (https://starlink.eao.hawaii.edu/starlink/2021A
    • Description: The Starlink Software Collection provides access to a large number of Astronomical data reduction, analysis, and visualisation tools.
    • To load it:
    • Version installed: 2021A ?
    • Acknowledgement: 
  • TRM: Tom Marsh's ultracam and molly (and related) software packages (https://cygnus.astro.warwick.ac.uk/phsaap/software/)
    • Description: molly: 1D spectrum analysis program. ultracam: aperture photometry pipeline.
    • To load it:
    • Version installed:
    • Acknowledgement: T. Marsh, V. Dhillon, ...
  • astroconda: Third-party add-on channel to provide easy access to STScI’s software packages (https://astroconda.readthedocs.io/en/latest/#
    • Description: AstroConda is a free Conda channel maintained (until Feb. 2023) by the Space Telescope Science Institute (STScI) in Baltimore, Maryland. This channel provides tools and utilities required to process and analyze data from the Hubble Space Telescope (HST), James Webb Space Telescope (JWST), and others.
    • To load it:
    • Version installed:
    • Acknowledgement: STScI
  • psrcat: ATNF pulsar catalog (https://www.atnf.csiro.au/research/pulsar/psrcat/)
    • Description:
    • To load it:
    • Version installed: $ psrcat -v. Software version: 1.50. Catalogue version number = 1.69
    • Acknowledgement: Manchester, R. N., Hobbs, G. B., Teoh, A. & Hobbs, M., Astron. J., 129, 1993-2006 (2005) (astro-ph/0412641)
  • ne2001 and ymw16: Dispersion measure (pulsar) distance models.
    • Description:
    • To load it:
    • Version installed: 
    • Acknowledgement:

NE2001: Cordes & Lazio (2002) https://arxiv.org/abs/astro-ph/0207156
See also: /cluster/apps/software/ASTRO/ne2001/src.NE2001/code.ps

YMW16: Yao, Manchester and Wang, 2017, Astrophys. J., 835, 29; arXiv:1610.09448
See also: /cluster/apps/software/ymw16/src/README

...