Local Linux Cluster
- info about the cluster status using ganglia (works only internally). If you choose then "cluster linux" , you see in "choose a node" the available computers.
- from outside, you have first to connect via ssh to login.stud.ntnu.no (or login.ansatt.ntnu.no). From there you can connect in the next step to a local computer.
- use nice/renice to lower the priority of your processes
- you can ran processes in parallel using MPI. Multithread libraries in C++, such as pthread, are also available.
Ganglia
Ganglia provides a web-based user interface that can be used to view activity and available processes on the Linux cluster. It can only be accessed internally. To do this, use ssh to access one of the computers in the cluster.
If one wants to use a graphical web browser, such as FireFox, one needs to use the optional argument -X (enables X11 forwarding). The arguments -Y (enables trusted X11 forwarding) and C (Requests compression of all data) can speed up the web browser.
Alternatively, you can use text-based web browsers such as "links" or "lynx".
Example:
ssh -XYC username@login.stud.ntnu.no
ssh -XYC computername.phys.ntnu.no
# Open browser
firefox -no-remote -no-xshm http://bird/ganglia/
# links http://bird/ganglia/
# lynx http://bird/ganglia/
Software
All centralized astro-software (binaries and source) is placed under /cluster/apps/software
To use astro-software you should have these lines in your .bashrc so you will be able to see all the modulefiles for the configured workspaces and software.
if [ -d "/cluster/apps/software/_profile" ]; then if [ -z $SOFTWARE ]; then source "/cluster/apps/software/_profile/globals.bash" fi fi if [ -d "/cluster/apps/software/.profile.d" ]; then if [ -z $SOFTWARE ]; then source "/cluster/apps/software/.profile.d/globals.bash" fi fi
#show available modules to load [username@computername ~]$ ml av ----------------------------------------------- /cluster/apps/software/_profile/lmod/WORKSPACE ----------------------------------------- ASTRO CORE ----------------------------------------- /cluster/apps/spack/2022a/lmod/linux-fedora36-x86_64/Core ------------------------------------ # load the ASTRO module then show again the available modules [username@computername ~]$ ml ASTRO [username@computername ~]$ ml av ------------------------------------------- /cluster/apps/software/_profile/lmod/modulefiles/ASTRO ------------------------------------- CALDB conda+/ciao conda+/fermi (D) ds9/8.3 heasoft/6.30.1 heasoft/6.31 (D) iraf/2.17 starlink/fc36 starlink/2021A (D) trm ---------------------------------------------- /cluster/apps/software/_profile/lmod/WORKSPACE ------------------------------------------ ASTRO (L) CORE ----------------------------------------- /cluster/apps/spack/2022a/lmod/linux-fedora36-x86_64/Core ------------------------------------ anaconda3/2021.05 anaconda3/2022.05 (D) gcc/10.2.0
Run first ml ASTRO to load the ASTRO workspace. This will set the environment and make available the ASTRO software for loading.
For questions or software updates, contact Bogdan.
- Anaconda
Some python packages are installed via anaconda (https://anaconda.cloud/package-categories/scientific). To load it type:
[username@computername ~]$ ml anaconda3 ################################################################################# #### Please remember to run "init-conda" to fully setup the environment #### Run "conda env list" to list all the environments #### Load environment "conda activate env_name" where env_name is one from the list #### Remember to run "conda deactivate" before unload/purge anaconda3 #### otherwise you will have to restart your shell to avoid possible issues ################################################################################# [username@computername ~]$ init-conda [username@computername ~]$ conda env list # conda environments: # base /cluster/apps/software/anaconda3/2022.05 astroconda /cluster/apps/software/anaconda3/2022.05/envs/astroconda astrosource /cluster/apps/software/anaconda3/2022.05/envs/astrosource ciao /cluster/apps/software/anaconda3/2022.05/envs/ciao fermi /cluster/apps/software/anaconda3/2022.05/envs/fermi iraf /cluster/apps/software/anaconda3/2022.05/envs/iraf phoebe /cluster/apps/software/anaconda3/2022.05/envs/phoebe posydon /cluster/apps/software/anaconda3/2022.05/envs/posydon python-2.7 /cluster/apps/software/anaconda3/2022.05/envs/python-2.7 [username@computername ~]$ conda activate astrosource (astrosource) [username@computername ~]$
- heasoft: A Unified Release of the FTOOLS and XANADU Software Packages (https://heasarc.gsfc.nasa.gov/docs/software/heasoft/)
- Description:
XANADU: High-level, multi-mission tasks for X-ray astronomical spectral, timing, and imaging data analysis (Xspec, Xronos, Ximage, qdp...)
FTOOLS: General and mission-specific tools to manipulate FITS files (https://heasarc.gsfc.nasa.gov/lheasoft/ftools/ftools_subpacks.html)
FITSIO: Core library responsible for reading and writing FITS files
fv: General FITS file browser/editor/plotter with a graphical user interface
XSTAR: Tool for calculating the physical conditions and emission spectra of photoionized gases - To load it: ml ASTRO heasoft
which also loads CALDB automatically - Version installed (fversion): 6.31 (6.30)
- Acknowledgement:
If FTOOLS has been useful in your research, please reference this site (http://heasarc.gsfc.nasa.gov/ftools) and use the ASCL reference for HEASoft [ascl:1408.004] or the ASCL reference for the original FTOOLs paper [ascl:9912.002]:
Blackburn, J. K. 1995, in ASP Conf. Ser., Vol. 77, Astronomical Data Analysis Software and Systems IV, ed. R. A. Shaw, H. E. Payne, and J. J. E. Hayes (San Francisco: ASP), 367.
- Description:
- caldb: HEASARC's calibration database (CALDB) system (https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_intro.html)
- Description: The HEASARC's calibration database (CALDB) system stores and indexes datasets associated with the calibration of high energy astronomical instrumentation.
- To load it: ml ASTRO CALDB
- Version installed: LAST UPDATE: 20220331
- Acknowledgement: see HEASOFT above.
- ciao: Chandra Interactive Analysis of Observations (https://cxc.cfa.harvard.edu/ciao/)
- Description: CIAO is the software package developed by the Chandra X-Ray Center for analysing data from the Chandra X-ray Telescope. It can also be used with data from other Astronomical observatories, whether ground or space based.
- To load it: ml anaconda3 then init-conda then conda activate ciao
or from own module with: ml ASTRO conda+/ciao - Version installed (ciaover): caldb_main 4.9.7, ciao 4.14.0, ciao-contrib 4.14.2, ds9 8.3, sherpa 4.14.0
- Acknowledgement: Fruscione et al. 2006, SPIE Proc. 6270, 62701V, D.R. Silvia & R.E. Doxsey, eds.
- fermitools: Fermi-LAT Data Analysis Tools (https://fermi.gsfc.nasa.gov/ssc/data/analysis/software/ https://github.com/fermi-lat/Fermitools-conda/wiki/The-Fermitools-and-Conda)
- Description: The Fermitools (also known as the Fermi ScienceTools) is a suite of data analysis software provided by the Fermi-LAT collaboration that is used in the analysis of data produced by the Fermi Gamma-Ray Space Telescope.
- To load it: ml anaconda3 then init-conda then conda activate fermi
or from own module with: ml ASTRO conda+/fermi - Version installed (conda list fermi): fermipy 1.2, fermitools 2.2.0, fermitools-data 0.18
- Acknowledgement: FSSC https://fermi.gsfc.nasa.gov/ssc/data/analysis/software/
- starlink: Starlink Software Collection (https://starlink.eao.hawaii.edu/starlink/2021A)
- Description: The Starlink Software Collection provides access to a large number of Astronomical data reduction, analysis, and visualisation tools.
- To load it: ml ASTRO starlink then init-starlink
- Version installed: 2021A
- Acknowledgement: “The Starlink software (Currie et al 2014) is currently supported by the East Asian Observatory.” Currie et al. 2014 2014ASPC..485..391C.
- TRM: Tom Marsh's ultracam and molly (and related) software packages (https://cygnus.astro.warwick.ac.uk/phsaap/software/ https://github.com/trmrsh)
- Description: molly: 1D spectrum analysis program. ultracam: aperture photometry pipeline. See all packages and documentation at /cluster/apps/software/trm/trm/html/
- To load it: ml ASTRO trm then init-starlink
- Version installed: molly v. 1.1.8, ultracam 8.1.17
- Acknowledgement: T. Marsh.
- ds9: SAO Image ds9 (https://sites.google.com/cfa.harvard.edu/saoimageds9)
- Description: An image display and visualization tool for astronomical data.
- To load it: ml ASTRO ds9
- Version installed: 8.3
- Acknowledgement: SAOImageDS9 development has been made possible by funding from the Chandra X-ray Science Center (CXC), the High Energy Astrophysics Science Archive Center (HEASARC) and the JWST Mission office at Space Telescope Science Institute. To cite SAOImageDS9 in your paper, please use: 2003adass..295..489J
- astroconda: Third-party add-on channel to provide easy access to STScI’s software packages (https://astroconda.readthedocs.io/en/latest/#)
- Description: AstroConda is a free Conda channel maintained (until Feb. 2023) by the Space Telescope Science Institute (STScI) in Baltimore, Maryland.
This channel provides tools and utilities required to process and analyze data from the Hubble Space Telescope (HST), James Webb Space Telescope (JWST), and others. - To load it: ml anaconda3 then init-conda then conda activate astroconda
- Version installed:
- Acknowledgement: STScI
- Description: AstroConda is a free Conda channel maintained (until Feb. 2023) by the Space Telescope Science Institute (STScI) in Baltimore, Maryland.
- astrosource: Analysis script for sources with variability in their brightness (https://pypi.org/project/astrosource/).
- Description:
- To load it: ml anaconda3 then init-conda then conda activate astrosource
- Version installed:
- Acknowledgement: Fitzgerald et al. (2021)
- iraf: Image Reduction and Analysis Facility (https://iraf-community.github.io/)
- Description:
- To load it: ml ASTRO iraf; init-iraf
- To set it up (first time only): setup-iraf
- Version installed: 2.17
- Acknowledgement:
- psrcat: ATNF pulsar catalog (https://www.atnf.csiro.au/research/pulsar/psrcat/)
- Description:
- To load it:
- Version installed: $ psrcat -v. Software version: 1.50. Catalogue version number = 1.69
- Acknowledgement: Manchester, R. N., Hobbs, G. B., Teoh, A. & Hobbs, M., Astron. J., 129, 1993-2006 (2005) (astro-ph/0412641)
- NE2001 and ymw16: Dispersion measure (pulsar) distance models.
- Description:
- To load/start it use NE2001 and ymw16 commands
- Version installed:
- Acknowledgement:
NE2001: Cordes & Lazio (2002) https://arxiv.org/abs/astro-ph/0207156
See also: /cluster/apps/software/ASTRO/ne2001/src.NE2001/code.ps
YMW16: Yao, Manchester and Wang, 2017, Astrophys. J., 835, 29; arXiv:1610.09448
See also: /cluster/apps/software/ymw16/src/README
See also: https://apps.datacentral.org.au/pygedm/
/cluster/apps/software/ASTRO/pygedm/README.md