SKA SDP Distributed Self-Cal Prototype

Prototype self-calibration pipeline to distribute processing across HPC cluster nodes, allowing scalability for large datasets and a reduction in overall computation time.

The pipeline can currently perform major and minor cycles to produce a clean image from a calibrated zarr dataset but more functionality is being actively developed.

The project repository can be found here.

This pipeline uses the SKA SDP Processing Function Library (https://gitlab.com/ska-telescope/sdp/ska-sdp-func) for gridding visibilities and the SwiFTly algorithm (https://gitlab.com/ska-telescope/sdp/ska-sdp-exec-swiftly) to perform distributed imaging. Dask (https://github.com/dask/dask) is used as the distributed computing framework to handle the scaling of computations on a HPC cluster.

• Installation
  • Installation using conda
  • Installation using spack

• Configuration

• Pipelines

• Tutorials

• Development
  • Make Commands
    • Run Tests
    • Build Docs Locally
    • Run All Checks

• Architecture

• API
  • deconvolution
  • swiftly
  • visibility_bin
  • visibility_io
  • binning
  • clean_beam
  • gridding
  • utils_fits
  • pipeline_config
  • tasks
  • utils
  • constants
  • units

Releases

• Changelog