Getting Started
-
Before using the
hpc-client
, you need to decide how it will run on your cluster.
Choose an integration method and keep it in mind for later. This sets how frequently thehpc-client
will look for, pull, and queue hpc jobs to your HPC from your Flywheel site. -
It is strongly recommended that you make a private GitHub repo to track your changes.
This will make thehpc-client
much easier to manage. -
Perform the initial cluster setup. If you are unfamiliar with singularity, it is recommended that you read--at a minimum--SingularityCE's introduction and quick start guides.
-
Create an authorization token so Singularity and Flywheel can work with each other.
-
If your queue type is not in the above table, or is sufficiently different, review the guide for adding a queue type.
-
Collaborate with Flywheel staff to install the Flywheel engine in your HPC repo. They will also configure the hold engine on your Flywheel site to ensure that other engines do not pick up gear jobs that are tagged with "hpc".
-
Complete the integration method you chose in step one.
Confirm thehpc-client
is running regularly by monitoringlogs/cast.log
and the Flywheel user interface. -
Test and run your first HPC job tests in collaboration with Flywheel. It is recommended that you test with MRIQC (non-BIDS version), a gear that's available from Flywheel's Gear Exchange.
Note: as of 11 May 2022, Flywheel will have to change the rootfs-url (location of where the Docker image resides) for any gears installed from the Gear Exchange. For more about how thehpc-client
uses a rootfs-url, see Background/Motivation of this article. -
Enjoy!