1 - Choose your integration method
Each time cast is launched, it moves any eligible Flywheel jobs into the HPC scheduler. This should be a fairly quick process, and it is likely a good idea to run it fairly frequently.
Depending on your cluster admins, you may want to launch the hpc-client
a variety of different ways. Check with your admin as to the best option for you.
Option 1 - cron
If cron is enabled, this is probably the best choice. The below example would run the hpc-client
once a minute.
-
Create a new crontab file. Note: you may have to prepend
sudo
if there are any permission issues. -
Select a text editor if prompted.
-
Enter the task for cron to run (at bottom of the file).
If the script does not run, try using
bash
orsh
in the command:Depending on your setup in Cluster Setup, you may have a different directory than
~/hpc-client-config/
as your hpc-client configuration directory. -
Save and exit. If you're using nano, save the file with
control + O
, and hit enter when prompted about the file name and location. Exit withcontrol + X
.
For further options, check out crontab guru or your system documentation.
Option 2 - tmux
If tmux is enabled, this is a very simple way to regularly run cast. Simply launch a named tmux session:
Then, a simple bash loop can take care of the rest:
By default, Ctrl+B
then d
exits. See the tmux cheatsheet or your system documentation for more.
Option 3 - ssh
If neither option is available, it may be possible to further run the hpc-client
on a separate device, and SSH into the cluster when there are jobs available. This would require additional development work to be feasible, and may not work for your cluster, in particular if there are MFA constraints on SSH sessions.
If exploring this option, let us know, we'd be interested to hear about it.