Resources

Architecture

sequenceDiagram
    box Flywheel
       actor User
       participant Job queue
       participant Assets
    end
    box HPC
       participant the `hpc-client`
       participant Scheduler
       participant Compute Node
    end

    User->>Job queue: Submit job
    Note over the `hpc-client`: the `hpc-client` wakes up on cron
    the `hpc-client`->>Job queue: Poll for HPC jobs
    Job queue->>the `hpc-client`: Return jobs
    loop
       the `hpc-client`->> Scheduler: Submit job
       Compute Node->>Scheduler: Poll task
       Scheduler->>Compute Node: Return "fw engine" task
       Note over Compute Node: Engine starts
       Assets->>Compute Node: Pull Docker image and job inputs / config
       Note over Compute Node: Gear execute with singularity runner
       Compute Node->>Assets: Push job outputs
       Compute Node->>Job queue: Mark job as complete
    end

HPC Types

The client, also called the hpc-client, can support several queue mechanisms out of the box. Flywheel, however, currently only provides support for Slurm. If you require assistance with other schedulers, contact Flywheel.

If your site uses one of these, it may well just need a config file to get running.

Otherwise, some light python development will be required.