... | ... | @@ -8,3 +8,33 @@ Slurm provides the [sbcat](https://slurm.schedmd.com/sbcast.html) to that purpos |
|
|
We will illustrate this technique through a [basic example](https://gitlab.oca.eu/DSI/HPC/tree/master/SLURM/local_data) that compute quadratic mean of floating point input files.
|
|
|
|
|
|
You can install the example with the [build.sh](https://gitlab.oca.eu/DSI/HPC/tree/master/SLURM/local_data/build.sh) script.
|
|
|
```
|
|
|
$ ./build.sh
|
|
|
building quad_mean..done
|
|
|
building gen_input..done
|
|
|
will generate 100 input files.
|
|
|
generating inputs/data_0 done
|
|
|
...
|
|
|
generating inputs/data_99 done
|
|
|
```
|
|
|
[build.sh](https://gitlab.oca.eu/DSI/HPC/tree/master/SLURM/local_data/quad_mean.cpp) is a small program that read a list of floating point values from it input (here stored in the input files in the `inputs` current directory) and print out theirs quadratic mean.
|
|
|
|
|
|
We will pretend that accessing those files from the shared disk space is too costly and that we need to copy them to the compute node '/tmp' space.
|
|
|
|
|
|
## Copying the files in the compute nodes '/tmp'
|
|
|
|
|
|
We will use the [sbcat](https://slurm.schedmd.com/sbcast.html) utility to do that.
|
|
|
|
|
|
**sbcast** can only be run from a Slurm script and is aware of the compute nodes assigned to the job. So it will assume its target directories and files are to be interpreted in the context of the compute nodes.
|
|
|
|
|
|
This is an extract of the relevant section of the [quand_mean.slurm](https://gitlab.oca.eu/DSI/HPC/tree/master/SLURM/local_data/quad_mean.slurm) submission script:
|
|
|
|
|
|
```
|
|
|
mkdir /tmp/$SLURM_JOB_ID/
|
|
|
|
|
|
for i in $(ls inputs)
|
|
|
do
|
|
|
sbcast ./inputs/$i /tmp/$SLURM_JOB_ID/$i
|
|
|
done
|
|
|
```
|
|
|
|