All the little things to save you time on the clusters
model_output/ (or any big folder) is too long on the clusterYes, it takes ages because IO can be so slow, and there are many small files. If you are in a hurry, you can do
The first command rename/move model_output, it is instantaneous. You can now re-run something. To delete the renamed folder, run the second command. the & at the end makes it execute in the background.
seff to analyze a jobAfter a job has run (either to completion or got terminated/fail), you may run:
to know how much ressources your job used in your node, what was the cause for termination and so on. If you don't remember the JOB_ID, look for the number in the filename of the slurm log (slurm_{JOB_ID}.out).