Create a post-processing script
These scripts are run automatically after an inference run
Some information to consider if you'd like your script to be run automatically after an inference run:
Most R/python packages are installed already installed. Try to run your script on the conda environment defined on the submission page (or easier if you are not set up on MARCC, ask me)
There will be some variables set in the environment. These variables are:
$CONFIG_PATH
the path to the configuration file$FLEPI_RUN_INDEX
the run id for this run (e.g `CH_R3_highVE_pesImm_2022_Jan29
`$JOB_NAME
this job name (e.gUSA-20230130T163847_inference_med
)$FS_RESULTS_PATH
the path where lies the model results. It's a folder that contains the model_ouput/ as a subfolder$FLEPI_PATH
path of the flepiMoP repository.$DATA_PATH
path of the Data directory (e.g Flu_USA or COVID19_USA).Anything you ask can theoretically be provided here.
The script must run without any user intervention.
The script is run from $DATA_PATH.
Your script lies in the flepiMoP directory (preferably) or it's ok if it is in a data directory if it makes sense.
It is run on a 64Gb of RAM multicore machine. All scripts combined must complete under 4 hours, and you can use multiprocessing (48 cores)
Outputs (pdf, csv, html, txt, png ...) must be saved in a directory named
pplot/
(you can assume that it exists) in order to be sent to slack by FlepiBot 🤖 after the run.an example postprocessing script (in python) is here.
You can test your script on MARCC on a run that is already saved in
/data/struelo1/flepimop-runs
or I can do it for you.Once your script works, add (or ask to add) the command line to run in file
batch/postprocessing_scripts.sh
(here) between the START and END lines, with a little comment about what your script does.
Last updated