LogoLogo
JHU-IDDCOVID-19 Scenario Modeling hubCOVID-19 Forecast Hub
  • Home
  • 🦠gempyor: modeling infectious disease dynamics
    • Modeling infectious disease dynamics
    • Model Implementation
      • flepiMoP's configuration file
      • Specifying population structure
      • Specifying compartmental model
      • Specifying initial conditions
      • Specifying seeding
      • Specifying observational model
      • Distributions
      • Specifying time-varying parameter modifications
      • Other configuration options
      • Code structure
    • Model Output
  • 📈Model Inference
    • Inference Description
    • Inference Implementation
      • Specifying data source and fitted variables
      • (OLD) Configuration options
      • (OLD) Configuration setup
      • Code structure
    • Inference Model Output
    • Inference with EMCEE
  • 🖥️More
    • Setting up the model and post-processing
      • Config writer
      • Diagnostic plotting scripts
      • Create a post-processing script
      • Reporting
    • Advanced
      • File descriptions
      • Numerical methods
      • Additional parameter options
      • Swapping model modules
      • Using plug-ins 🧩[experimental]
  • 🛠️How To Run
    • Before any run
    • Quick Start Guide
    • Multiple Configuration Files
    • Synchronizing Files
    • Advanced run guides
      • Running with Docker locally 🛳
      • Running locally in a conda environment 🐍
      • Running on AWS 🌳
      • Running On A HPC With Slurm
    • Common errors
    • Useful commands
    • Tips, tricks, FAQ
  • 🗜️Development
    • Git and GitHub Usage
    • Guidelines for contributors
  • Deprecated pages
    • Module specification
  • JHU Internal
    • US specific How to Run
      • Running with Docker locally (outdated/US specific) 🛳
      • Running on Rockfish/MARCC - JHU 🪨🐠
      • Running with docker on AWS - OLD probably outdated
        • Provisioning AWS EC2 instance
        • AWS Submission Instructions: Influenza
        • AWS Submission Instructions: COVID-19
      • Running with RStudio Server on AWS EC2
    • Inference scratch
Powered by GitBook
On this page
Edit on GitHub
Export as PDF
  1. More
  2. Setting up the model and post-processing

Create a post-processing script

These scripts are run automatically after an inference run

PreviousDiagnostic plotting scriptsNextReporting

Last updated 2 months ago

Some information to consider if you'd like your script to be run automatically after an inference run ;

  • Most R/python packages are installed already installed. Try to run your script on the conda environment defined on the (or easier if you are not set up on MARCC, ask me)

  • There will be some variables set in the environment. These variables are:

    • $CONFIG_PATH the path to the configuration fil ;

    • $FLEPI_RUN_INDEX the run id for this run (e.g `CH_R3_highVE_pesImm_2022_Jan29`

    • $JOB_NAME this job name (e.g USA-20230130T163847_inference_med)

    • $FS_RESULTS_PATH the path where lies the model results. It's a folder that contains the model_ouput/ as a subfolder

    • $FLEPI_PATH path of the flepiMoP repository.

    • $PROJECT_PATH path of the Data directory (e.g Flu_USA or COVID19_USA).

    • Anything you ask can theoretically be provided here.

  • The script must run without any user intervention.

  • The script is run from $PROJECT_PATH.

  • Your script lies in the flepiMoP directory (preferably) or it's ok if it is in a data directory if it makes sense ;

  • It is run on a 64Gb of RAM multicore machine. All scripts combined must complete under 4 hours, and you can use multiprocessing (48 cores)

  • Outputs (pdf, csv, html, txt, png ...) must be saved in a directory named pplot/ (you can assume that it exists) in order to be sent to slack by FlepiBot 🤖 after the run.

  • an example postprocessing script (in python) is .

  • You can test your script on MARCC on a run that is already saved in /data/struelo1/flepimop-runs or I can do it for you.

  • Once your script works, add (or ask to add) the command line to run in file batch/postprocessing_scripts.sh between the START and END lines, with a little comment about what your script does.

🖥️
submission page
here
(here)