Inference with EMCEE
Config Changes Relative To Classical Inference
The major changes are:
Under the 'inference' section add method: emcee entry, and
Under the 'statistics' section move the resample specific configuration under a 'resample' subsection as show bellow:
In addition to those configuration changes there are now new likelihood statistics offered: pois, norm/norm_homoskedastic, norm_cov/norm_heteroskedastic, nbinom, rmse, absolute_error. As well as new regularizations: forecast and allsubpops.
Running Locally
You can test your updated config by running:
If it works, it should produce:
Plots of simulation directly from your config,
Plots after the fits with the fits and the parameter chains,
An h5 file with all the chains, and
It will also immediately produce standard out that is similar to (dependent on config):
Here, it says the config fits 92 parameters, we'll keep that in mind and choose a number of walkers greater than (ideally 2 times) this number of parameters.
Running On An HPC Environment With Slurm
First, install flepiMoP on the cluster following the guide. Then manually create a batch file to submit to slurm like so:
Breaking down what each of these lines does:
#SBATCH --ntasks 1: Requests that this be run as a single job,
#SBATCH --nodes 1: Requests that the job be run on 1 node, as of right now EMCEE only supports single nodes,
#SBATCH --mem 450g: Requests that the whole job get 405GB of memory should be ~2-3GB per a walker,
For more details on other options provided by gempyor for calibration please see flepimop-calibrate --help.
Postprocessing EMCEE
At this stage postprocessing for EMCEE outputs is fairly manual. A good starting point can be found in postprocessing/emcee_postprocess.ipynb which plots the chains and can run forward projections from the sample drawn from calibration.
