1 of 66

FlepiMoP Documentation

Home

Welcome to flepiMoP documentation!

The “FLexible EPIdemic MOdeling Pipeline” (flepiMoP; formerly known as the COVID Scenario Modeling Pipeline or CSP) is an open-source software suite designed by researchers in the Johns Hopkins Infectious Disease Dynamics Group and at UNC Chapel Hill to simulate a wide range of compartmental models of infectious disease transmission. The disease transmission and observation models are defined by a no-code configuration file, which allows models of varying complexity to be specified quickly and consistently, from simple problems described by SIR-style models in a single population to more complicated models of multiple pathogen strains transmitting between thousands of connected spatial divisions and age groups.

It was initially designed in early 2020 and was routinely used to provide projections of the emerging COVID-19 epidemic to health authorities worldwide. Currently, flepiMoP provides COVID-19 projections to the US CDC-funded model aggregation sites, the COVID-19 Forecast Hub and the COVID-19 Scenario Modeling Hub, influenza projections to FluSight and to the Flu Scenario Modeling Hub, and RSV projections to the RSV Scenario Modeling Hub.

However, the pipeline is much more general and can be used to simulate the dynamics of any infection that can be expressed as a compartmental epidemic model. These include applications in chemical reaction kinetics, pharmacokinetics, within-host disease dynamics, or applications in the social sciences.

In addition to producing forward simulations given a specified model and parameter values, the pipeline can also attempt to optimize unknown parameters (e.g., transmission rate, case detection rate, intervention efficacy) to fit the model to datasets the user provides (e.g., hospitalizations due to severe disease) using a Bayesian inference framework. This feature allows the pipeline to be utilized for short-term forecasting or longer-term scenario projections for ongoing epidemics, since it can simultaneously be fit to data for dates in the past and then use best-fit parameters to make projections into the future.

General description of flepiMoP

The main features of flepiMoP are:

Open-source (GPL v3.0) infectious dynamics modeling software, written in R and Python
Versatile, no-code design applicable for most compartmental models and outcome observation models, allowing for quick iteration in reaction to epidemic events (e.g., emergence of new variants, vaccines, non-pharmaceutical interventions (NPIs))
Powerful, just-in-time compiled disease transmission model and distributed inference engine ready for large scale simulations on high-performance computing clusters or cloud workflows
Adapted to small- and large-scale problems, from a simple SIR model to a complex model structure with hundreds of compartments on thousands of connected populations
Strong emphasis on mechanistic processes, with a design aimed at leveraging domain knowledge in conjunction with statistical inference
Portable for Windows WSL, MacOS, and Linux with the provided Docker image and an Anaconda environment

The mathematical model within the pipeline is a compartmental epidemic model embedded within a well-mixed metapopulation. A compartmental epidemic model is a model that divides all individuals in a population into a discrete set of states (e.g. “infected”, “recovered”) and tracks – over time – the number of individuals in each state and the rates at which individuals transition between these states. The well-known SIR model is a classic example of such a model, and much more complex versions of this model type have been simulated with this framework (for example, an SEIR-style model in which individuals are further subdivided into multiple age groups and vaccination statuses).

The structure of the desired model, as well as the parameter values and initial conditions, can be specified flexibly by the user in a no-code fashion. The pipeline allows for parameter values to change over time at discrete intervals, which can be used to specify time-dependent aspects of disease transmission and control (such as seasonality or vaccination campaigns).

The model is embedded within a meta-population structure, which consists of a series of distinct subpopulations (e.g. states, provinces, or other communities) in which the model structure is repeated, albeit with potentially different parameter values. The subpopulations can interact, either through the movement of individuals or the influence of individuals in one subpopulation on the transition rate of individuals in another ;

Within each subpopulation, the population is assumed to be well-mixed, meaning that interactions are assumed to be equally likely between any pair of individuals (since unique identities of individuals are not explicitly tracked). The same model structure can be simulated in a continuous-time deterministic or discrete-time stochastic manner ;

In addition to the variables described by the compartmental model, the model can track other observable variables (“outcomes”) that are functions of the basic model variables but do not themselves influence the dynamics (i.e., some portion of infections are reported as cases, depending on a testing rate). The model can be run iteratively to tune the values of certain parameters so that these outcome variables best match timeseries data provided by the user for a certain time period ;

Fitting is done using a Bayesian-like framework, where the user can specify the likelihood of observed outcomes in data given modeled outcomes, and the priors on any parameters to be fit. Multiple data streams (e.g., cases and deaths) can be fit simultaneously. A custom Markov Chain Monte Carlo method is used to sequentially propose and accept or reject parameter values based on the model fit to data, in a way that balances fit quality within each individual subpopulation with that of the total aggregate population, and that takes advantage of parallel computing environments.

The code is written in a combination of R and Python, and the vast majority of users only need to interact with the pipeline via the components written in R. It is structured in a modular fashion, such that individual components – such as the epidemic model, the observable variables, the population structure, or the parameters – can be edited or completely replaced without any handling of other parts of the code ;

When model simulation is combined with fitting to data, the code is designed to run most efficiently on a supercomputing cluster with many cores. We most commonly run the code on Amazon Web Services or on high-performance computers using SLURM. However, even relatively large models can be run efficiently on most personal computers. Typically, the memory of the machine will limit the number of compartments (i.e., variables) that can be included in the epidemic model, while the machine’s CPU will determine the speed at which each model run is completed and the number of iterations of the model that can be run during parameter searches when fitting the model to data. While the pipeline can be installed on any computer, it is sometime easier to use an Anaconda environment or the provided Docker container, where all the software dependencies (e.g., standardized R and Python versions along with required packages) are included, independent of the user’s local machine. All the code is maintained on our GitHub and shared with the GNU General Public License v3.0 license. It is build on top of a fully open-source software stack.

This documentation is organized as follows. The Model Description section describes the mathematical framework for the compartmental epidemic models that can be simulated forward in time by the pipeline. The Model Inference section describes the statistical framework for fitting the model to data. The Data and Parameter section describes the inputs the user must provide to the pipeline, in terms of the model structure and parameters, the population characteristics, the initial conditions, time-varying interventions, data to be fit, and more. The How to Run section provides concrete guidance on setting up and running the model and analyzing the output. The Quick Start Guide provides a simple example model setup. The Advanced section goes into more detail on specific features of the model and the code that are likely to only be of interest to users who want to run more complex models or data fitting routines or substantially edit the code. It includes a subsection describing each file and package used in the pipeline and their interactions during a model run.

Users who wish to jump to running the model themselves can see Quick Start Guide.

For questions about the pipeline or to report a bug, please use the “Issues” or "Discussions" feature on our GitHub.

Acknowledgments

flepiMoP is actively developed by its current contributors, including Joseph C Lemaitre, Sara L Loo, Emily Przykucki, Clifton McKee, Claire Smith, Sung-mok Jung, Koji Sato, Pengcheng Fang, Erica Carcelen, Alison Hill, Justin Lessler, and Shaun Truelove, affiliated with the ;

Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA for (JCL, JL)
Johns Hopkins University International Vaccine Access Center, Department of International Health, Baltimore, MD, USA for (SLL, KJ, EC, ST)
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA for (CM, CS, JL, ST)
Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA for (S-m.J, JL)
Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, USA for (AH).

The development of this model was supported by from funds the National Science Foundation (2127976; ST, CPS, JK, ECL, AH), Centers for Disease Control and Prevention (200-2016-91781; ST, CPS, JK, AH, JL, JCL, SL, CM, EC, KS, S-m.J), US Department of Health and Human Services / Department of Homeland Security (ST, CPS, JK, ECL, AH, JL), California Department of Public Health (ST, CPS, JK, ECL, JL), Johns Hopkins University (ST, CPS, JK, ECL, JL), Amazon Web Services (ST, CPS, JK, ECL, AH, JL, JCL), National Institutes of Health (R01GM140564; JL, 5R01AI102939; JCL), and the Swiss National Science Foundation (200021-172578; JCL)

We need to also acknowledge past contributions to the development of the COVID Scenario Pipeline, which evolved into flepiMoP. These include contributions by Heramb Gupta, Kyra H. Grantz, Hannah R. Meredith, Stephen A. Lauer, Lindsay T. Keegan, Sam Shah, Josh Wills, Kathryn Kaminsky, Javier Perez-Saez, Joshua Kaminsky, and Elizabeth C. Lee.

gempyor: modeling infectious disease dynamics

Modeling infectious disease dynamics

Within flepiMoP, gempyor is an open-source Python package that constructs and simulates compartmental infectious disease dynamics models. gempyor is meant to be used within flepiMoP, where it integrates with parameter inference and data processing scripts, but can also be run standalone with a command-line interface, generating simulations of disease incidence under different scenario assumptions.

To simulate an infectious disease dynamics problem, the following building blocks needs to be defined:

The over which the disease is transmitted
The , defining the compartments and the transitions between compartments
An , defining different observable outcomes (serology, hospitalization, deaths, cases) from the transmission model
The parameters and modifiers that apply to them

Generalized compartmental infection model

At the core of our pipeline is a dynamic mathematical model that categorizes individuals in the population into a discrete set of states ('compartments') and describes the rates at which transitions between states can occur. Our modeling approach was developed to describe classic infectious disease transmission models, like the SIR model, but is much more general. It can encode any compartmental model in which transitions between states are of the form

where , , and are time-dependent variables describing the number of individuals in each state, is a rate parameter (units of time) and is a scaling parameter (unitless). may be , , a different variable, or 1, and the rate may also be the sum of terms of this form. Rates that involve non-linear functions or more than two variables are currently not possible. For simplicity, we omitted the time dependencies on parameters (e.g is in fact and , are ,).

The model can be simulated as a continuous-time, deterministic process (i.e., a set of ordinary differential equations), which in this example would be in the form

Details on the numerical integration procedure for simulating such an equation is given in the section.

Alternatively, the model can be simulated as a discrete-time stochastic process, where the number of individuals transitioning between states and at time is a binomial random variable

where the second term is the expected fraction of individuals in the state at time who would transition to by time if there were no other changes to in this time, and time step is a chosen parameter that must be small for equivalence between continuous- and discrete-time versions of the model.

SEIR model

For example, an SEIR model – which describes an infection for which susceptible individuals () who are infected first pass through a latent or exposed () phase before becoming infectious () and that confers perfect lifelong immunity after recovery () – could be encoded as

where is the transmission rate (rate of infectious contact per infectious individual), is the rate of progression ( is the average latent/incubation period), and is the recovery rate ( is the average duration of the infectious period), and is the total population size (). In differential equation form, this model is

and as a stochastic process, it is

A common COVID-19 model is a variation of this SEIR model that incorporates:

multiple identical stages of the infectious period, which allows us to model gamma-distributed durations of infectiousness, and
an infection rate modified by a 'mixing coefficient', , which is a rough heuristic for the slowdown in disease spread that occurs in realistically heterogeneous populations where more well-connected individuals are infected first.

A three-stage infectious period model is given by

The flepiMoP model structure is specifically designed to make it simple to encode the type of more complex "stratified'' models that often arise in infectious disease dynamics. The following are some examples of possible stratifications.

Age groups

To describe an SEIR-type disease that spreads and progresses differently among children versus adults, one may want to repeat each compartment of the model for each of the two age groups (C – Children, A – Adults), creating an age-stratified model

where is the transmission rate between age X and Y, and we have assumed individuals do not age on the timescale relevant to the model.

Vaccination status

Vaccination status could influence disease progression and infectiousness, and could also change over time as individuals choose to get the vaccine (V – vaccinated, U – unvaccinated)

where is the vaccination rate (we assume that individuals do not receive the vaccine while they are exposed or infectious) and is the vaccine efficacy against infection. Similar structures could be used for other sources of prior immunity or other dynamic risk groups.

Pathogen strain

Another common stratification would be pathogen strain, such as COVID-19 variants. Individuals may be infected with one of several variants, strains, or serotypes. Our framework can easily create multistrain models, for example

where is the immune cross-protection conferred from infection with strain A to subsequent infection with strain B. Co-infection is ignored. All individuals are assumed to be initially equally susceptible to both infections and are just categorized as (vs ) for convenience.

All combinations of these situations can be quickly specified in flepiMoP. Details on how to encode these models is provided in the section, with examples given in the section.

Clinical outcomes and observations model

The pipeline allows for an additional type of dynamic state variable beyond those included in the mathematical model. We refer to these extra variables as "Outcomes" or "Observations". Outcome variables can be functions of model variables, but do not feed back into the model by influencing other state variables. Typically, we use outcome variables to describe the process through which some subset of individuals in a compartment are "observed'' and become part of the data to which models are compared and attempt to predict. For example, in the context of a model for an infectious disease like COVID-19, outcome variables include reported cases, hospitalizations, and deaths.

An outcome variable can be generated from a state variable of the mathematical model using the following properties:

The proportion of all individuals in who will be observed as ,
The delay between when an individual enters state and when they are observed as , which can follow a class of probability distributions where is the parameters of the distribution (e.g., the mean and standard deviation of a normal distribution)
(optional) the duration spent in observable , in which case the output will also contain the prevalence (number of individuals currently in in addition to the incidence into

In addition to single values (drawn from a distribution), the duration and delay can be inputted as distributions, producing a convolution of the output.

The number of individuals in at time who become part of the outcome variable is a random variable, and individuals who are observed in at time could have entered at different times in the past.

Formally, for a deterministic, continuous-time model

For a discrete-time, stochastic model

Note that outcomes constructed in this way always represent incidence values; meaning they describe the number of individuals newly entering this state at time . If the model state is also an incidence, then is a unitless probability, whereas if is a prevalence (number of individuals currently in state at time ), then is instead a probability per time unit.

Outcomes can also be constructed as functions of other outcomes. For example, a fraction of hospitalized patients may end up in the intensive care unit (ICU).

There are several benefits to separating outcome variables from the mathematical model. Firstly, these variables can be calculated after the model is run, and only at the timepoints of interest, which can dramatically reduce the memory needed during model simulation. Secondly, outcome variables can be fully stochastic even when the mathematical model is simulated deterministically. This becomes useful when an infection might be at high enough prevalence that a deterministic simulation is appropriate, but when there is a rare and therefore quite stochastic outcome reported in the data (e.g., severe cases) that the model is tasked with predicting. Thirdly, outcome variables can have arbitrary delay distributions, to take into account the complexities of health reporting practices, whereas our mathematical modeling framework is designed mainly for exponentially distributed delays and only easily permits extensions to gamma-distributed delays. Finally, this separation keeps the pipeline modular and allow for easy editing of one component of the model without disrupting the other.

Details on how to specify these outcomes in the model configuration files is provided in the section, with examples given in the section.

Population structure

The pipeline was designed specifically to simulate infection dynamics in a set of connected subpopulations. These subpopulations could represent geographic divisions, like countries, states, provinces, or neighborhoods, or demographic groups, or potentially even different host species. The equations and parameters of the transmission and outcomes models are repeated for each subpopulation, but the values of the parameters can differ by location. Within each subpopulation, infection is equally likely to spread between any pair of susceptible/infected individuals after accounting for their infection class, whereas between subpopulations there may be varying levels of mixing.

Formally, this type of population structure is often referred to as a “metapopulation”, and each subpopulation may be called a “deme”.

The following properties may be different between subpopulations:

the population size
the parameters of the transmission model (see LINK)
the parameters of the outcomes model (see LINK)
the amount of transmission that occurs within this subpopulation versus from any other subpopulation (see LINK)
the timing and extent of any interventions that modify these parameters (see LINK)
the initial timing and number of external introductions of infections into the population (see LINK)
the ground truth timeseries data used to compare to model output and infer model parameters (see LINK)

Currently, the following properties must be the same across all subpopulations:

the compartmental model structure
the form of the likelihood function used to estimate parameters by fitting the model to data (LINK)
...

Mixing between subpopulations

The generalized compartmental model allows for second order “interaction” terms that describe transitions between model states that depend on interactions between pairs of individuals. For example, in the context of a classical SIR model, the rate of new infections depends on interactions between susceptible and infectious individuals and the transmission rate

For a model with multiple subpopulations, each of these interactions can occur either between individuals in the same or different subpopulations, with specific rate parameters for each combination of individual locations

where is the per-contact per-time rate of disease transmission between an infected individual residing in subpopulation and a susceptible individual from subpopulation .

In general for infection models in connected subpopulations, the transmission rates can take on arbitrary values. In this pipeline, however, we impose an additional structure on these terms. We assume that interactions between subpopulations occur when individuals temporarily relocate to another subpopulation, where they interact with locals. We call this movement “mobility”, and it could be due to regular commuting, special travel, etc. There is a transmission rate () associated with each subpopulation , and individuals physically in that subpopulation – permanently or temporarily – are exposed and infected with this local rate whenever they encounter local susceptible individuals.

The transmission matrix is then

where is the onward transmission rate from infected individuals in subpopulation , is the number of individuals in subpopulation i who are interacting with individuals in subpopulation at any given time (for example, fraction who commute each day), and is a fractional scaling factor for the strength of inter-population contacts (for example, representing the fraction of hours in a day commuting individuals spend outside vs. inside their subpopulation).

The list of all pairwise mobility values and the interaction scaling factor are model input parameters. Details on how to specify them are given in the section.

If an alternative compartmental disease model is created that has other interactions (second order terms), then the same mobility values are used to determine the degree of interaction between each pair of subpopulations.

Initial conditions

Initial conditions can be specified by setting the values of the compartments in the disease transmission model at time zero, or the start of the simulation. For example, we might assume that for day zero of an outbreak the whole population is susceptible except for one single infected individual, i.e. and . Alternatively, we might assume that a certain proportion of the population has prior immunity from previous infection or vaccination.

It might also be necessary to model instantaneous changes in values of model variables at any time during a simulation. We call this 'seeding'. For example, individuals may import infection from other external populations, or instantaneous mutations may occur, leading to new variants of the pathogen. These processes can be modeled with seeding, allowing individuals to change state at specified times independently of model equations.

We also note that seeding can also be used as a convenient way to specify initial conditions, particularly ealy in an outbreak where the outbreak is triggered by a few 'seedings'.

Time-dependent interventions

Parameters in the disease transmission model or the observation model may change over time. These changes could be, for example: environmental drivers of disease seasonality; “non-pharmaceutical interventions” like social distancing, isolation policies, or wearing of personal protective equipment; “pharmaceutical interventions” like vaccination, prophylaxis, or therapeutics; changes in healthcare seeking behavior like testing and diagnosis; changes in case reporting, etc.

The model allows for any parameter of the disease transmission model or the observation model to change to a new value for a time interval specified by start and end times (or multiple start and end times, for interventions that are recurring). Each change may be subpopulation-specific or apply to the entire population. Changes may be overlapping in time.

The magnitude of these changes are themselves model parameters, and thus may along with other parameters when the model is fit to data. Currently, the start and end times of interventions must be fixed and cannot be varied or inferred.

For example, the rate of transmission in subpopulation , , may be reduced by an intervention that acts between times and , and another intervention that acts between times and

In this case, and are both considered simple SinglePeriodModifier interventions. There are four possible types of interventions that can be included in the model

SinglePeriodModifier - an intervention that leads to a fractional reduction in a parameter value in subpopulation (i.e., ) between two timepoints

MultiPeriodModifier - an intervention that leads to a fractional reduction in a parameter value in subpopulation (i.e., ) value between multiple sets of timepoints

ModifierModifier- an intervention that leads to a fractional reduction in the value of another intervention between two timepoints

StackedModifier - TBA

Model Implementation

flepiMoP's configuration file

About configuration files

flepiMop is set up so that all parameters and other options for running the pipeline can be specified in a single "configuration" file (aka "config"). Users do not need to edit any other code files, or even be aware of their contents, to create and run complex model scenarios. Configuration files also provide a convenient record of model options and promote reproducibility of model results.

We use the YAML language syntax to write config files, which are typically named something like config.yml. The file has simple plain text contents and follows a tabbed outline structure. When config files are read by the model code, a data structure encoding the model options is created.

Comments can be added to the config file by starting with the hash key (#) then a space. Comments can start anywhere on a line and continue until the end, but if they run over to a new line, a new # must be used at the start of the new line.

Example

(give a simple configuration for a toy model with two subpopulations, SEIR, single "cases" outcome, single seeded infection, single NPI that starts after some time? this page is currently under development, please see our example repo _for some simple configurations) ;

When referring to config items (individual parameters), we use their full position in the outline. For example, in the sample config file above, we denote

subpop_setup:
  ...
  geodata: minimal

as subpop_setup::geodata having a value of minimal

Notation

Parameters and other options specified in the configuration files can take on a variety of types of values, using the following notations:

dates are specified as [year]-[month]-[day]. (e.g., 2020-01-31)
boolean values are either "TRUE" or "FALSE"
files names are strings
probability is a float between 0 and 1
distribution is a probability distribution from which a random value for the parameter is drawn each time a new simulation is run (or chain, if doing inference). See here for the require schema.

Configuration files sections

Global header

Required section

These global configuration options typically sit at the top of the configuration file.

Item

Required?

Type/Format

Description

name

required

string

Name of this configuration. Will be used in file names created to store model output.

start_date

required

date

model simulation start date

end_date

required

date

model simulation end date

start_date_groundtruth

optional for non-inference runs, required for inference runs

date

start date for comparing model to data

end_date_groundtruth

optional for non-inference runs, required for inference runs

date

end date for comparing model to data

nslots

optional (can also be defined by an environmental variable)

int

number of independent simulations to run

setup_name

optional

string

setup name used to describe the run, used in setting up file names

model_output_dirname

optional

folder path

path to folder where all the outputs created by the model are stored, if not specified, default is model_output

For example, for a configuration file to simulate the spread of COVID-19 in the US during 2020 and compare to data from March 1 onwards, with 1000 independent simulations, the header of the config might read:

name: USA_covid19_2020
model_output_dirname: model_output
start_date: 2020-01-01
end_date: 2020-12-31
start_date_groundtruth: 2020-03-01
end_date_groundtruth: 2020-12-31
nslots: 1000

`subpop_setup` section

Required section

This section specifies the population structure on which the model will be simulated, including the names and sizes of each subpopulation and the connectivity between them. More details here.

`compartments` section

Required section

This section is where users can specify the variables (infection states) that will be tracked in the infectious disease transmission model. More details can be found here. The other details of the model are specified in the seir section, including transitions between these compartments (seir::transitions), the names of the parameters governing the transitions (seir::parameters), and the numerical method used to simulate the equations over time (seir::integration). The initial conditions of the model can be specified in the initial_conditions section, and any other inputs into the model from external populations or instantaneous transitions between states that occur at later times can be specified in the seeding section. ;

`seir` section

Required section

This section is where users can specify the details of the infectious disease transmission model they wish to simulate (e.g., SEIR). This model describes the allowed transitions (seir::transitions) between the compartments that were specified in the compartments section, the values of the parameters involved in these transitions (seir::parameters), and the numerical method used to simulate the equations over time (seir::integration). More details here. The initial conditions of the model can be specified in the separate initial_conditions section, and any other inputs into the model from external populations or instantaneous transitions between states that occur at later times can be specified in the seeding section. ;

`initial_conditions` section

Optional section

This section is used to specify the initial conditions of the model, which define how individuals are distributed between the model compartments at the time the model simulation begins. Importantly, the initial conditions specify the time and location where infection is first introduced. If this section is omitted, default values are used. If users want to add infections to the population at later times, or add or remove individuals from compartments separately from the model rules, they can do so via the related seeding section. More details here ;

`seeding` section

Optional section

This section is used to specify how individuals are instantaneously "seeded" from one compartment to another, where they then continue to be governed by the model equations. For example, this seeding could be used to represent importations of infected individuals from an outside population, mutation events that create new strains, or vaccinations that alter disease susceptibility. Seeding events can occur at any time in the simulation. The seeding section specifies the numeric values added to or removed from any compartment of the model. More details here ;

`outcomes` section

Optional section

This section is where users can define new variables representing the observed quantities and how they are related to the underlying state variables in the model (e.g., the fraction of infections that are detected as cases). More details here ;

`interventions` section

Required section

This section is where users can specify time-varying changes to parameters governing either the infectious disease model or the observational model. More details here ;

`inference` section

Optional section

This section is where users can specify the details of how the model is fit to data, including what data streams they will be included and which outcome variables they represent and the likelihood functions describing the probability of the data given the model. More details here. ;

Specifying population structure

This page describes how users specify the names, sizes, and connectivities of the different subpopulations comprising the total population to be modeled

Overview

The subpop_setup section of the configuration file is where users can input the information required to define a population structure on which to simulate the model. The options allow the user to determine the population size of each subpopulation that makes up the overall population, and to specify the amount of mixing that occurs between each pair of subpopulations.

An example configuration file with the global header and the spatial_setup section is below:

name: test_simulation
model_output_dirname: model_output
start_date: 2020-01-01
end_date: 2020-12-31
nslots: 100

subpop_setup:
  geodata: model_input/geodata.csv
  mobility: model_input/mobility.csv

Items and options

Config Item

Required?

Type/Format

Description

geodata

required

path to file

path to geodata file

mobility

required

path to file

path to mobility file

selected

optional

string or list of strings

name of selected location ingeodata

`geodata` file and `selected` option

geodata is a .csv with column headers, with at least two columns: subpop and population.
selected if provided, is the subset of locations in geodata file (as determined by subpop column) to be modeled. Requesting subpopulation(s) that are not present will lead to an error.

Example geodata file format

subpop,population
10001,1000
20002,2000

`mobility` file

The mobility file is a .csv file (it has to contain .csv as extension) with long form comma separated values. Columns have to be named ori, dest, amount, with amount being the average number individuals moving from the origin subpopulation ori to destination subpopulation dest on any given day. Details on the mathematics of this model of contact are explained in the Model Description section. Unassigned relations are assumed to be zero. The location entries in the ori and dest columns should correspond to an entry in the subpop column in geodata.csv. When using selected, the mobility data will also be filtered.

Example mobility file format

ori, dest, amount
10001, 20002, 3
20002, 10001, 3

It is also possible, but not recommended to specify the mobility file as a .txt with space-separated values in the shape of a matrix. This matrix is symmetric and of size K x K, with K being the number of rows in geodata. The above example corresponds to

0 3
3 0

Examples

Example 1

To simulate a simple population structure with two subpopulations, a large province with 10,000 individuals and a small province with only 1,000 individuals, where every day 100 residents of the large province travel to the small province and interact with residents there, and 50 residents of the small province visit the large province

subpop_setup:
  geodata: model_input/geodata.csv
  mobility: model_input/mobility.csv

geodata.csv contains the population structure (with columns subpop and population)

subpop,          population
large_province, 10000
small_province, 1000

mobility.csv contains

ori,            dest,           amount
large_province, small_province, 100
small_province, large_province, 50

Specifying compartmental model

This section describes how to specify the compartmental model of infectious disease transmission.

We want to allow users to work with a wide variety of infectious diseases or, one infectious disease under a wide variety of modeling assumptions. To facilitate this, we allow the user to specify their compartmental model of disease dynamics via the configuration file.

We originally considered asking users to specify each compartment and transition manually. However, we quickly found that this created long, confusing configuration files, and so we created a shorthand to more succinctly specify both compartments and transitions between them. This works especially well for models where individuals are stratified by other properties (like age, vaccination status, etc.) in addition to their infection status.

The model is specified in two separate sections of the configuration file. In the compartments section, users define the possible states individuals can be categorized into. Then in the seir section, users define the possible transitions between states, the values of parameters that govern the rates of these transitions, and the numerical method used to simulate the model.

An example section of a configuration file defining a simple SIR model is below.

compartments:
  infection_stage: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 0.1
    gamma: 0.2
  integration:
     method: rk4
     dt: 1.00

Specifying model compartments (`compartments`)

The first stage of specifying the model is to define the infection states (variables) that the model will track. These "compartments" are defined first in the compartments section of the config file, before describing the processes that lead to transitions between them. The compartments are defined separately from the rest of the model because they are also used by the seeding section that defines initial conditions and importations.

For simple disease models, the compartments can simply be listed with whatever notation the user chooses. For example, for a simple SIR model, the compartments could be ["S", "I", "R"]. The config also requires that there be a variable name for the property of the individual that these compartments describe, which for example in this case could be infection_stage

compartments:
  infection_stage: ["S", "I", "R"]

Our syntax allows for more complex models to be specified without much additional notation. For example, consider a model of a disease that followed SIR dynamics but for which individuals could receive vaccination, which might change how they experience infection.

In this case we can specify compartments as the cross product of multiple states of interest. For example:

 compartments:
   infection_stage: ["S", "I", "R"]
   vaccination_status: ["unvaccinated", "vaccinated"]

Corresponds to 6 compartments, which the code internally converts to this data frame

infection_stage, vaccination_status, compartment_name
S,               unvaccinated,       S_unvaccinated
I,               unvaccinated,       I_unvaccinated
R,               unvaccinated,       R_unvaccinated
S,               vaccinated,         S_vaccinated
I,               vaccinated,         I_vaccinated
R,               vaccinated,         R_vaccinated

In order to more easily describe transitions, we want to be able to refer to a compartment by its components, but then use it by its compartment name.

If the user wants to specify a model in which some compartments are repeated across states but others are not, there will be pros and cons of how the model is specified. Specifying it using the cross product notation is simpler, less error prone, and makes config files easier to read, and there is no issue with having compartments that have zero individuals in them throughout the model. However, for very large models, extra compartments increase the memory required to conduct the simulation, and so having unnecessary compartments tracked may not be desired.

For example, consider a model of a disease that follows SI dynamics in two separate age groups (children and adults), but for which only adults receive vaccination, with one or two doses of vaccine. With the simplified notation, this model could be specified as:

 compartments:
   infection_stage: ["S", "I"]
   age_group: ["child", "adult"]
   vaccination_status: ["unvaccinated", "1dose", "2dose"]

corresponding to 12 compartments, 4 of which are unnecessary to the model

infection_stage, age_group, vaccination_status, compartment_name
S,		 child,	    unvaccinated,	S_child_unvaccinated	
I,		 child,	    unvaccinated,	I_child_unvaccinated
S,		 adult,	    unvaccinated,	S_adult_unvaccinated
I,		 adult,	    unvaccinated,	I_adult_unvaccinated
S,		 child,	    1dose,		S_child_1dose
I,		 child,	    1dose,		I_child_1dose
S,		 adult,     1dose,		S_adult_1dose
I,		 adult,     1dose,		I_adult_1dose
S,		 child,     2dose,		S_child_2dose	
I,		 child,     2dose,		I_child_2dose
S,		 adult,	    2dose,		S_adult_2dose
I,		 adult,	    2dose,		I_adult_2dose

Or, it could be specified with the less concise notation

compartments:
   overall_state: ["S_child", "I_child", "S_adult_unvaccinated", "I_adult_unvaccinated", "S_adult_1dose", "I_adult_1dose", "S_adult_2dose", "I_adult_2dose"]

which does not result in any unnecessary compartments being included.

These compartments are referenced in multiple different subsequent sections of the config. In the seeding (LINK TBA) section the user can specify how the initial (or later imported) infections are distributed across compartments; in the seir section the user can specify the form and rate of the transitions between these compartments encoded by the model; in the outcomes section the user can specify how the observed variables are generated from the underlying model states.

Notation must be consistent between these sections.

Specifying compartmental model transitions (`seir::transitions`)

The way we specify transitions between compartments in the model is a bit more complicated than how the compartments themselves are specified, but allows users to specify complex stratified infectious disease models with minimal code. This makes checking, sharing, and updating models more efficient and less error-prone.

We specify one or more transition globs, each of which corresponds to one or more transitions. Since transition globs are shorthand for collections of transitions, we will first explain how to specify a single transition before discussing transition globs.

A transition has 5 pieces of associated information that a user can specify:

source
destination
rate
proportional_to
proportion_exponent

For more details on the mathematical forms possible for transitions in our models, read the Model Description section.

We first consider a simple example of an SI model where individuals may either be vaccinated (v) or unvaccinated (u), but the vaccine does not change the susceptibility to infection nor the infectiousness of infected individuals.

We will focus on describing the first transition of this model, the rate at which unvaccinated individuals move from the susceptible to infected state.

Specifying a single transition

Source

The compartment the transition moves individuals out of (e.g., the source compartment) is an array. For example, to describe a transition that moves unvaccinated susceptible individuals to another state, we would write

[S,unvaccinated]

which corresponds to the compartment S_unvaccinated

Destination

The compartment the transition moves individuals into (e.g. the destination compartment) is an array. For example, to describe a transition that moves individuals into the unvaccinated but infected state, we would write

[I,unvaccinated]

which corresponds to the compartment I_unvaccinated

Rate

The rate constant specifies the probability per time that an individual in the source compartment changes state and moves to the destination compartment. For example, to describe a transition that occurs with rate 5/time, we would write:

instead, we could describe the rate using a parameter beta, which can be given a numeric value later:

beta

The interpretation and unit of the rate constant depend on the model details, as the rate may potentially also be per number (or proportion) of individuals in other compartments (see below).

Proportional to

A vector of groups of compartments (each of which is an array) that modify the overall rate of transition between the source and destination compartment. Each separate group of compartments in the vector are first summed, and then all entries of the vector are multiplied to get the rate modifier. For example, to specify that the transition rate depends on the product of the number of unvaccinated susceptible individuals and the total infected individuals (vaccinated and unvaccinated), we would write:

[[[S,unvaccinated]], [[I,unvaccinated], [I, vaccinated]]]

To understand this term, consider the compartments written out as strings

[[S_unvaccinated], [I_unvaccinated, I_vaccinated]]

and then sum the terms in each group

[S_unvaccinated, I_unvaccinated + I_vaccinated]

From here, we can say that the transition we are describing is proportional to S_unvaccinated and I_unvaccinated + I_vaccinated, i.e., the rate depends on the product S_unvaccinated * (I_unvaccinated + I_vaccinated).

For transitions that occur at a constant per-capita rate (ie, E -> I at rate $\gamma$ in an SEIR model), it is possible to simply write proportional_to: ["source"].

Proportion exponent

This is an exponent modifying each group of compartments that contribute to the rate. It is equivalent to the "order" term in chemical kinetics. For example, if the reaction rate for the model above depends linearly on the number of unvaccinated susceptible individuals but on the total infected individuals sub-linearly, for example to a power 0.9, we would write:

[1, 0.9]

or a power parameter alpha, which can be given a numeric value later:

[1, alpha]

The (top level) length of the proportion_exponent vector must be the same as the (top level) length of the proportional_to vector, even if the desire of the user is to have the same exponent for all terms being multiplied together to get the rate.

Summary

Putting it all together, the model transition is specified as

source: [S, unvaccinated]
destination: [I, unvaccinated]
proportional_to: [[[S,unvaccinated]], [[I,unvaccinated], [I,vaccinated]]]
rate: [5]
proportion_exponent: [1, 0.9]

would correspond to the following model if expressed as an ordinary differential equation

\frac{\delta \text{S}_\text{unvaccinated}}{\delta t} = - \beta \text{S}_\text{unvaccinated}^1 (\text{I}_\text{unvaccinated}+\text{I}_\text{vaccinated})^{\alpha}

\frac{\delta \text{I}_\text{unvaccinated}}{\delta t} = \beta \text{S}_\text{unvaccinated}^1 (\text{I}_\text{unvaccinated}+\text{I}_\text{vaccinated})^{\alpha}

with parameter and parameter (we will describe how to use parameter symbols in the transitions and specify their numeric values separately in the section Specifying compartmental model parameters).

Transition globs

We now explain a shorthand we have developed for specifying multiple transitions that have similar forms all at once, via transition globs. The basic idea is that for each component of the single transitions described above where a term corresponded to a single model compartment, we can instead specify one or more compartment. Similarly, multiple rate values can be specified at once, for each involved compartment. From one transition glob, multiple individual transitions are created, by broadcasting across the specified compartments.

For transition globs, any time you could specify multiple arguments as a list, you may instead specify one argument as a non-list, which will be used for every broadcast. So [1,1,1] is equivalent to 1 if the dimension of that broadcast is 3.

We continue with the same SI model example, where individuals are stratified by vaccination status, but expand it to allow infection to occur at different rates in vaccinated and unvaccinated individuals:

Source

We allow one or more arguments to be specified for each compartment. So to specify the transitions out of both susceptible compartments (S_unvaccinated and S_unvaccinated), we would use

[[S], [unvaccinated,vaccinated]]

Destination

The destination variable should be the same shape as the source, and in the same relative order. So to specify a transition from S_unvaccinated to I_unvaccinated and S_vaccinated to I_vaccinated, we would write the destination as:

[[I], [unvaccinated,vaccinated]]

If instead we wrote:

[[I], [vaccinated,unvaccinated]]

we would have a transition from S_unvaccinated to I_vaccinated and S_vaccinated to I_unvaccinated.

Rate

The rate vector allows users to specify the rate constant for all the source -> destination transitions that are defined in a shorthand way, by instead specifying how the rate is altered depending on the compartment type. For example, the rate of transmission between a susceptible (S) and an infected (I) individual may vary depending on whether the susceptible individual is vaccinated or not and whether the infected individual is vaccinated or not. The overall rate constant is constructed by multiplying together or "broadcasting" all the compartment type-specific terms that are relevant to a given compartment.

For example,

rate: [[3], [0.6,0.5]]

This would mean our transition from S_unvaccinated to I_unvaccinated would have a rate of 3 * 0.6 while our transition from S_vaccinated to I_vaccinated would have a rate of 3 * 0.5.

The rate vector should be the same shape as source and destination and in the same relative order.

Note that if the desire is to make a model where the difference in the rate constants varies in a more complicated than multiplicative way between different compartment types, it would be better to specify separate transitions for each compartment type instead of using this shorthand.

Proportional to

The broadcasting here is a bit more complicated. In other cases, each broadcast is over a single component. However, in this case, we have a broadcast over a group of components. We allow a different group to be chosen for each broadcast.

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I,unvaccinated],[I, vaccinated]]
]

Again, let's unpack what it says. Since the broadcast is over groups, let's split the config back up

into those groups

[
  [S,unvaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]
[
  [S,vaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]

From here, we can say that we are describing two transitions. Both occur proportionally to the same compartments: S_unvaccinated and the total number of infections (I_unvaccinated+I_vaccinated).

If, for example, we want to model a situation where vaccinated susceptibles cannot be infected by unvaccinated individuals, we would instead write:

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
]

Proportion exponent

Similarly to rate and proportional_to, we provide an exponent for each component and every group across the broadcast. So we could for example use:

[[1,1], [0.9,0.8]]

Summary

Putting it all together, the transition glob

seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[3], [0.6,0.5]]
    proportion_exponent: [[1,1], [0.9,0.8]]

is equivalent to the following transitions

seir:
  transitions:
    - source: [S,unvaccinated]
      destination: [I,unvaccinated]
      proportional_to: [[[S,unvaccinated]], [[I,unvaccinated],[I, vaccinated]]]
      proportion_exponent: [1 * 0.9]
      rate: [3*0.6]
    - source: [S,vaccinated]
      destination: [I,vaccinated]
      proportional_to: [[[S,vaccinated]], [[I, vaccinated]]]
      proportion_exponent: [1 * 0.8]
      rate: [3*0.5]

Warning

We warn the user that with this shorthand, it is possible to specify large models with few lines of code in the configuration file. The more compartments and transitions you specify, the longer the model will take to run, and the more memory it will require.

Specifying compartmental model parameters (`seir::parameters`)

When the transitions of the compartmental model are specified as described above, they can either be entered as numeric values (e.g., 0.1) or as strings which can be assigned numeric values later (e.g., beta). We recommend the latter method for all but the simplest models, since parameters may recur in multiple transitions and so that parameter values may be edited without risk of editing the model structure itself. It also improves readability of the configuration files.

Parameters can take on three types of values:

Fixed values
Value drawn from distributions
Values read from timeseries specified in a data file

Specifying fixed parameter values

Parameters can be assigned values by using the value argument after their name and then simply stating their numeric argument. For example, in a config describing a simple SIR model with transmission rate $\beta$ (beta) = 0.1/day and recovery rate $\gamma$ (gamma) = 0.2/day. This could be specified as

seir:
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

The full model section of the config could then read

compartments:
  infection_state: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: [1,1]
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

For the stratified SI model described above, this portion of the config would read

compartments:
  infection_stage: ["S", "I", "R"]
  vaccination_status: ["unvaccinated", "vaccinated"]
  
seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[beta], [theta_u,theta_v]]
    proportion_exponent: [[1,1], [alpha_u,alpha_v]]
  parameters:
    beta: 
      value: 0.1
    theta_u: 
      value: 0.6
    theta_v: 
      value: 0.5
    alpha_u: 
      value: 0.9
    alpha_v: 
      value: 0.8

If there are no parameter values that need to be specified (all rates given numeric values when defining model transitions), the seir::parameters section of the config can be left blank or omitted.

Specifying parameters values from distributions

Parameter values can also be specified as random values drawn from a distribution, as a way of including uncertainty in parameters in the model output. In this case, every time the model is run independently, a new random value of the parameter is drawn. For example, to choose the same value of beta = 0.1 each time the model is run but to choose a random values of gamma with mean on a log scale of $e^{-1.6} = 0.2$ and standard deviation on a log scale of $e^{0.2} = 1.2$ (e.g., 1.2-fold variation):

seir:
  parameters:
    beta: 
      value:
        distribution: fixed
        value: 0.1
    gamma: 
      value:
        distribution: lognorm
        logmean: -1.6
        logsd: 0.2

Details on the possible distributions that are currently available, and how to specify their parameters, is provided in the Distributions section.

Note that understanding when a new parameter values from this distribution is drawn becomes more complicated when the model is run in Inference mode. In Inference mode, we distinguish model runs as occurring in different "slots" – i.e., completely independent model instances that could be run on different processing cores in a parallel computing environment – and different "iterations" of the model that occur sequentially when the model is being fit to data and update fitted parameters each time based on the fit quality found in the previous iteration. A new parameter values is only drawn from the above distribution once per slot. Within a slot, at each iteration during an inference run, the parameter is only changed if it is being fit and the inference algorithm decides to perturb it to test a possible improved fit. Otherwise, it would maintain the same value no matter how many times the model was run within a slot.

Specifying parameter values as timeseries from data files

Sometimes, we want to be able to specify model parameters that have different values at different timepoints. For example, the relative transmissibility may vary throughout the year based on the weather conditions, or the rate at which individuals are vaccinated may vary as vaccine programs are rolled out. One way to do this is to instead specify the parameter values as a timeseries.

This can be done by providing a data file in .csv or .parquet format that has a list of values of the parameter for a corresponding timepoint and subpopulation name. One column should be date, which should have an entry for every calendar day of the simulation, with the first and last date corresponding to the start_date and end_date for the simulation specified in the header of the config. There should be another column for each subpopulation, where the column name is the subpop name used in other files and the values are the desired parameter values for that subpopulation for the corresponding day. If any day or subpopulation is missing, an error will occur. However, if you want all subpopulations to have the same parameter value for every day, then only a single column in addition to date is needed, which can have any name, and will be applied to every subpop ;

For example, for an SIR model with a simple two-province population structure where the relative transmissibility peaks on January 1 then decreases linearly to a minimal value on June 1 then increases linearly again, but varies more in the small province than the large province, the theta parameter could be constructed from the file seasonal_transmission_2pop.csv with contents including

date,        small_province,    large_province
2022-01-01,  1.5,               1.3
.....
2022-05-01,  0.5,               0.7 
....
2022-12-31,  1.5,               1.3

as a part of a configuration file with the model sections:

compartments:
  infection_stage: ["S", "I", "R"]

seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta*theta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2
    theta:
       timeseries: data/seasonal_transmission.csv

Note that there is an alternative way to specify time dependence in parameter values that is described in the Specifying time-varying parameter modifications section. That method allows the user to define intervention parameters that apply specific additive or multiplicative shifts to other parameter values for a defined time interval. Interventions are useful if the parameter doesn't vary frequently and if the values of the shift is unknown and it is desired to either sample over uncertainty in it or try to estimate its value by fitting the model to data. If the parameter varies frequently and its value or relative value over time is known, specifying it as a timeseries is more efficient.

Compartmental model parameters can have an additional attribute beyond value or timeseries, which is called stacked_modifier_method. This value is explained in the section on coding time-dependent parameter modifications (also known as "modifiers") as it determines what happens when two different modifiers act on the same parameter at the same time (are they combined additively or multiplicatively?) ;

Config item

Required?

Type/Format

Description

value

either value or timeseries is required

numerical, or distribution

This defines the value of the parameter, as described above.

timeseries

either value or timeseries is required

path to a csv file

This defines a timeseries for each day, as above.

stacked_modifier_method

optional

string: sum, product, reduction_product

This option defines the method used when modifiers are applied. The default is product.

rolling_mean_windows

optional

integer

The size of the rolling mean window if a rolling mean is applied.

Specifying model simulation method `(seir::integration)`

A compartmental model defined using the notation in the previous sections describes rules for classifying individuals in the population based on infection state dynamically, but does not uniquely specify the mathematical framework that should be used to simulate the model.

Our framework allows for two major methods for implementing compartmental models of disease transmission:

ordinary differential equations, which are completely deterministic, operate in continuous time (consider infinitesimally small timesteps), and allow for arbitrary fractions of the population (i.e., not just discrete individuals) to move between model compartments
discrete-time stochastic process, which tracks discrete individuals and produces random variation in the number of individuals transitioning between states for any given rate, and which allows transitions between states only to occur at discrete time intervals

The mathematics behind each implementation is described in the Model Description section

Config item

Required?

Type/format

Description

method

optional

string: rk4 (default),euler, stochastic

The algorithm used to simulate the model equations. If rk4, model is simulated deterministically by numerical integration using a 4th order Runge-Kutta algorithm. If euler or stochastic, uses a discrete-time process, with steps proceeding either deterministically (at the average rate) or stochastically. For both of these cases, the algorithm ensures no compartment goes below zero for the requested time step. The -(-m)ethod option can be used (see ) to override this configuration option.

dt

optional

positive real number (default: 2)

The timestep used for the numerical integration or discrete time stochastic update; for rk4 method, this is a reasonable value, but for other options, this should be 0.2 or less.

For example, to simulate a model deterministically using the 4th order Runge-Kutta algorithm for numerical integration with a timestep of 1 day:

seir:
  integration:
     method: rk4
     dt: 1.00

Alternatively, to simulate a model stochastically with a timestep of 0.1 days

seir:
  integration:
     method: stochastic
     dt: 0.1

For any method, the results of the model will be more accurate when the timestep is smaller (i.e., output will more precisely match the mathematics of the model description and be invariant to the choice of timestep). However, the computing time required to simulate the model for a certain time range of interest increases with the number of timesteps required (i.e., with smaller timesteps). In our experience, the 4th order Runge-Kutta algorithm (for details see Advanced section) is a very accurate method of numerically integrating such models and can handle timesteps as large as roughly a day for models with the maximum per capita transition rates in this same order of magnitude. However, both of the discrete time engines require smaller timesteps to be accurate (around 0.1 for COVID-19-like dynamics in our experience).

Specifying initial conditions

This section describes how to specify the values of each model state at the time the simulation starts, and how to make instantaneous changes to state values at other times (e.g., due to importations)

Overview

In order for the models specified previously to be dynamically simulated, the user must provide initial conditions, in addition to the model structure and parameter values. Initial conditions describe the value of each variable in the model at the time point that the simulation is to start. For example, on day zero of an outbreak, we may assume that the entire population is susceptible except for one single infected individual. Alternatively, we could assume that some portion of the population already has prior immunity due to vaccination or previous infection. Different initial conditions lead to different model trajectories.

The initial_conditions section of the configuration file is detailed below. Note that in some cases, can replace or complement the initial condition, the table below provides a quick comparison of these sections.

Feature

initial_conditions

seeding

Specifying model initial conditions

The configuration items in the initial_conditions section of the config file are

initial_conditions:method Must be either "Default", "SetInitialConditions", or "FromFile".

initial_conditions:initial_conditions_fileRequired for methods “SetInitialConditions” and “FromFile” . Path to a .csv or .parquet file containing the list of initial conditions for each compartment.

initial_conditions:initial_file_type Only required for method: “FolderDraw”. Description TBA

initial_conditions::allow_missing_subpops Optional for all methods, determines what will happen if initial_conditions_file is missing values for some subpopulations. If FALSE, the default behavior, or unspecified, an error will occur if subpopulations are missing. If TRUE, then for subpopulations missing from the initial_conditions file, it will be assumed that all individuals begin in the first compartment (the “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group), unless another compartment is designated to hold the rest of the individuals ;

initial_conditions::allow_missing_compartments Optional for all methods. If FALSE, the default behavior, or unspecified, an error will occur if any compartments are missing for any subpopulation. If TRUE, then it will be assumed there are zero individuals in compartments missing from the initial_conditions file.

initial_conditions::proportional If TRUE, assume that the user has specified all input initial conditions as fractions of the population, instead of numbers of individuals (the default behavior, or if set to FALSE). Code will check that initial values in all compartments sum to 1.0 and throw an error if not, and then will multiply all values by the total population size for that subpopulation ;

Details on implementing each initial conditions method and the options that go along with it are below.

`initial_conditions::method`

Default

The default initial conditions are that the initial value of all compartments for each subpopulation will be zero, except for the first compartment, whose value will be the population size. The “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group.

For example, a model with the following compartments

with the accompanying geodata file

will be started with 1000 individuals in the S_child_unvaxxed in the "small province" and 10,000 in that compartment in the "large province".

SetInitialConditions

With this method users can specify arbitrary initial conditions in a convenient formatted input .csv or .parquet file.

For example, for a model with the following compartments and initial_conditions sections

with the accompanying geodata file

where initial_conditions.csv contains

the model will be started with half of the population of both subpopulations, consisting of children and the other half of adults, everyone unvaccinated, and 5 infections (in exposed-but-not-yet-infectious class) among the unvaccinated adults in the large province, with the remaining individuals susceptible (4995). All other compartments will contain zero individuals initially ;

initial_conditions::initial_conditions_file must contain the following columns:

subpop – the name of the subpopulation for which the initial condition is being specified. By default, all subpopulations must be listed in this file, unless the allow_missing_subpops option is set to TRUE.
mc_name – the concatenated name of the compartment for which an initial condition is being specified. The order of the compartment groups in the name must be the same as the order in which these groups are defined in the config for the model, e.g., you cannot say unvaccinated_S.
amount – the value of the initial condition; either a numeric value or the string "rest".

For each subpopulation, if there are compartments that are not listed in SetInitialConditions, an error will be thrown unless allow_missing_compartments is set to TRUE, in which case it will be assumed there are zero individuals in them. If the sum of the values of the initial conditions in all compartments in a location does not add up to the total population of that location (specified in the geodata file), an error will be thrown. To allocate all remaining individuals in a subpopulation (the difference between the total population size and those allocated by defined initial conditions) to a single pre-specified compartment, include this compartment in the initial_conditions_file but instead of a number in the amount column, put the word "rest" ;

If allow_missing_subpops is FALSE or unspecified, an error will occur if initial conditions for some subpopulations are missing. If TRUE, then for subpopulations missing from the initial_conditions file, it will be assumed that all individuals begin in the first compartment. (The “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group.)

FromFile

Similar to "SetInitialConditions", with this method users can specify arbitrary initial conditions in a formatted .csv or .parquet input file. However, the format of the input file is different. The required file format is consistent with the from the compartmental model, so the user could take output from one simulation and use it as input into another simulation with the same model structure ;

For example, for an input configuration file containing

with the accompanying geodata file

where initial_conditions_from_previous.csv contains

The simulation would be initiated on 2021-06-01 with these values in each compartment (no children vaccinated, only adults in the small province vaccinated, some past and current infection in both compartments but ).

initial_conditions::initial_conditions_file must contain the following columns:

mc_value_type – in model output files, this is either prevalence or incidence. Prevalence values only are selected to be used as initial conditions, since compartmental models described the prevalence (number of individuals at any given time) in each compartment. Prevalence is taken to be the value measured instantaneously at the start of the day
mc_name – The name of the compartment for which the value is reported, which is a concatenation of the compartment status in each state type, e.g. "S_adult_unvaxxed" and must be in the same order as these groups are defined in the config for the model, e.g., you cannot say unvaxxed_S_adult.
subpop_1, subpop_2, etc. – one column for each different subpopulation, containing the value of the number of individuals in the described compartment in that subpopulation at the given date. Note that these are named after the nodenames defined by the user in the geodata file.
date – The calendar date in the simulation, in YYYY-MM-DD format. Only values with a date that matches to the simulation start_date will be used ;

SetInitialConditionsFolderDraw, FromFileFolderDraw

The way that initial conditions is specified with SetInitialConditions and FromFile results in a single value for each compartment and does not easily allow the user to instead specify a distribution (like is possible for compartmental or outcome model parameters). If a user wants to use different possible initial condition values each time the model is run, the way to do this is to instead specify a folder containing a set of file with initial condition values for each simulation that will be run. The user can do this using files with the format described in initial_conditions::method::SetInitialConditions using instead method::SetInitialConditionsFolder draw. Similarly, to provide a folder of initial condition files with the format described in initial_conditions::method:FromFile using instead method::FromFileFolderDraw ;

Each file in the folder needs to be named according to the same naming conventions as the model output files: run_number.runID.file_type.[csv or parquet] where ....[DESCRIBE] as it is now taking the place of the seeding files the model would normally outpu ;

Only one additional config argument is needed to use a FolderDraw method for initial conditions:

initial_file_type: either seir or seed

When using FolderDraw methods, initial_conditions_file should now be the path to the directory that contains the folder with all the initial conditions files. For example, if you are using output from another model run and so the files are in an seir folder within a model_output folder which is in within your project directory, you would use initial_conditions_file: model_outpu ;

Specifying observational model

This page describes how to specify the outcomes section of the configuration file

Thinking about `outcomes` variables

Our pipeline allows users to encode state variables describing the infection status of individuals in the population in two different ways. The first way is via the state variables and transitions of the compartmental model of disease transmission, which are specified in the and sections of the config. This model should include all variables that influence the natural course of the epidemic (i.e., all variables that feed back into the model by influencing the rate of change of other variables). For example, the number of infected individuals influences the rate at which new infections occur, and the number of immune individuals influences the number of individuals at risk of acquiring infection.

However, these intrinsic model variables may be difficult to observe in the real world and so directly comparing model predictions about the values of these variables to data might not make sense. Instead, the observable outcomes of infection may include only a subset of individuals in any state, and may only be observed with a time delay. Thus, we allow users to define new outcome variables that are functions of the underlying model variables. Commonly used examples include detected cases or hospitalizations ;

Variables should not be included as outcomes if they influence the infection trajectory. The choice of what variables to include in the compartmental disease model vs. the outcomes section may be very model specific. For example, hospitalizations due to infection could be encoded as an outcome variable that is some fraction of infections, but if we believe hospitalized individuals are isolated from the population and don't contribute to onward infection, or that the number of hospitalizations feeds back into the population's perception of risk of infection and influences everyone's contact behavior, this would not be the best choice. Similarly, we could include deaths due to infection as an outcome variable that is also some fraction of infections, but unless death is a very rare outcome of infection and we aren't worried about actually removing deceased individuals from the modeled populations, deaths should be in the compartmental model instead.

The outcomes section is not required in the config. However, there are benefits to including it, even if the only outcome variable is set to be equivalent to one of the infection model variables. If the compartmental model is complicated but you only want to visualize a few output variables, the will be much easier to work with. Outcome variables always occur with some fixed delay from their source infection model variable, which can be more convenient than the exponential distribution underlying the infection model. Outcome variables can be created to automatically sum over multiple compartments of the infection model, removing the need for post-processing code to do this. If the model is being fit to data, then the outcomes section is required, as only outcome variables can be compared to data.

As an example, imagine we are simulating an SIR-style model and want to compare it to real epidemic data in which cases of infection and death from infection are reported. Our model doesn't explicitly include death, but suppose we know that 1% of all infections eventually lead to hospitalization, and that hospitalization occurs on average 1 week after infection. We know that not all infections are reported as cases, and assume that only 50% are detected and are reported 2 days after infection begins. The model and outcomes section of the config for these outcomes, which we call incidC (daily incidence of cases) and incidH (daily incidence of hospital admission) would be

in the following sections we describe in more detail how this specification works

Specifying `outcomes` in the configuration file

The outcomes config section consists of a list of defined outcome variables (observables), which are defined by a user-created name (e.g., "incidH"). For each of these outcome variables, the user defines the source compartment(s) in the infectious disease model that they draw from and whether they draw from the incidence (new individuals entering into that compartment) or prevalence (total current individuals in that compartment). Each new outcome variable is always associated with two mandatory parameters ;

probability of being counted in this outcome variable if in the source compartment
;delay between when an individual enters the source compartment and when they are counted in the outcome variable

and one optional parameter

duration after entering that an individual is counted as part of the outcome variable

The value of the probability, delay, and duration parameters can be a single value or come from ;

Outcome model parameters probability, delay, and distribution can have an additional attribute beyond value called modifier_key. This value is explained in the section on coding (also known as "modifiers") as it provides a way to have the same modifier act on multiple different outcomes ;

Just like the case for , when outcome parameters are drawn from a distribution, each time the model is run, a different value for this parameter will be drawn from this distribution, but that value will be used for all calculations within this model run. Note that understanding when a new parameter values from this distribution is drawn becomes more complicated when the model is run in mode. In Inference mode, we distinguish model runs as occurring in different "slots" – i.e., completely independent model instances that could be run on different processing cores in a parallel computing environment – and different "iterations" of the model that occur sequentially when the model is being fit to data and update fitted parameters each time based on the fit quality found in the previous iteration. A new parameter values is only drawn from the above distribution once per slot. Within a slot, at each iteration during an inference run, the parameter is only changed if it is being fit and the inference algorithm decides to perturb it to test a possible improved fit. Otherwise, it would maintain the same value no matter how many times the model was run within a slot.

Example

Config item

Required?

Type/format

Description

`source ;

Required, unless option is used instead. This sub-section describes the compartment(s) in the infectious disease model from which this outcome variable is drawn. Outcome variables can be drawn from the incidence of a variable - meaning that some fraction of new individuals entering the infection model state each day are chosen to contribute to the outcome variable - or from the prevalence, meaning that each day some fraction of individuals currently in the infection state are chosen to contribute to the outcome variable. Note that whatever the source type, the named outcome variable itself is always a measure of incidence ;

To specify which compartment(s) contribute the user must specify the state(s) within each model stratification. For stratifications not mentioned, the outcome will sum over that states in all strata ;

For example, consider a configuration in which the compartmental model was constructed to track infection status stratified by vaccination status and age group. The following code would be used to create an outcome called incidH_child (incidence of hospitalization for children) and incidH_adult (incidence of hospitalization for adults) where some fraction of infected individuals would become hospitalized and we wanted to track separately track pediatric vs adult hospitalizations, but did not care about tracking the vaccination status of hospitalized individuals as in reality it was not tracked by the hospitals ;

to instead create an outcome variable for cases where on each day of infection there is some probability of testing positive (for example, for the situation of an asymptomatic infection where testing is administered totally randomly), the following code would be used

The source of an outcome variable can also be a previous defined outcome variable. For example, t to create a new variable for the number of individuals recruited to be part of a contact tracing program (incidT), which is just some fraction of diagnosed cases ;

`probability ;

Required, unless option is used instead. Probability is the fraction of individuals in the source compartment who are counted as part of this outcome variable (if the source is incidence; if the source is prevalence it is the fraction of individuals per day). It must be between 0 and 1 ;

Specifying the probability creates a parameter called outcome_name::probability that can be referred to in the section of the config. The value of this parameter can be changed using the probability::intervention_param_name option ;

For example, to track the incidence of hospitalization when 5% of children but only 1% of adults infected require hospitalization, and to create a modifier_key such that both of these rates could be modified by the same amount during some time period using the section:

To track the incidence of diagnosed cases iterating over uncertainty in the case detection rate (ranging 20% to 30%), and naming this parameter "case_detect_rate"

Each time the model is run a new random value for the probability of case detection will be chosen ;

Delay

Required, unless option is used instead. delay is the time delay between when individuals are chosen from the source compartment and when they are counted as part of this outcome variable ;

For example, to track the incidence of hospitalization when 5% of children are hospitalized and hospitalization occurs 7 days after infection:

To iterate over uncertainty in the exact delay time, we could include some variation between simulations in the delay time using a normal distribution with standard deviation of 2 (truncating to make sure the delay does not become negative). Note that a delay distribution here does not mean that the delay time varies between individuals - it is identical) ;

Duration

By default, all outcome variables describe incidence (new individuals entering each day). However, they can also track an associated "prevalence" if the user specifies how long individuals will stay classified as the outcome state the outcome variable describes. This is the duration parameter ;

When the duration parameter is set, a new outcome variable is automatically created and named with the name of the original outcome variable + "_curr". This name can be changed using the duration::name option ;

For example, to track the incidence and prevalence of hospitalization when 5% of children are hospitalized, hospitalization occurs 7 days after infection, and the duration of hospitalization is 3 days:

which creates the variable "incidH_child_curr" to track all currently hospitalized children. Since it doesn't make sense to call this new outcome variable an incidence, as it is a prevalence, we could instead rename it:

Sum

Optional. sum is used to create new outcome variables that are sums over other previously defined outcome variables ;

If sum is included, source, probability, delay, and duration will be ignored ;

For example, to track new hospital admissions and current hospitalizations separately for children and adults, as well as for all ages combined

outcomes::settings

There are other required and optional configuration items for the outcomes section which can be specified under outcomes::settings:

method: delayframe.This is the mathematical method used to create the outcomes variable values from the transmission model variables. Currently, the only model supported is delayframe, which .. ;

param_from_file: Optional, TRUE or FALSE. It is possible to allow any of the outcomes variables to have values that vary across the subpopulations. For example, disease severity rates or diagnosis rates may differ by demographic group. In this case, all the outcome parameter values defined in will represent baseline values, and then you can define a relative change from this baseline for any particular subpopulation using the paths section. If params_from_file: TRUE is specified, then these relative values will be read from the params_subpop_file. Otherwise, if params_from_file: FALSE or is not listed at all, all subpopulations will have the same values for the outcome parameters, defined below ;

param_subpop_file: Required if params_from_file: TRUE. The path to a .csv or .parquet file that contains the relative amount by which a given outcome variable is shifted relative to baseline in each subpopulation. File must contain the following columns:

subpop: The subpopulation for which the parameter change applies. Must be a subpopulation defined in the file. For example, small_province
parameter: The outcomes parameter which will be altered for this subpopulation. For example, incidH_child: probability
value: The amount by which the baseline value will be multiplied, for example, 0.75 or 1.1

Examples

Consider a disease described by an SIR model in a population that is divided into two age groups, adults and children, which experience the disease separately. We are interested in comparing the predictions of the model to real world data, but we know we cannot observe every infected individual. Instead, we have two types of outcomes that are observed.

First, via syndromic surveillance, we have a database that records how many individuals in the population are experiencing symptoms from the disease at any given time. Suppose careful cohort studies have shown that 50% of infected adults and 80% of infected children will develop symptoms, and that symptoms occur in both age groups around 3 days after infection (following a log-normal distribution with log mean X and log standard deviation of Y). The duration that symptoms persist is also a variable, following a ...

Secondly, via laboratory surveillance we have a database of every positive test result for the infection. We assume the test is 100% sensitive and specific. Only individuals with symptoms are tested, and they are always tested exactly 1 day after their symptom onset. We are unsure what portion of symptomatic individuals are seeking out testing, but are interested in considering two extreme scenarios: 95% of symptomatic individuals are tested, or only 75% of individuals are tested.

The configuration file we could use to model this situation includes

Distributions

This page describes the configuration schema for specifying distributions

Distribution

Parameters

Type/Format

Description

fixed

value

Any real number

Draws all values exactly equal to value

uniform

low

Any real number

Draws all values randomly from a uniform distribution with range [low, high]

high

Any real number greater than low

poisson

lam

Any positive real number

Draws all values randomly from a Poisson distribution with rate parameter (mean) lam (lambda)

binomial

size

Any non-negative integer

Draws all values randomly from a binomial distribution with number of trials (n) = size and probability of success on each trial (p) = prob

prob

Any number in [0,1]

lognormal

meanlog

Any real number

Draws all values randomly from a lognormal distribution (natural log, base e) with mean on a log scale of meanlog and standard deviation on a log scale of sdlog

sdlog

Any non-negative real number

truncnorm

mean

Any real number

Draws all values randomly from a truncated normal distribution with mean mean and standard deviation sd, truncated to have a maximum value of a and a minimum value of b

sd

Any non-negative real number

a

Any real number, or -Inf

b

Any real number greater than a, or Inf

Specifying time-varying parameter modifications

This section describes how to specify modifications to any of the parameters of the transmission model or observational model during certain time periods.

Modifiers are a powerful feature in flepiMoP to enable users to modify any of the parameters being specified in the model during particular time periods. They can be used, for example, to mirror public health control interventions, like non-pharmaceutical interventions (NPIs) or increased access to diagnosis or care, or annual seasonal variations in disease parameters. Modifiers can act on any of the transmission model parameters or observation model parameters ;

In the seir_modifiers and outcome_modifiers sections of the configuration file the user can specify several possible types of modifiers which will then be implemented in the model. Each modifier changes a parameter during one or multiple time periods and for one or multiple specified subpopulations.

We currently support the following intervention types. Each of these is described in detail below:

"SinglePeriodModifier" – Modifies a parameter during a single time period
"MultiPeriodModifier" – Modifies a parameter by the same amount during a multiple time periods
"ModifierModifier" – Modifies another intervention during a single time period
"StackedModifier" – Combines two or more interventions additively or multiplicatively, and is used to be able to turn on and off groups of interventions easily for different runs ;

Note that if you want a parameter to vary continuously over time (for example, a daily transmission rate that is influenced by temperature and humidity), then it is easier to do this by using a "timeseries" parameter value than by combining many separate modifiers. Timeseries parameter values are described in the seir::parameters section. Timeseries parameters foroutcomes parameters (e.g., a testing rate that fluctuates rapidly due to test availability) are in development but not currently available ;

Within flepiMoP, modifiers can be run as "scenarios". With scenarios, we can use the same configuration file to run multiple versions of the model where only the modifiers applied differ.

The modifiers section contains two sub-sections: modifiers::scenarios, which lists the name of the modifiers that will run in each separate scenario, and modifiers::modifiers, where the details of each modifier are specified (e.g., the parameter it acts on, the time it is active, and the subpopulation it is applied to). An example is outlined below

seir_modifiers:
  scenarios:
    -NameOfIntervention1
    -NameofIntervention2
  modifiers:
    NameOfIntervention1:
      ...
    NameOfIntervention2:
      ...

In this example, each scenario runs a single intervention, but more complicated examples are possible. ;

The major benefit of specifying both "scenarios" and "modifiers" is that the user can use "StackedModifier" option to combine other modifiers in different ways, and then run either the individual or combined modifiers as scenarios. This way, each scenario may consist of one or more individual parameter modifications, and each modification may be part of multiple scenarios. This provides a shorthand to quickly consider multiple different versions of a model that have different combinations of parameter modifications occurring. For example, during an outbreak we could evaluate the impact of school closures, case isolation, and masking, or any one or two of these three measures. An example of a configuration file combining modifiers to create new scenarios is given below

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs
  modifiers:
    SchoolClosures:
      method:SinglePeriodModifier
      ...
    CaseIsolation:
      method:SinglePeriodModifier
      ...
    Masking:
      method:SinglePeriodModifier
      ....
    AllNPIs
      method: StackedModifier
      modifiers: ["SchoolClosures","CaseIsolation","Masking"]

The seir_modifiers::scenarios andoutcome_modifiers::scenarios sections are optional. If the scenariossection is not included, the model will run with all of the modifiers turned "on" ;

If thescenariossection is included for either seir or outcomes, then each time a configuration file is run, the user much specify which modifier scenarios will be run. If not specified, the model will be run one time for each combination of seir and outcome scenario ;

Example

[Give a configuration file that tries to use all the possible option available. Based on simple SIR model with parameters beta and gamma in 2 subpopulations. Maybe a SinglePeriodModifier on beta for a lockdown and gamma for isolation, one having a fixed value and one from a distribution, MultiPeriodModifier for school year in different places, ModifierModifer for ..., StackedModifier for .... ]

`modifiers::scenarios`

A optional list consisting of a subset of the modifiers that are described in modifiers::settings, each of which will be run as a separate scenario. For example

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs

outcome_modifiers
  scenarios:
    -BaselineTesting
    -TestShortage

`modifiers::settings`

A formatted list consisting of the description of each modifier, including its name, the parameter it acts on, the duration and amount of the change to that parameter, and the subset of subpopulations in which the parameter modification takes place. The list items are summarized in the table below and detailed in the sections below.

Config item

Required

Type/format

Description

method

required

string

one of SinglePeriodModifier, MultiPeriodModifier, ModifierModifier, or StackedModifier

parameter

required

string

The parameter on which the modification is acting. Must be a parameter defined in seir::parameters or outcomes

period_start_date or periods::start_date

required

numeric, YYYY-MM-DD

The date when the modification starts. Notation depends on value of method.

period_end_date or periods::end_date

required

numeric, YYYY-MM-DD

The date when the modification ends. Notation depends on value of method.

subpop

required

String, or list of strings

The subpopulations to which the modifications will be applied, or "all" . Subpopulations must appear in the geodata file.

value

required

Distribution, or single value

The relative amount by which a modification reduces the value of a parameter.

subpop_groups

optional

string or a list of lists of strings

A list of lists defining groupings of subpopulations, which defines how modification values should be shared between them, or 'all' in which case all subpopulations are put into one group with identical modification values. By default, if parameters are chosen randomly from a distribution or fit based on data, they can have unique values in each subpopulation.

baseline_scenario

Used only for ModifierModifier

String

Name of the original modification which will be further modified

modifiers

Used only for StackedModifier

List of strings

List of modifier names to be grouped into the new combined modifier/scenario name

SinglePeriodModifier

SinglePeriodModifier interventions enable the user to specify a multiplicative reduction to a parameter of interest. It take a parameter, and reduces it's value by value (new = (1-value) * old) for the subpopulations listed insubpop during the time interval [period_start_date, period_end_date]

For example, if you would like to create an SEIR modifier called lockdown that reduces transmission by 70% in the state of California and the District of Columbia between two dates, you could specify this with a SinglePeriodModifier, as in the example below

Example

seir_modifiers:
  modifiers:
    lockdown: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: ['06000', '11000']
      value: 0.7

Or to create an outcome variable modifier called enhanced_testing during which the case detection rate double ;

outcome_modifiers:
  modifiers:
    enhanced_testing: 
      method: SinglePeriodModifier
      parameter: incidC::probability
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: ['06000', '11000']
      value: -1.0

Configuration options

method: SinglePeriodModifier

parameter: The name of the parameter that will be modified. This could be a parameter defined for the transmission model in seir::parameters or for the observational model in outcomes. If the parameter is used in multiple transitions in the model then all those transitions will be modified by this amount ;

period_start_date: The date when the modification starts, in YYYY-MM-DD format. The modification will only reduce the value of the parameter after (inclusive of) this date.

period_end_date: The date when the modification ends, in YYYY-MM-DD format. The modification will only reduce the value of the parameter before (inclusive of) this date.

subpop:A list of subpopulation names/ids in which the specified modification will be applied. This can be a single subpop, a list, or the word "all" (specifying the modification applies to all existing subpopulations in the model). The modification will do nothing for any subpopulations not listed here.

value:The fractional reduction of the parameter during the time period the modification is active. This can be a scalar number, or a distribution using the notation described in the Distributions section. The new parameter value will be

new_parameter_value = old_parameter_value * (1 - value)

subpop_groups: An optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. See subpop_groups section below for more details ;

MultiPeriodModifier

MultiPeriodModifier interventions enable the user to specify a multiplicative reduction to the parameter of interest by value (new = (1-value) * old) for the subpopulations listed in subpop during multiple different time intervals each defined by a start_date and end_date.

For example, if you would like to describe the impact that transmission in schools has on overall disease spread, you could create a modifier that increases transmission by 30% during the dates that K-12 schools are in session in different regions (e.g., Massachusetts and Florida):

Example

school_year:
  method: MultiPeriodModifier
  parameter: beta
  groups:
    - subpop: ["25000"] 
      periods:
        - start_date: 2021-09-09
          end_date: 2021-12-23
        - start_date: 2022-01-04
          end_date: 2022-06-22
    - subpop: ["12000"]
      periods:
        - start_date: 2021-08-10
          end_date: 2021-12-17
        - start_date: 2022-01-04
          end_date: 2022-05-27
  value: -0.3

Configuration options

method: MultiPeriodModifier

groups: A list of subpopulations (subpops) or groups of them, and time periods the modification will be active in each of them

groups:subpop A list of subpopulation names/ids in which the specified modification will be applied. This can be a single subpop, a list, or the word "all" (specifying the modification applies to all existing subpopulations in the model). The modification will do nothing for any subpopulations not listed here.
groups: periods A list of time periods, each defined by a start and end date, when the modification will be applied
- groups:periods:start_date The date when the modification starts, in YYYY-MM-DD format. The modification will only reduce the value of the parameter after (inclusive of) this date.
- groups:periods:end_date The date when the modification ends, in YYYY-MM-DD format. The modification will only reduce the value of the parameter before (inclusive of) this date.

new_parameter_value = old_parameter_value * (1 - value)

ModifierModifier

ModifierModifier interventions allow the user to specify an intervention that acts to modify the value of another intervention, as opposed to modifying a baseline parameter value. The intervention multiplicatively reduces the modifier of interest by value (new = (1-value) * old) for the subpopulations listed in subpop during the time interval [period_start_date, period_end_date].

Example

For example, ModifierModifier could be used to describe a social distancing policy that is in effect between two dates and reduces transmission by 60% if followed by the whole population, but part way through this period, adherence to the policy drops to only 50% of in one of the subpopulations population:

seir_modifiers:
  modifiers:
    social_distancing: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-06-30
      subpop: ['all']
      value: 0.6
    fatigue: 
      method: ModifierModifier
      baseline_scenario: social_distancing
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['large_province']
      value: 0.5

Note that this configuration is identical to the following alternative specification

seir_modifiers:
  modifiers:
    social_distancing_initial: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-04-31
      subpop: ['all']
      value: 0.6
    social_distancing_fatigue_sp: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['small_province']
      value: 0.6
    social_distancing_fatigue_lp: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['large_province']
      value: 0.3

However, there are situations when the ModiferModifier notation is more convenient, especially when doing parameter fitting. ;

Configuration options

method: ModifierModifier

baseline_modifier: The name of the original parameter modification which will be further modified.

parameter: The name of the parameter in the baseline_scenario that will be modified ;

period_start_date: The date when the intervention modifier starts, in YYYY-MM-DD format. The intervention modifier will only reduce the value of the other intervention after (inclusive of) this date.

period_end_date: The date when the intervention modifier ends, in YYYY-MM-DD format. The intervention modifier will only reduce the value of the other intervention before (inclusive of) this date.

subpop:A list of subpopulation names/ids in which the specified intervention modifier will be applied. This can be a single subpop, a list, or the word "all" (specifying the interventions applies to all existing subpopulations in the model). The intervention will do nothing for any subpopulations not listed here.

value:The fractional reduction of the baseline intervention during the time period the modifier intervention is active. This can be a scalar number, or a distribution using the notation described in the Distributions section. The new parameter value will be

new_intervention_value = old_intervention_value * (1 - value)

and so the value of the underlying parameter that was modified by the baseline intervention will be

new_parameter_value = original_parameter_value * (1 - baseline_intervention_value * (1 - value) )

StackedModifier

Combine two or more modifiers into a scenario, so that they can easily be singled out to be run together without the other modifiers. If multiply modifiers act during the same time period in the same subpopulation, their effects are combined multiplicatively. Modifiers of different types (i.e. SinglePeriodModifier, MultiPeriodModifier, ModifierModifier, other StackedModifiers) can be combined ;

Examples

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs
  modifiers:
    SchoolClosures:
      method:SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.7
    CaseIsolation:
      method:SinglePeriodModifier
      parameter: gamma
      period_start_date: 2020-04-01
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -1.0
    Masking:
      method:SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-04-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.5
    AllNPIs
      method: StackedModifier
      modifiers: ["SchoolClosures","CaseIsolation","Masking"]

outcome_modifiers:
  scenarios:
    - ReducedTesting
    - AllDelays
  modifiers:
    DelayedTesting
      method:SinglePeriodModifier
      parameter: incidC::probability
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.5
    DelayedHosp
      method:SinglePeriodModifier
      parameter: incidD::delay
      period_start_date: 2020-04-01
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -1.0
    LongerHospStay
      method:SinglePeriodModifier
      parameter: incidH::duration
      period_start_date: 2020-04-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -0.5

Configuration options

method: StackedModifier

modifiers: A list of names of the other modifiers (specified above) that will be combined to create the new modifier (which we typically refer to as a "scenario")

modifiers::modifiers::groups

subpop_groups: For any of the modifier types, subpop_groups is an optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. All other subpopulations not listed will have unique intervention values unlinked to other areas. If the value is 'all', then all subpopulations will be assumed to have the same modifier value. When the subpop_groups option is not specified, all subpopulations will be assumed to have unique values of the modifier ;

For example, for a model of disease spread in Canada where we want to specify that the (to be varied) value of a modification to the transmission rate should be the same in all the Atlantic provinces (Nova Scotia, Newfoundland, Prince Edward Island, and New Brunswick), the same in all the prairie provinces (Manitoba, Saskatchewan, Alberta), the same in the three territories (Nunavut, Northwest Territories, and Yukon), and yet take unique values in Ontario, Quebec, and British Columbia, we could write

seir_modifiers:
  modifiers:
    lockdown: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      subpop_groups: [['NS','NB','PE','NF'],['MB','SK','AB'],['NV','NW','YK']]
      value: 
        distribution: uniform
        low: 0.3
        high: 0.7

Other configuration options

Command line inputs

flepiMoP allows some input parameters/options to be specified in the command line at the time of model submission, in addition to or instead of in the configuration file. This can be helpful for users who want to quickly run different versions of the model – typically a different number of simulations or a different intervention scenario from among all those specified in the config – without having to edit or create a new configuration file every time. In addition, some arguments can only be specified via the command line.

In addition to the configuration file and the command line, the inputs described below can also be specified as environmental variables.

In all cases, command line arguments override configuration file entries which override environmental variables. The order of command line arguments does not matter.

Details on how to run the model, including how to add command line arguments or environmental variables, are in the section How to Run.

Command-line only inputs

Argument

Env. Variable

Value type

Description

Required?

Default

-c or --config

CONFIG_PATH

file path

Name of configuration file. Must be located in the current working directory, or else relative or absolute file path must be provided.

Yes

-i or --first_sim_index

FIRST_SIM_INDEX

integar 1

The index of the first simulation

-j or --jobs

FLEPI_NJOBS

integar 1

Number of parallel processors used to run the simulation. If there are more slots that jobs, slots will be divided up between processors and run in series on each.

Number of processors on the computer used to run the simulation

--interactiveor --batch

Choose either option

Run simulation in interactive or batch mode

batch

--write-csv or --no-write-csv

Choose either option

Whether model output will be saved as .csv files

no_write_csv

--write-parquet or --no-write-parquet

Choose either option

Whether model output will be saved as .parquet files (a compressed representation that can be opened and manipulated with minimal memory. May be required for large simulations). Read more about .

write_parquet

Command-line versions of configuration file inputs

Argument

Config item

Env. Variable

Value type

Description

Required?

Default

-s or --npi_scenario

interventions: scenarios

FLEPI_NPI_SCENARIOS

list of strings

Names of the intervention scenarios described in the config file that will be run. Must be a subset of scenarios defined.

All scenarios described in config

-n or --nslots

nslots

FLEPI_NUM_SLOTS

integar 1

Number of independent simulations of the model to be run

Config value

--method or -m

seir: integration: method

`rk4`, `euler`, or `stochastic`

If provided, will override the `seir::integration::method` (including the default, if unspecified in the configuration file)

Config value if present, otherwise `rk4`

--in-id

FLEPI_RUN_INDEX

string

Unique ID given to the model runs. If the same config is run multiple times, you can avoid the output being overwritten by using unique model run IDs.

Constructed from current date and time as YYYY.MM.DD.HH/MM/SS

--out-id

FLEPI_RUN_INDEX

string

Unique ID given to the model runs. If the same config is run multiple times, you can avoid the output being overwritten by using unique model run IDs.

Constructed from current date and time as YYYY.MM.DD.HH/MM/SS

Example

As an example, consider running the following configuration file

name: sir
setup_name: minimal
start_date: 2020-01-31
end_date: 2020-05-31
nslots: 1

subpop_setup:
  geodata: geodata_sample_1pop.csv
  mobility: mobility_sample_1pop.csv
  popnodes: population
  nodenames: name

seeding:
  method: FromFile
  seeding_file: data/seeding_1pop.csv

compartments:
  infection_stage: ["S", "I", "R"]

seir:
  integration:
    method: stochastic
    dt: 1 / 10
  parameters:
    gamma:
      value:
        distribution: fixed
        value: 1 / 5
    Ro:
      value:
        distribution: uniform
        low: 2
        high: 3
  transitions:
    - source: ["S"]
      destination: ["I"]
      rate: ["Ro * gamma"]
      proportional_to: [["S"],["I"]]
      proportion_exponent: ["1","1"]
    - source: ["I"]
      destination: ["R"]
      rate: ["gamma"]
      proportional_to: ["I"]
      proportion_exponent: ["1"]

interventions:
  scenarios:
    - None
    - Lockdown
  modifiers:
    None:
      method: SinglePeriodModifier
      parameter: r0
      period_start_date: 2020-04-01
      period_end_date: 2020-05-15
      value:
        distribution: fixed
        value: 0
        settings:
    Lockdown:
      method: SinglePeriodModifier
      parameter: r0
      period_start_date: 2020-04-01
      period_end_date: 2020-05-15
      value:
        distribution: fixed
        value: 0.7

To run this model directly in Python (it can alternatively be run from R, for all details see section How to Run), we could use the command line entry

> flepimop simulate sir_control.yml

Alternatively, to run 100 simulations using only 4 of the available processors on our computer, but only running the "" scenario with a deterministic model, and to save the files as .csv (since the model is relatively simple), we could call the model using the command line entry

/> flepimop simulate -n 100 -j 4 -npi_scenario None -m euler --write_csv sir_control.yml

Environmental variables

TBA

US-specific configuration file options

Things below here are very out of date. Put here as place holder but not updated recently.

global: smh_round, setup_name, disease

spatial_setup: census_year, modeled_states, state_level

For US-specific population structures

For creating US-based population structures using the helper script build_US_setup.R which is run before the main model simulation script, the following extra parameters can be specified

Config Item

Required?

Type/Format

Description

census_year

optional

integer (year)

Determines the year for which census population size data is pulled.

state_level

optional

boolean

Determines whether county-level population-size data is instead grouped into state-level data (TRUE). Default FALSE

modeled_states

optional

list of location codes

A vector of locations that will be modeled; others will be ignored

Example 2

To simulate an epidemic across all 50 states of the US or a subset of them, users can take advantage of built in machinery to create geodata and mobility files for the US based on the population size and number of daily commuting trips reported in the US Census.

Before running the simulation, the script build_US_setup.R can be run to get the required population data files from online census data and filter out only states/territories of interest for the model. More details are provided in the How to Run section.

This example simulates COVID-19 in the New England states, assuming no transmission from other states, using 2019 census data for the population sizes and a pre-created file for estimated interstate commutes during the 2011-2015 period.

subpop_setup:
  census_year: 2010
  state_level: TRUE
  geodata: geodata_2019_statelevel.csv
  mobility: mobility_2011-2015_statelevel.csv
  modeled_states:
    - CT
    - MA
    - ME
    - NH
    - RI
    - VT

geodata.csv contains

USPS	subpop	population
AL	01000	4876250
AK	02000	737068
AZ	04000	7050299
AR	05000	2999370
CA	06000	39283497
.....

mobility_2011-2015_statelevel.csv contains

ori	dest	amount
01000	02000	198
01000	04000	292
01000	05000	570
01000	06000	1030
01000	08000	328
.....

`importation` section (optional)

This section is optional. It is used by the covidImportation package to import global air importation data for seeding infections into the United States.

If you wish to include it, here are the options.

Config Item

Required?

Type/Format

Description

census_api_key

required

string

travel_dispersion

required

number

ow dispersed daily travel data is; default = 3.

maximum_destinations

required

integer

number of airports to limit importation to

dest_type

required

categorical

location type

dest_country

required

string (Country)

ISO3 code for country of importation. Currently only USA is supported

aggregate_to

required

categorical

location type to aggregate to

cache_work

required

boolean

whether to save case data

update_case_data

required

boolean

deprecated; whether to update the case data or used saved

draw_travel_from_distribution

required

boolean

whether to add additional stochasticity to travel data; default is FALSE

print_progress

required

boolean

whether to print progress of importation model simulations

travelers_threshold

required

integer

include airports with at least the travelers_threshold mean daily number of travelers

airport_cluster_distance

required

numeric

cluster airports within airport_cluster_distance km

param_list

required

See section below

see below

`importation::param_list`

Config Item

Required?

Type/Format

Description

incub_mean_log

required

numeric

incubation period, log mean

incub_sd_log

required

numeric

incubation period, log standard deviation

inf_period_nohosp_mean

required

numeric

infectious period, non-hospitalized, mean

inf_period_nohosp_sd

required

numeric

infectious period, non-hospitalized, sd

inf_period_hosp_mean_log

required

numeric

infectious period, hospitalized, log-normal mean

inf_period_hosp_sd_log

required

numeric

infectious period, hospitalized, log-normal sd

p_report_source

required

numeric

reporting probability, Hubei and elsewhere

shift_incid_days

required

numeric

mean delay from infection to reporting of cases; default = -10

delta

required

numeric

days per estimations period

importation:
  census_api_key: "fakeapikey00000"
  travel_dispersion: 3
  maximum_destinations: Inf
  dest_type: state
  dest_county: USA
  aggregate_to: airport
  cache_work: TRUE
  update_case_data: TRUE
  draw_travel_from_distribution: FALSE
  print_progress: FALSE
  travelers_threshold: 10000
  airport_cluster_distance: 80
  param_list:
    incub_mean_log: log(5.89)
    incub_sd_log: log(1.74)
    inf_period_nohosp_mean: 15
    inf_period_nohosp_sd: 5
    inf_period_hosp_mean_log: 1.23
    inf_period_hosp_sd_log: 0.79
    p_report_source: [0.05, 0.25]
    shift_incid_days: -10
    delta: 1

`report` section

The report section is completely optional and provides settings for making an R Markdown report. For an example of a report, see the Supplementary Material of our preprint

If you wish to include it, here are the options.

Config Item

Required?

Type/Format

Description

data_settings::pop_year

integer

plot_settings::plot_intervention

boolean

formatting::scenario_labels_short

list of strings; one for each scenario in interventions::scenarios

formatting::scenario_labels

list of strings; one for each scenario in interventions::scenarios

formatting::scenario_colors

list of strings; one for each scenario in interventions::scenarios

formatting::pdeath_labels

list of strings

formatting::display_dates

list of dates

formatting::display_dates2

optional

list of dates

a 2nd string of display dates that can optionally be supplied to specific report functions

report:
  data_settings:
    pop_year: 2018
  plot_settings:
    plot_intervention: TRUE
  formatting:
    scenario_labels_short: ["UC", "S1"]
    scenario_labels:
      - Uncontrolled
      - Scenario 1
    scenario_colors: ["#D95F02", "#1B9E77"]
    pdeath_labels: ["0.25% IFR", "0.5% IFR", "1% IFR"]
    display_dates: ["2020-04-15", "2020-05-01", "2020-05-15", "2020-06-01", "2020-06-15"]
    display_dates2: ["2020-04-15", "2020-05-15", "2020-06-15"]

Code structure

Model Output

(This section describes the location and contents of each of the output files produced during a non-inference model run)

The model will output 2–6 different types of files depending on whether the configuration file contains optional sections (such interventions, outcomes, and outcomes interventions) and whether model inference is conducted ;

These files contain the values of the variables for both the infection and (if included) observational model at each point in time and for each subpopulation. A new file of the same type is produced for each independent simulation and each intervention scenario. Other files report the values of the initial conditions, seeding, and model parameters for each subpopulation and independent simulation (since parameters may be chosen to vary randomly between simulations). When model inference is run, there are also file types reporting the model likelihood (relative to the provided data) and files for each iteration of the inference algorithm.

Within the model_output directory in the project's directory, the files will be organized into folders named for the file types: seir, spar, snpi, hpar, hnpi, seed, init, or llik (see descriptions below). Within each file type folder, files will further be organized by the simulation name (setup_name in config), the modifier scenario names - if scenarios exist for either seir or outcome parameters (specified with seir_modifiers::scenarios and outcome_modifiers::scenarios in config), and the run_id (the date and time of the simulation, by default). For example:

flepiMoP/examples/tutorials
├── model_output
│   ├── {setup_name}_{seir_modifier_scenario}_{outcome_modifier_scenario}
│   │   └── run_id
│   │       └── seir
│   │           └── 000000001.run_id.seir.parquet
│   ├── spar
│   ├── snpi

The name of each individual file contains (in order) the slot, run_id and file type. The first index indicates the slot (chain, in MCMC language). If multiple iterations or blocks are run, the filename will look like 000000001.000000001.000000001.run_id.seir.parquet indicating slot.block.iteration ;

Each file is a data table that is by default saved as a parquet file (a compressed representation that can be opened and manipulated with minimal memory) but can alternatively be saved as a csv file. See options for specifying output type in Other Configuration Options.

The example files outputs we show were generated with the following configuration file

TBA

The types and contents of the model output files changes slightly depending on whether the model is run as a forward simulation only, or is run in inference mode, in which parameter values are estimated by comparing the model to data. Output specific to model inference is described in a separate section ;

SEIR (infection model output)

Files in the seir folder contain the output of the infection model over time. They contain the value of every variable for each day of the simulation for every subpopulation.

For the example configuration file shown above, the seir file is

// Some code

The meanings of the columns are:

mc_value_type – either prevalence or incidence. Variable values are reported both as a prevalence (number of individuals in that state measured instantaneously at the start of the day, equivalent to the meaning of the S, I, or R variable in the differential equations or their stochastic representation) and as incidence (total number of individuals who newly entered this state, from all other states, over the course of the 24-hour period comprising that calendar day).

mc_infection_stage, mc_vaccination_status, etc. – The name of the compartment for which the value is reported, broken down into the infection stage for each state type (eg. vaccination, age).

mc_name – The name of the compartment for which the value is reported, which is a concatenation of the compartment status in each state type.

subpop_1, subpop_2, etc. – one column for each different subpopulation, containing the value of the number of individuals in the described compartment in that subpopulation at the given date. Note that these are named after the nodenames defined by the user in the geodata file.

date – The calendar date in the simulation, in YYYY-MM-DD format.

There will be a separate seir file output for each slot (independent simulation) and for each iteration of the simulation if Model Inference is conducted.

SPAR (infection model parameter values)

The files in the spar folder contain the parameters that define the transitions in the compartmental model of disease transmission, defined in the seir::parameters section of the config ;

The value column gives the numerical values of the parameters defined in the corresponding column parameter.

SNPI (infection model parameter intervention values)

Files in the snpi folder contain the time-dependent modifications to the transmission model parameter values (defined in seir_modifiers section of the config) for each subpopulation. They contain the modifiers that apply to a given subpopulation and the dates within which they apply, and the value of the reduction to the given parameter.

The meanings of the columns are:

subpop – The subpopulation to which this intervention parameter applies.

modifier_name – The name of the intervention parameter.

start_date – The start date of this intervention, as defined in the configuration file.

end_date – The end date of this intervention, as defined in the configuration file.

parameter – The parameter to which the intervention applies, as defined in the configuration file.

value – The size of the modifier to the parameter either from the config, or fit by inference if that is run.

HPAR (observation model parameter values)

Files in the hpar folder contain the output parameters of the observational model. They contain the values of the probabilities, delays or durations for each outcome in a given subpopulation.

The meanings of the columns are:

subpop – Values in this column are the names of the nodes as defined in the geodata file given by the user.

quantity – The values in this column are the types of parameter values described in the config. The options are probability, delay, and duration. These are the quantities to which there is some parameter defined in the config.

outcome – The values here are the outcomes to which this parameter applies. These are names of the outcome compartments defined in the model.

value – The values in this column are the parameter values of the quantity that apply to the given subpopulation and outcome.

HOSP (observation model output)

Files in the hosp folder contain the output of the infection model over time. They contain the value of every outcome variable for each day of the simulation for every subpopulation.

Columns are:

date – The calendar date in the simulation, in YYYY-MM-DD format.

subpop – Values in this column are the names of the nodes as defined in the geodata file given by the user.

outcome_variable_1, outcome_variable_2, ... - one column for each different outcome variable as defined in the config, containing the value of the number of individuals in the described compartment in that subpopulation at the given date ;

HNPI (observation model parameter intervention values)

Files in the hnpi folder contain any parameter modifier values that apply to the outcomes model, defined in the outcome_modifiers section of the config. They contain the values of the outcome parameter modifiers, and the dates to which they apply in a given subpopulation.

The meanings of the columns are:

subpop – The values of this column are the names of the nodes from the geodata file.

modifier_name – The names/labels of the modifier parameters, defined by the user in the config file, which applies to the given node and time period.

start_date – The start date of this intervention, as defined in the configuration file.

end_date – The end date of this intervention, as defined in the configuration file.

parameter – The outcome parameter to which the intervention applies.

value – The values in this column are the modifier values of the intervention parameters, which apply to the given parameter in a given subpopulation. Note that these are strictly reductions; thus a negative value corresponds to an increase in the parameter, while a positive value corresponds to a decrease in the parameter.

SEED (model seeding values)

Files in the seed folder contain the seeded values of the infection model. They contain the amounts seeded into each variable, the variable they are seeded from, and the time at which the seeding occurs. The user can provide a single seeding file (which will be used across all simulations), or, if multiple simulations are being run the user can provide a separate file for each simulation ;

The meanings of the columns are:

subpop - The values of this column are the names of the nodes from the geodata file.

date - The values in this column are the dates of seeding.

amount - The amount seeded in the given subpopulation from source variables to destination variables, at the given date ;

source_infection_stage, source_vaccination_status, etc. - The name of the compartment from which the amount is seeded, broken down into the infection stage for each state type (eg. vaccination, age).

destination_infection_stage, destination_vaccination_status, etc. - The name of the compartment into which the amount is seeded, broken down into the infection stage for each state type (eg. vaccination, age).

no_perturb - The values in this column can be either true or false. If true, then the amount and/or date can be perturbed if running an inference run. Whether the amount or date is perturbed is defined in the config using perturb_amount and perturb_date ;

INIT (model initial conditions)

Files in the init folder contain the initial values of the infection model. Either seed or init files will be present, depending on the configuration of the model . These files contain the initial conditions of the infection model at the start date defined in the configuration file. As with seeding, the user can provide a single initial conditions file (which will be used across all simulations), or, if multiple simulations are being run the user can provide a separate file for each simulation ;

The meanings of the columns are:

subpop - The values of this column are the names of the nodes from the geodata file.

mc_infection_stage, mc_vaccination_status, etc. - The name of the compartment for which the value is reported, broken down into the infection stage for each state type (eg. vaccination, age).

amount - The amount initialized seeded in the given subpopulation at the start date defined in the configuration file ;

Model Inference

Inference Implementation

(OLD) Configuration options

`filtering` section

The filtering section configures the settings for the inference algorithm. The below example shows the settings for some typical default settings, where the model is calibrated to the weekly incident deaths and weekly incident confirmed cases for each subpop. Statistics, hierarchical_stats_geo, and priors each have scenario names (e.g., sum_deaths, local_var_hierarchy, and local_var_prior, respectively).

filtering:
  simulations_per_slot: 350
  do_filtering: TRUE
  gt_data_path: data/observed_data.csv
  likelihood_directory: importation/likelihood/
  statistics:
    sum_deaths:
      name: sum_deaths
      aggregator: sum ## function applied over the period
      period: "1 weeks"
      sim_var: incidD
      data_var: death_incid
      remove_na: TRUE
      add_one: FALSE
      likelihood:
        dist: sqrtnorm
        param: [.1]
    sum_confirmed:
      name: sum_confirmed
      aggregator: sum
      period: "1 weeks"
      sim_var: incidC
      data_var: confirmed_incid
      remove_na: TRUE
      add_one: FALSE
      likelihood:
        dist: sqrtnorm
        param: [.2]
  hierarchical_stats_geo:
    local_var_hierarchy:
      name: local_variance
      module: seir
      geo_group_col: USPS
      transform: none
    local_conf:
      name: probability_incidI_incidC
      module: hospitalization
      geo_group_col: USPS
      transform: logit
  priors:
    local_var_prior:
      name: local_variance
      module: seir
      likelihood:
        dist: normal
        param:
        - 0
        - 1

`filtering` settings

With inference model runs, the number of simulations nsimulations refers to the number of final model simulations that will be produced. The filtering$simulations_per_slot setting refers to the number of iterative simulations that will be run in order to produce a single final simulation (i.e., number of simulations in a single MCMC chain).

Item

Required?

Type/Format

simulations_per_slot

required

number of iterations in a single MCMC inference chain

do_filtering

required

TRUE if inference should be performed

data_path

required

file path where observed data are saved

likelihood_directory

required

folder path where likelihood evaluations will be stored as the inference algorithm runs

statistics

required

specifies which data will be used to calibrate the model. see filtering::statistics for details

hierarchical_stats_geo

optional

specifies whether a hierarchical structure should be applied to any inferred parameters. See filtering::hierarchical_stats_geo for details.

priors

optional

specifies prior distributions on inferred parameters. See filtering::priors for details

`filtering::statistics`

The statistics specified here are used to calibrate the model to empirical data. If multiple statistics are specified, this inference is performed jointly and they are weighted in the likelihood according to the number of data points and the variance of the proposal distribution.

Item

Required?

Type/Format

name

required

name of statistic, user defined

aggregator

required

function used to aggregate data over the period, usually sum or mean

period

required

duration over which data should be aggregated prior to use in the likelihood, may be specified in any number of days, weeks, months

sim_var

required

column name where model data can be found, from the hospitalization outcomes files

data_var

required

column where data can be found in data_path file

remove_na

required

logical

add_one

required

logical, TRUE if evaluating the log likelihood

likelihood::dist

required

distribution of the likelihood

likelihood::param

required

parameter value(s) for the likelihood distribution. These differ by distribution so check the code in inference/R/functions.R/logLikStat function.

`filtering::hierarchical_stats_geo`

The hierarchical settings specified here are used to group the inference of certain parameters together (similar to inference in "hierarchical" or "fixed/group effects" models). For example, users may desire to group all counties in a given state because they are geograhically proximate and impacted by the same statewide policies. The effect should be to make these inferred parameters follow a normal distribution and to observe shrinkage among the variance in these grouped estimates.

Item

Required?

Type/Format

scenario name

required

name of hierarchical scenario, user defined

name

required

name of the estimated parameter that will be grouped (e.g., the NPI scenario name or a standardized, combined health outcome name like probability_incidI_incidC)

module

required

name of the module where this parameter is estimated (important for finding the appropriate files)

geo_group_col

required

geodata column name that should be used to group parameter estimation

transform

required

type of transform that should be applied to the likelihood: "none" or "logit"

`filtering::priors`

It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.

Item

Required?

Type/Format

scenario name

required

name of prior scenario, user defined

name

required

name of NPI scenario or parameter that will have the prior

module

required

name of the module where this parameter is estimated

likelihood

required

specifies the distribution of the prior

Ground truth data

Likelihood function

Fitting parameters

Ground truth data

(OLD) Configuration setup

Need to add MultiPeriodModifier and hospitalization interventions

Overview

This documentation describes the new YAML configuration file options that may be used when performing inference on model runs. As compared to previous model releases, there are additions to the seeding and interventions sections, the outcomes section replaces the hospitalization section, and the filtering section added to the file.

Importantly, we now name our pipeline modules: seeding, seir, hospitalization and this becomes relevant to some of the new filtering specifications.

Models may be calibrated to any available time series data that is also an outcome of the model (COVID-19 confirmed cases, deaths, hospitalization or ICU admissions, hospital or ICU occupancy, and ventilator use). Our typical usage has calibrated the model to deaths, confirmed cases, or both. We can also perform inference on intervention effectiveness, county-specific baseline R0, and the risk of specific health outcomes.

We describe these options below and present default values in the example configuration sections.

Modifications to `seeding`

The model can perform inference on the seeding date and initial number of seeding infections in each subpop. An example of this new config section is:

seeding:
  method: FolderDraw
  seeding_file_type: seed
  folder_path: importation/minimal/
  lambda_file: data/minimal/seeding.csv
  perturbation_sd: 3

Config Item

Required?

Type/Format

Description

method

required

"FolderDraw"

seeding_file_type

required for FolderDraw

"seed" or "impa"

indicates which seeding file type the SEIR model will look for, "seed", which is generated from create_seeding.R, or "impa", which refers to importation

folder_path

required

path to folder where importation inference files will be saved

lambda_file

required

path to seeding file

perturbation_sd

required

standard deviation for the proposal value of the seeding date, in number of days

The method for determining the proposal distribution for the seeding amount is hard-coded in the inference package (R/pkgs/inference/R/functions/perturb_seeding.R). It is pertubed with a normal distribution where the mean of the distribution 10 times the number of confirmed cases on a given date and the standard deviation is 1.

Modifications to `interventions`

The model can perform inference on the effectiveness of interventions as long as there is at least some calibration health outcome data that overlaps with the intervention period. For example, if calibrating to deaths, there should be data from time points where it would be possible to observe deaths from infections that occurred during the intervention period (e.g., assuming 10-18 day delay between infection and death, on average).

An example configuration file where inference is performed on scenario planning interventions is as follows:

interventions:
  scenarios:
    - Scenario1
  settings:
    local_variance:
      template: SinglePeriodModifierR0
      value:
        distribution: truncnorm
        mean: 0
        sd: .1
        a: -1
        b: 1
      perturbation:
        distribution: truncnorm
        mean: 0
        sd: .1
        a: -1
        b: 1
    stayhome:
      template: SinglePeriodModifierR0
      period_start_date: 2020-04-04
      period_end_date: 2020-04-30
      value:
        distribution: truncnorm
        mean: 0.6
        sd: 0.3
        a: 0
        b: 0.9
      perturbation:
        distribution: truncnorm
        mean: 0
        sd: .1
        a: -1
        b: 1
    Scenario1:
      template: StackedModifier
      scenarios: 
        - local_variance
        - stayhome

`interventions::settings::[setting_name]`

This configuration allows us to infer subpop-level baseline R0 estimates by adding a local_variance intervention. The baseline subpop-specific R0 estimate may be calculated as $R0*(1-local_variance),$ where R0 is the baseline simulation R0 value, and local_variance is an estimated subpop-specific value.

Interventions may be specified in the same way as before, or with an added perturbation section that indicates that inference should be performed on a given intervention's effectiveness. As previously, interventions with perturbations may be specified for all modeled locations or for explicit subpop. In this setup, both the prior distribution and the range of the support of the final inferred value are specified by the value section. In the configuration above, the inference algorithm will search 0 to 0.9 for all subpops to estimate the effectiveness of the stayhome intervention period. The prior distribution on intervention effectiveness follows a truncated normal distribution with a mean of 0.6 and a standard deviation of 0.3. The perturbation section specifies the perturbation/step size between the previously-accepted values and the next proposal value.

Item

Required?

Type/Format

template

Required

"SinglePeriodModifierR0" or "StackedModifier"

period_start_date

optional for SinglePeriodModifierR0

date between global start_date and end_date; default is global start_date

period_end_date

optional for SinglePeriodModifierR0

date between global start_date and end_date; default is global end_date

value

required for SinglePeriodModifierR0

specifies both the prior distribution and range of support for the final inferred values

perturbation

optional for SinglePeriodModifierR0

this option indicates whether inference will be performed on this setting and how the proposal value will be identified from the last accepted value

subpop

optional for SinglePeriodModifierR0

list of subpops, which must be in geodata

New `outcomes` section

This section is now structured more like the interventions section of the config, in that it has scenarios and settings. We envision that separate scenarios will be specified for each IFR assumption.

outcomes:
  method: delayframe
  param_from_file: TRUE
  param_subpop_file: "usa-subpop-params-output.parquet" ## ../../Outcomes/data/usa-subpop-params-output.parquet
  scenarios:
    - med
  settings:
    med:
      incidH:
        source: incidI
        probability:
          value:
            distribution: fixed
            value: .035
        delay:
          value:
            distribution: fixed
            value: 7
        duration:
          value:
            distribution: fixed
            value: 7
          name: hosp_curr
      incidD:
        source: incidI
        probability:
          value:
            distribution: fixed
            value: .01
        delay:
          value:
            distribution: fixed
            value: 20
      incidICU:
        source: incidH
        probability: 
          value:
            distribution: fixed
            value: 0.167
        delay:
          value:
            distribution: fixed
            value: 3
        duration:
          value:
            distribution: fixed
            value: 8
      incidVent:
        source: incidICU
        probability: 
          value:
            distribution: fixed
            value: 0.463
        delay:
          value:
            distribution: fixed
            value: 1
        duration:
          value:
            distribution: fixed
            value: 7
      incidC:
        source: incidI
        probability:
          value:
            distribution: truncnorm
            mean: .1
            sd: .1
            a: 0
            b: 10
          perturbation:
            distribution: truncnorm
            mean: 0
            sd: .1
            a: -1
            b: 1
        delay:
          value:
            distribution: fixed
            value: 7

Item

Required?

Type/Format

method

required

"delayframe"

param_from_file

required

if TRUE, will look for param_subpop_file

param_subpop_file

optional

path to subpop-params parquet file, which indicates location specific risk values. Values in this file will override values in the config if there is overlap.

scenarios

required

user-defined scenario name

settings

required

See details below

`outcomes::settings::[setting_name]`

The settings for each scenario correspond to a set of different health outcome risks, most often just differences in the probability of death given infection (Pr(incidD|incidI)) and the probability of hospitalization given infection (Pr(incidH|incidI)). Each health outcome risk is referenced in relation to the outcome indicated in source. For example, the probability and delay in becoming a confirmed case (incidC) is most likely to be indexed off of the number and timing of infection (incidI).

Importantly, we note that incidI is automatically defined from the SEIR transmission model outputs, while the other compartment sources must be defined in the config before they are used.

Users must specific two metrics for each health outcome, probability and delay, while a duration is optional (e.g., duration of time spent in the hospital). It is also optional to specify a perturbation section (similar to perturbations specified in the NPI section) for a given health outcome and metric. If you want to perform inference (i.e., if perturbation is specified) on a given metric, that metric must be specified as a distribution (i.e., not fixed) and the range of support for the distribution represents the range of parameter space explored in the inference.

Item

Required?

Type/Format

(health outcome metric)

required

"incidH", "incidD", "incidICU", "incidVent", "incidC", corresponding to variable names

source

required

name of health outcome metric that is used as the reference point

probability

required

health outcome risk

probability::value

required

specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution

probability::perturbation

optional

inference settings for the probability metric

delay

required

time delay between source and the specified health outcome

delay::value

required

specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution

delay::perturbation

optional

inference settings for the time delay metric (coming soon)

duration

optional

duration that health outcome status endures

duration::value

required

specifies whether the value is fixed or distributional and the parameters specific to that metric and distribution

duration::perturbation

optional

inference settings for the duration metric (coming soon)

New `filtering` section

This section configures the settings for the inference algorithm. The below example shows the settings for some typical default settings, where the model is calibrated to the weekly incident deaths and weekly incident confirmed cases for each subpop. Statistics, hierarchical_stats_geo, and priors each have scenario names (e.g., sum_deaths, local_var_hierarchy, and local_var_prior, respectively).

filtering:
  simulations_per_slot: 350
  do_filtering: TRUE
  data_path: data/observed_data.csv
  likelihood_directory: importation/likelihood/
  statistics:
    sum_deaths:
      name: sum_deaths
      aggregator: sum ## function applied over the period
      period: "1 weeks"
      sim_var: incidD
      data_var: death_incid
      remove_na: TRUE
      add_one: FALSE
      likelihood:
        dist: sqrtnorm
        param: [.1]
    sum_confirmed:
      name: sum_confirmed
      aggregator: sum
      period: "1 weeks"
      sim_var: incidC
      data_var: confirmed_incid
      remove_na: TRUE
      add_one: FALSE
      likelihood:
        dist: sqrtnorm
        param: [.2]
  hierarchical_stats_geo:
    local_var_hierarchy:
      name: local_variance
      module: seir
      geo_group_col: USPS
      transform: none
    local_conf:
      name: probability_incidI_incidC
      module: hospitalization
      geo_group_col: USPS
      transform: logit
  priors:
    local_var_prior:
      name: local_variance
      module: seir
      likelihood:
        dist: normal
        param:
        - 0
        - 1

`filtering` settings

Item

Required?

Type/Format

simulations_per_slot

required

number of iterations in a single MCMC inference chain

do_filtering

required

TRUE if inference should be performed

data_path

required

file path where observed data are saved

likelihood_directory

required

folder path where likelihood evaluations will be stored as the inference algorithm runs

statistics

required

specifies which data will be used to calibrate the model. see filtering::statistics for details

hierarchical_stats_geo

optional

specifies whether a hierarchical structure should be applied to any inferred parameters. See filtering::hierarchical_stats_geo for details.

priors

optional

specifies prior distributions on inferred parameters. See filtering::priors for details

`filtering::statistics`

Item

Required?

Type/Format

name

required

name of statistic, user defined

aggregator

required

function used to aggregate data over the period, usually sum or mean

period

required

duration over which data should be aggregated prior to use in the likelihood, may be specified in any number of days, weeks, months

sim_var

required

column name where model data can be found, from the hospitalization outcomes files

data_var

required

column where data can be found in data_path file

remove_na

required

logical

add_one

required

logical, TRUE if evaluating the log likelihood

likelihood::dist

required

distribution of the likelihood

likelihood::param

required

parameter value(s) for the likelihood distribution. These differ by distribution so check the code in inference/R/functions.R/logLikStat function.

`filtering::hierarchical_stats_geo`

Item

Required?

Type/Format

scenario name

required

name of hierarchical scenario, user defined

name

required

name of the estimated parameter that will be grouped (e.g., the NPI scenario name or a standardized, combined health outcome name like probability_incidI_incidC)

module

required

name of the module where this parameter is estimated (important for finding the appropriate files)

geo_group_col

required

geodata column name that should be used to group parameter estimation

transform

required

type of transform that should be applied to the likelihood: "none" or "logit"

`filtering::priors`

It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.

Item

Required?

Type/Format

scenario name

required

name of prior scenario, user defined

name

required

name of NPI scenario or parameter that will have the prior

module

required

name of the module where this parameter is estimated

likelihood

required

specifies the distribution of the prior

Code structure

Files where these algorithms are contained

COVIDScenarioPipeline

/R/scripts

filter_MC.R
full_filter.R
build_US_setup.R
build_nonUS_setup.R

/R/pkgs

inference
- groundtruth.R
- functions.R
- filter_MC_runner_funcs.R

Inference Model Output

(This section describes the location and contents of the additional output files produced during an inference model run)

Updates to other files

\LLIK (inference runs only)

During inference runs, an additional file type, llik, is created, which is described in the Inference Model Output section ;

These files contain the log-likelihoods of the model simulation for each subpopulation, as well as some diagnostics on acceptance.

The meanings of the columns are:

ll - These values are the log-likelihoods of data given the model and parameter values for a single subpopulation (in subpop column) ;

filename - ...

subpop - The values of this column are the names of the nodes from the geodata file.

accept - Either 0 or 1, depending on whether the parameters during this iteration of the simulation were accepted (1) or rejected (0) in that subpopulation ;

accept_avg ;

accept_prob ;

For inference runs, ... flepiMoP produces one file per parallel slot, for both global and chimeric outputs...

flepiMoP/examples/tutorials
├── model_output
│   ├── seir
│   ├── spar
│   ├── snpi
│   └── llik
│       └── sample_2pop
│           └── None
│               └── 2023.05.24.02/12/48.
│                   ├── chimeric
│                   └── global
│                       ├── final
│                       │   └── 000000001.2023.05.24.02/12/48..llik.parquet
│                       └── intermediate
│                           └── 000000001.000000001.2023.05.24.02/12/48..llik.parquet

Config writer

The model needs the configurations file to run (described in previous sections). These configs become lengthy and sometimes difficult to type manually. The config writer helps to generate configs provided the relevant files are present.

Print Functions:

These functions are used to print specific sections of the configuration files.

print_header

Used to generate the global header. For more information on global headers click HERE.

Variable name

Required (default value if optional)

Description

sim_name

Required

Name of the configuration file to be generated. Generally based on the type of simulation

setup_name

Optional (SMH)

Type of run - a Scenario Modeling Hub ("SMH") or Forecasting Hub ("FCH") Simulation.

disease

Optional (covid19)

Pathogen or disease being simulated

smh_round

Optional (NA)

Round number for Scenario Modeling Hub Submission

model_output_dir_name

Optional (model_output)

Folder path where the outputs of the simulated model are stored

sim_start_date

Required

Start date for model simulation

sim_end_date

Required

End date for model simulation

start_date_groundtruth

Optional (NA)

Start date for fitting data for inference runs

end_date_groundtruth

Optional (NA)

End date for fitting data for inference runs

nslots

Required

Number of independent simulations to run

print_spatial_setup

Used to generate the spatial setup section of the configuration. For more information on spatial setup click HERE.

Variable name

Required (default value if optional)

Description

census_year

Optional (2019)

The year of data uses to generate the geodata files for US simulations ?? [Unsure about this]

sim_states

Required

Vector of locations that will be modeled (US Specific?)

geodata_file

Optional (geodata.csv)

Name of the geodata file which is imported

mobility_file

Optional (mobility.csv)

Name of the mobility file which is imported

popnodes

Optional (pop2019est)

Name of a column in the geodata file that specifies the population of every subpopulation column

nodenames

Optional (subpop)

Name of a column in the geodata file that specifies the name of the subpopulation

state_level

Optional (TRUE)

Specifies if the subpopulations are US states

print_compartments

Used to generate the compartment list for each way a population can be divided.

Variable Name

Required (default value if optional)

Description

inf_stages

Optional (S,E,I1,I2,I3,R,W)

Various infection stages an individual can be in

vaccine_compartments

Optional (unvaccinated, 1dose, 2dose, waned)

Various levels of vaccinations an individual can have

variant_compartments

Optional (WILD, ALPHA, DELTA, OMICRON)

Variants of the pathogen

age_strata

Optional (age0to17, age18to64, age65to100)

Different age groups, the population has been stratified in

Parts of the configuration files that are printed but not needed for FlepiMop runs (need to be mentioned for US or COVID-19 specific runs??)

Spatial Setup:

census year: year of geodata files
modeled states (sim_states): This has US state abbreviations. Do we include the names of the sub-populations in the geodata file? Eg: small_province, large_province
state_level: Specifies if the runs are run for US states

How To Run

Development

Deprecated pages

JHU Internal

Running with docker on AWS - OLD probably outdated

This page, along with the other AWS run guides, are not deprecated in case we need to run flepiMoP on AWS again in the future, but also are not maintained as other platforms (such as longleaf and rockfish) are preferred for running production jobs.

For large simulations, running the model on a cluster or cloud computing is required. AWS provides a good solution for this, if funding or credits are available (AWS can get very expensive).

Introduction

This document contains instructions for setting up and running the two different kinds of SEIR modeling jobs supported by the COVIDScenarioPipeline repository on AWS:

Inference jobs, using AWS Batch to coordinate hundreds/thousands of jobs across a fleet of servers, and
Planning jobs, using a single relatively large EC2 instance (usually an r5.24xlarge) to run one or more planning scenarios on a single high-powered machine.

Most of the steps required to setup and run the two different types of jobs on AWS are identical, and I will explicitly call out the places where different steps are required. Throughout the document, we assume that your client machine is a UNIX-like environment (e.g., OS X, Linux, or WSL).

Local Client Setup

I need a few things to be true about the local machine that you will be using to connect to AWS that I'll outline here:

You have created and downloaded a .pem file for connecting to an EC2 instance to your ~/.ssh directory. When we provision machines, you'll need to use the .pem file as a secret key for connecting. You may need to change the permission of the .pem file:
```
chmod 400 ~/.ssh/<your .pem file goes here>
```

You have created a ~/.ssh/config file that contains an entry that looks like this so we can use staging as an alias for your provisioned EC2 instance in the rest of the runbook:

host staging
HostName <IP address of provisioned server goes here>
IdentityFile ~/.ssh/<your .pem file goes here>
User ec2-user
IdentitiesOnly yes
StrictHostKeyChecking no

You can connect to Github via SSH. This is important because we will need to use your Github SSH key to interact with private repositories from the staging server on EC2.

Provisioning The Staging Server

If you are running an Inference job, you should use a small instance type for your staging server (e.g., an m5.xlarge will be more than enough.) If you are running a Planning job, you should provision a beefy instance type (I am especially partial to the memory-and-CPU heavy r5.24xlarge, but given how fast the planning code has become, an r5.8xlarge should be perfectly adequate.)

If you have access to the jh-covid account, you should use the IDD Staging AMI (ami-03641dd0c8554e5d0) to provision and launch new staging servers; it is already setup with all of the dependencies described in this section, however you will need to alter it's default network settings, iam role and security group(Please refer this page in details). You can find the AMI here, select it, and press the Launch button to walk you through the Launch Wizard to choose your instance type and .pem file to provision your staging server. When going through the Launch Wizard, be sure to select Next: Configure Instance details instead of Review and Launch. You will need to continue selecting the option that is not Review and Launch until you have selected a security group. In these screens, most of the default options are fine, but you will want to set the HPC VPC network, choose a public subnet (it will say public or private in the name), and set the iam role to EC2S3FullAccess on the first screen. You can also name the machine by providing a Name tag in the tags screen. Finally, you will need to set your security group to dcv_usa and/or dcv_usa2. You can then finalize the machine initialization with Review and Launch. Once your instance is provisioned, be sure to put its IP address into the HostName section of the ~/.ssh/config file on your local client so that you can connect to it from your client by simply typing ssh staging in your terminal window.

If you are having connection timeout issues when trying to ssh into the AWS machine, you should check that you have SSH TCP Port 22 permissions in the dcv_usa/ security group.

If you do not have access to the jh-covid account, you should walk through the regular EC2 Launch Wizard flow and be sure to choose the Amazon Linux 2 AMI (HVM), SSD Volume Type (ami-0e34e7b9ca0ace12d, the 64-bit x86 version) AMI. Once the machine is up and running and you can SSH to it, you will need to run the following code to install the software you will need for the rest of the run:

sudo yum -y update
sudo yum -y install awscli 
sudo yum -y install git 
sudo yum -y install docker.io 
sudo yum -y install pbzip2 

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
sudo yum -y install git-lfs
git lfs install

Connect to Github

Once your staging server is provisioned and you can connect to it, you should scp the private key file that you use for connecting to Github to the /home/ec2-user/.ssh directory on the staging server (e.g., if the local file is named ~/.ssh/id_rsa, then you should run scp ~/.ssh/id_rsa staging:/home/ec2-user/.ssh to do the copy). For convenience, you should create a /home/ec2-user/.ssh/config file on the staging server that has the following entry:

host github.com
 HostName github.com
 IdentityFile ~/.ssh/id_rsa
 User git

This way, the git clone, git pull, etc. operations that you run on the staging server will use your SSH key without constantly prompting you to login. Be sure to chmod 600 ~/.ssh/config to give the new file the correct permissions. You should now be able to clone a COVID19 data repository into your home directory on the staging server to do work against. For this example, to use the COVID19_Minimal repo, run:

git clone git@github.com:HopkinsIDD/COVID19_Minimal.git

to get it onto the staging server. By convention, we do runs with the COVIDScenarioPipeline repository nested inside of the data repository, so we then do:

cd COVID19_Minimal
git clone git@github.com:HopkinsIDD/COVIDScenarioPipeline.git

to clone the modeling code itself into a child directory of the data repository.

Getting and Launching the Docker Container

The previous section is only for getting a minimal set of dependencies setup on your staging server. To do an actual run, you will need to download the Docker container that contains the more extensive set of dependencies we need for running the code in the COVIDScenarioPipeline repository. To get the development container on your staging server, please run:

sudo docker pull hopkinsidd/covidscenariopipeline:latest-dev

There are multiple versions of the container published on DockerHub, but latest-dev contains the latest-and-greatest dependencies and can support both Inference and Planning jobs. In order to launch the container and run a job, we need to make our local COVID19_Minimal directory visible to the container's runtime. For Inference jobs, we do this by running:

sudo docker run \
  -v /home/ec2-user/COVID19_Minimal:/home/app/src \
  -v /home/ec2-user/.ssh:/home/app/.ssh \
  -it hopkinsidd/covidscenariopipeline:latest-dev

The -v option to docker run maps a file in the host filesystem (i.e., the path on the left side of the colon) to a file in the container's filesystem. Here, we are mapping the /home/ec2-user/COVID19_Minimal directory on the staging server where we checked out our data repo to the /home/app/src directory in the container (by convention, we run commands inside of the container as a user named app.) We also map our .ssh directory from the host filesystem into the container so that we can interact with Github if need be using our SSH keys. Once the container is launched, we can cd src; ls -ltr to look around and ensure that our directory mapping was successful and we see the data and code files that we are expecting to run with.

Once you are in the src directory, there are a few final steps required to install the R packages and Python modules contained within the COVIDScenarioPipeline repository. First, checkout the correct branch of COVIDScenarioPipeline. Then, assuming that you created a COVIDScenarioPipeline directory within the data repo in the previous step, you should be able to run:

Rscript COVIDScenarioPipeline/local_install.R
(cd COVIDScenarioPipeline/; python setup.py install)

to install the local R packages and then install the Python modules.

Once this step is complete, your machine is properly provisioned to run Planning jobs using the tools you normally use (e.g., make_makefile.R or running simulate.py and hospdeath.R directly, depending on the situation.) Running Inference jobs requires some extra steps that are covered in the next two sections.

Running Inference Jobs

Once the container is setup from the previous step, we are ready to test out and then launch an inference job against a configuration file (I will use the example of config.yml for the rest of this document.) First, I setup and run the build_US_setup.R script against my configuration file to ensure that the mobility data is up to date:

export CENSUS_API_KEY=<your census api key>
cd COVIDScenarioPipeline
git lfs pull
cd ..
Rscript COVIDScenarioPipeline/R/scripts/build_US_setup.R -c config.yml

Next, I kick off a small local run of the full_filter.R script. This serves two purposes: first, we can verify that the configuration file is in good shape and can support a few small iterations of the inference calculations before we kick off hundreds/thousands of jobs via AWS Batch. Second, it downloads the case data that we need for inference calculations to the staging server so that it can be cached locally and used by the batch jobs on AWS- if we do not have a local cache of this data at the start of the run, then every job will try to download the data itself, which will force the upstream server to deny service to the worker jobs, which will cause all of the jobs to fail. My small runs usually look like:

Rscript COVIDScenarioPipeline/R/scripts/full_filter.R -c config.yml -k 2 -n 1 -j 1 -p COVIDScenarioPipeline

This will run two sequential simulations (-k 2) for a single slot (-n 1) using a single CPU core (-j 1), looking for the modeling source code in the COVIDScenarioPipeline directory (-p COVIDScenarioPipeline). (We need to use the command line arguments here to explicitly override the settings of these parameters inside of config.yml since this run is only for local testing.) Assuming that this run succeeds, we are ready to kick off a batch job on the cluster.

The COVIDScenarioPipeline/batch/inference_job.py script will use the contents of the current directory and the values of the config file and any commandline arguments we pass it to launch a run on AWS Batch via the AWS API. To run this script, you need to have access to your AWS access keys so that you can enable access to the API by running aws configure at the command line, which will prompt you to enter your access key, secret, and preferred region, which should always be us-west-2 for jh-covid runs. (You can leave the Default format entry blank by simply hitting Enter.) IMPORTANT REMINDER: (Do not give anyone your access key and secret. If you lose it, deactivate it on the AWS console and create a new one. Keeep it safe.)

The simplest way to launch an inference job is to simply run

./COVIDScenarioPipeline/batch/inference_job.py -c config.yml

This will use the contents of the config file to determine how many slots to run, how many simulations to run for each slot, and how to break those simulations up into blocks of batch jobs that run sequentially. If you need to override any of those settings at the command line, you can run

./COVIDScenarioPipeline/batch/inference_job.py --help

to see the full list of command line arguments the script takes and how to set them.

One particular type of command line argument cannot be specified in the config: arguments to resume a run from a previously submitted run. This takes two arguments based on the previous run:

./COVIDScenarioPipeline/batch/inference_job.py --restart-from-s3-bucket=s3://idd-inference-runs/USA-20210131T170334/ --restart-from-run-id=2021.01.31.17:03:34.

Both the s3 bucket and run id are printed as part of the output for the previous submission. We store that information on a slack channel #csp-production, and suggest other groups find similar storage.

Inference jobs are parallelized by NPI scenarios and hospitalization rates, so if your config file defines more than one top-level scenario or more than one set of hospitalization parameters, the inference_job.py script will kick off a separate batch job for the cross product of scenarios * hospitalizations. The script will announce that it is launching each job and will print out the path on S3 where the final output for the job will be written. You can monitor the progress of the running jobs using either the AWS Batch Dashboard or by running:

./COVIDScenarioPipeline/batch/inference_job_status.py

which will show you the running status of the jobs in each of the queues.

Operating Inference Jobs

By default, the AWS Batch system will usually run around 50% of your desired simultaneously executable jobs concurrently for a given inference run. For example, if you are running 300 slots, Batch will generally run about 150 of those 300 tasks at a given time. If you need to force Batch to run more tasks concurrently, this section provides instructions for how to cajole Batch into running harder.

You can see how many tasks are running within each of the different Batch Compute Environments corresponding to the Batch Job Queues via the Elastic Container Service (ECS) Dashboard. There is a one-to-one correspondence between Job Queues, Compute Environments, and ECS Clusters (the matching ones all end with the same numeric identifier.) You can force Batch to scale up the number of CPUs available for running tasks by selecting the radio button corresponding to the compute environment that you want to scale on the Batch Compute Environment dashboard, clicking Edit, increasing the Desired CPU (and possibly the Minimum CPU, see below), and clicking the Save button. You will be able to see new containers and tasks coming online via the ECS Dashboard after a few minutes.

If you want to force new tasks to come online ASAP, you should consider increasing the Minimum CPU for the Compute Environment as well as the Desired CPU (the Desired CPU is not allowed to be lower than the Minimum CPU, so if you increase the Minimum you must increase the Desired as well to match it.) This will cause Batch to spin new containers up quickly and get them populated with running tasks. There are two downsides to doing this: first, it overrides the allocation algorithm that makes cost/performance tradeoff decisions in favor of spending more money in order to get more tasks running. Second, you must remember to update the Compute Environment towards the end of the job run to set the Minimum CPU to zero again so that the ECS cluster can spin down when the job is finished; if you do not do this, ECS will simply leave the machines up and running, wasting money without doing any actual work. (Note that you should never manually try to lower the value of the Desired CPU setting for the Compute Environment- the Batch system will throw an error if you attempt to do this.)

Technical Reference

Specifying compartmental model

This section describes how to specify the compartmental model of infectious disease transmission.

An example section of a configuration file defining a simple SIR model is below.

compartments:
  infection_stage: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 0.1
    gamma: 0.2
  integration:
     method: rk4
     dt: 1.00

Specifying model compartments (`compartments`)

compartments:
  infection_stage: ["S", "I", "R"]

In this case we can specify compartments as the cross product of multiple states of interest. For example:

 compartments:
   infection_stage: ["S", "I", "R"]
   vaccination_status: ["unvaccinated", "vaccinated"]

Corresponds to 6 compartments, which the code internally converts to this data frame

infection_stage, vaccination_status, compartment_name
S,               unvaccinated,       S_unvaccinated
I,               unvaccinated,       I_unvaccinated
R,               unvaccinated,       R_unvaccinated
S,               vaccinated,         S_vaccinated
I,               vaccinated,         I_vaccinated
R,               vaccinated,         R_vaccinated

In order to more easily describe transitions, we want to be able to refer to a compartment by its components, but then use it by its compartment name.

 compartments:
   infection_stage: ["S", "I"]
   age_group: ["child", "adult"]
   vaccination_status: ["unvaccinated", "1dose", "2dose"]

corresponding to 12 compartments, 4 of which are unnecessary to the model

infection_stage, age_group, vaccination_status, compartment_name
S,		 child,	    unvaccinated,	S_child_unvaccinated	
I,		 child,	    unvaccinated,	I_child_unvaccinated
S,		 adult,	    unvaccinated,	S_adult_unvaccinated
I,		 adult,	    unvaccinated,	I_adult_unvaccinated
S,		 child,	    1dose,		S_child_1dose
I,		 child,	    1dose,		I_child_1dose
S,		 adult,     1dose,		S_adult_1dose
I,		 adult,     1dose,		I_adult_1dose
S,		 child,     2dose,		S_child_2dose	
I,		 child,     2dose,		I_child_2dose
S,		 adult,	    2dose,		S_adult_2dose
I,		 adult,	    2dose,		I_adult_2dose

Or, it could be specified with the less concise notation

compartments:
   overall_state: ["S_child", "I_child", "S_adult_unvaccinated", "I_adult_unvaccinated", "S_adult_1dose", "I_adult_1dose", "S_adult_2dose", "I_adult_2dose"]

which does not result in any unnecessary compartments being included.

Notation must be consistent between these sections.

Specifying compartmental model transitions (`seir::transitions`)

A transition has 5 pieces of associated information that a user can specify:

source
destination
rate
proportional_to
proportion_exponent

For more details on the mathematical forms possible for transitions in our models, read the Model Description section.

We will focus on describing the first transition of this model, the rate at which unvaccinated individuals move from the susceptible to infected state.

Specifying a single transition

Source

[S,unvaccinated]

which corresponds to the compartment S_unvaccinated

Destination

[I,unvaccinated]

which corresponds to the compartment I_unvaccinated

Rate

instead, we could describe the rate using a parameter beta, which can be given a numeric value later:

beta

The interpretation and unit of the rate constant depend on the model details, as the rate may potentially also be per number (or proportion) of individuals in other compartments (see below).

Proportional to

[[[S,unvaccinated]], [[I,unvaccinated], [I, vaccinated]]]

To understand this term, consider the compartments written out as strings

[[S_unvaccinated], [I_unvaccinated, I_vaccinated]]

and then sum the terms in each group

[S_unvaccinated, I_unvaccinated + I_vaccinated]

For transitions that occur at a constant per-capita rate (ie, E -> I at rate $\gamma$ in an SEIR model), it is possible to simply write proportional_to: ["source"].

Proportion exponent

[1, 0.9]

or a power parameter alpha, which can be given a numeric value later:

[1, alpha]

Summary

Putting it all together, the model transition is specified as

source: [S, unvaccinated]
destination: [I, unvaccinated]
proportional_to: [[[S,unvaccinated]], [[I,unvaccinated], [I,vaccinated]]]
rate: [5]
proportion_exponent: [1, 0.9]

would correspond to the following model if expressed as an ordinary differential equation

\frac{\delta \text{S}_\text{unvaccinated}}{\delta t} = - \beta \text{S}_\text{unvaccinated}^1 (\text{I}_\text{unvaccinated}+\text{I}_\text{vaccinated})^{\alpha}

\frac{\delta \text{I}_\text{unvaccinated}}{\delta t} = \beta \text{S}_\text{unvaccinated}^1 (\text{I}_\text{unvaccinated}+\text{I}_\text{vaccinated})^{\alpha}

with parameter and parameter (we will describe how to use parameter symbols in the transitions and specify their numeric values separately in the section Specifying compartmental model parameters).

Transition globs

Source

We allow one or more arguments to be specified for each compartment. So to specify the transitions out of both susceptible compartments (S_unvaccinated and S_unvaccinated), we would use

[[S], [unvaccinated,vaccinated]]

Destination

[[I], [unvaccinated,vaccinated]]

If instead we wrote:

[[I], [vaccinated,unvaccinated]]

we would have a transition from S_unvaccinated to I_vaccinated and S_vaccinated to I_unvaccinated.

Rate

For example,

rate: [[3], [0.6,0.5]]

This would mean our transition from S_unvaccinated to I_unvaccinated would have a rate of 3 * 0.6 while our transition from S_vaccinated to I_vaccinated would have a rate of 3 * 0.5.

The rate vector should be the same shape as source and destination and in the same relative order.

Proportional to

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I,unvaccinated],[I, vaccinated]]
]

Again, let's unpack what it says. Since the broadcast is over groups, let's split the config back up

into those groups

[
  [S,unvaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]
[
  [S,vaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]

From here, we can say that we are describing two transitions. Both occur proportionally to the same compartments: S_unvaccinated and the total number of infections (I_unvaccinated+I_vaccinated).

If, for example, we want to model a situation where vaccinated susceptibles cannot be infected by unvaccinated individuals, we would instead write:

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
]

Proportion exponent

Similarly to rate and proportional_to, we provide an exponent for each component and every group across the broadcast. So we could for example use:

[[1,1], [0.9,0.8]]

Summary

Putting it all together, the transition glob

seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[3], [0.6,0.5]]
    proportion_exponent: [[1,1], [0.9,0.8]]

is equivalent to the following transitions

seir:
  transitions:
    - source: [S,unvaccinated]
      destination: [I,unvaccinated]
      proportional_to: [[[S,unvaccinated]], [[I,unvaccinated],[I, vaccinated]]]
      proportion_exponent: [1 * 0.9]
      rate: [3*0.6]
    - source: [S,vaccinated]
      destination: [I,vaccinated]
      proportional_to: [[[S,vaccinated]], [[I, vaccinated]]]
      proportion_exponent: [1 * 0.8]
      rate: [3*0.5]

Warning

Specifying compartmental model parameters (`seir::parameters`)

Parameters can take on three types of values:

Fixed values
Value drawn from distributions
Values read from timeseries specified in a data file

Specifying fixed parameter values

seir:
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

The full model section of the config could then read

compartments:
  infection_state: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: [1,1]
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

For the stratified SI model described above, this portion of the config would read

compartments:
  infection_stage: ["S", "I", "R"]
  vaccination_status: ["unvaccinated", "vaccinated"]
  
seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[beta], [theta_u,theta_v]]
    proportion_exponent: [[1,1], [alpha_u,alpha_v]]
  parameters:
    beta: 
      value: 0.1
    theta_u: 
      value: 0.6
    theta_v: 
      value: 0.5
    alpha_u: 
      value: 0.9
    alpha_v: 
      value: 0.8

If there are no parameter values that need to be specified (all rates given numeric values when defining model transitions), the seir::parameters section of the config can be left blank or omitted.

Specifying parameters values from distributions

seir:
  parameters:
    beta: 
      value:
        distribution: fixed
        value: 0.1
    gamma: 
      value:
        distribution: lognorm
        logmean: -1.6
        logsd: 0.2

Details on the possible distributions that are currently available, and how to specify their parameters, is provided in the Distributions section.

Specifying parameter values as timeseries from data files

date,        small_province,    large_province
2022-01-01,  1.5,               1.3
.....
2022-05-01,  0.5,               0.7 
....
2022-12-31,  1.5,               1.3

as a part of a configuration file with the model sections:

compartments:
  infection_stage: ["S", "I", "R"]

seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta*theta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2
    theta:
       timeseries: data/seasonal_transmission.csv

Config item

Required?

Type/Format

Description

value

either value or timeseries is required

numerical, or distribution

This defines the value of the parameter, as described above.

timeseries

either value or timeseries is required

path to a csv file

This defines a timeseries for each day, as above.

stacked_modifier_method

optional

string: sum, product, reduction_product

This option defines the method used when modifiers are applied. The default is product.

rolling_mean_windows

optional

integer

The size of the rolling mean window if a rolling mean is applied.

Specifying model simulation method `(seir::integration)`

Our framework allows for two major methods for implementing compartmental models of disease transmission:

ordinary differential equations, which are completely deterministic, operate in continuous time (consider infinitesimally small timesteps), and allow for arbitrary fractions of the population (i.e., not just discrete individuals) to move between model compartments
discrete-time stochastic process, which tracks discrete individuals and produces random variation in the number of individuals transitioning between states for any given rate, and which allows transitions between states only to occur at discrete time intervals

The mathematics behind each implementation is described in the Model Description section

Config item

Required?

Type/format

Description

method

optional

string: rk4 (default),euler, stochastic

dt

optional

positive real number (default: 2)

The timestep used for the numerical integration or discrete time stochastic update; for rk4 method, this is a reasonable value, but for other options, this should be 0.2 or less.

For example, to simulate a model deterministically using the 4th order Runge-Kutta algorithm for numerical integration with a timestep of 1 day:

seir:
  integration:
     method: rk4
     dt: 1.00

Alternatively, to simulate a model stochastically with a timestep of 0.1 days

seir:
  integration:
     method: stochastic
     dt: 0.1

FlepiMoP Documentation

Home

General description of flepiMoP

Acknowledgments

gempyor: modeling infectious disease dynamics

Modeling infectious disease dynamics

Generalized compartmental infection model

SEIR model

Age groups

Vaccination status

Pathogen strain

Clinical outcomes and observations model

Population structure

Mixing between subpopulations

Initial conditions

Time-dependent interventions

Model Implementation

flepiMoP's configuration file

About configuration files

Example

Notation

Configuration files sections

Global header

subpop_setup section

compartments section

seir section

initial_conditions section

seeding section

outcomes section

interventions section

inference section

Specifying population structure

Overview

Items and options

geodata file and selected option

Example geodata file format

mobility file

Example mobility file format

Examples

Example 1

Specifying compartmental model

Specifying model compartments (compartments)

Specifying compartmental model transitions (seir::transitions)

Specifying a single transition

Source

Destination

Rate

Proportional to

Proportion exponent

Summary

Transition globs

Source

Destination

Rate

Proportional to

Proportion exponent

Summary

Warning

Specifying compartmental model parameters (seir::parameters)

Specifying fixed parameter values

Specifying parameters values from distributions

Specifying parameter values as timeseries from data files

Specifying model simulation method (seir::integration)

Specifying initial conditions

Overview

Specifying model initial conditions

initial_conditions::method

Default

SetInitialConditions

FromFile

SetInitialConditionsFolderDraw, FromFileFolderDraw

Specifying observational model

Thinking about outcomes variables

Specifying outcomes in the configuration file

`source ;

`probability ;

Delay

Duration

Sum

outcomes::settings

`subpop_setup` section

`compartments` section

`seir` section

`initial_conditions` section

`seeding` section

`outcomes` section

`interventions` section

`inference` section

`geodata` file and `selected` option

`mobility` file

Specifying model compartments (`compartments`)

Specifying compartmental model transitions (`seir::transitions`)

Specifying compartmental model parameters (`seir::parameters`)

Specifying model simulation method `(seir::integration)`

`initial_conditions::method`

Thinking about `outcomes` variables

Specifying `outcomes` in the configuration file

`modifiers::scenarios`

`modifiers::settings`

`importation` section (optional)

`importation::param_list`

`report` section

`filtering` section

`filtering` settings

`filtering::statistics`

`filtering::hierarchical_stats_geo`

`filtering::priors`

Modifications to `seeding`

Modifications to `interventions`

`interventions::settings::[setting_name]`

New `outcomes` section

`outcomes::settings::[setting_name]`

New `filtering` section

`filtering` settings

`filtering::statistics`

`filtering::hierarchical_stats_geo`

`filtering::priors`