Overview

The subpop_setup section of the configuration file is where users can input the information required to define a population structure on which to simulate the model. The options allow the user to determine the population size of each subpopulation that makes up the overall population, and to specify the amount of mixing that occurs between each pair of subpopulations.

An example configuration file with the global header and the spatial_setup section is below:

name: test_simulation
data_path: data
model_output_dirname: model_output
start_date: 2020-01-01
end_date: 2020-12-31
nslots: 100

subpop_setup:
  geodata: model_input/geodata.csv
  mobility: model_input/mobility.csv

Items and options

geodata file

• geodata is a .csv with column headers, with at least two columns: subpop and population.

• nodenames is the name of a column in geodata that specifies unique geographical identification strings for each subpopulation.

• selected is the list of selected locations in geodata to be modeled

Example geodata file format

subpop,population
10001,1000
20002,2000

mobility file

The mobility file is a .csv file (it has to contain .csv as extension) with long form comma separated values. Columns have to be named ori, dest, amount, with amount being the average number individuals moving from the origin subpopulation ori to destination subpopulation dest on any given day. Details on the mathematics of this model of contact are explained in the Model Description section. Unassigned relations are assumed to be zero. The location entries in the ori and dest columns should match exactly the subpop column in geodata.csv

Example mobility file format

ori, dest, amount
10001, 20002, 3
20002, 10001, 3

It is also possible, but not recommended to specify the mobility file as a .txt with space-separated values in the shape of a matrix. This matrix is symmetric and of size K x K, with K being the number of rows in geodata. The above example corresponds to

0 3
3 0

Examples

Example 1

To simulate a simple population structure with two subpopulations, a large province with 10,000 individuals and a small province with only 1,000 individuals, where every day 100 residents of the large province travel to the small province and interact with residents there, and 50 residents of the small province visit the large province

subpop_setup:
  geodata: model_input/geodata.csv
  mobility: model_input/mobility.csv

geodata.csv contains the population structure (with columns subpop and population)

subpop,          population
large_province, 10000
small_province, 1000

mobility.csv contains

ori,            dest,           amount
large_province, small_province, 100
small_province, large_province, 50

We want to allow users to work with a wide variety of infectious diseases or, one infectious disease under a wide variety of modeling assumptions. To facilitate this, we allow the user to specify their compartmental model of disease dynamics via the configuration file.

We originally considered asking users to specify each compartment and transition manually. However, we quickly found that this created long, confusing configuration files, and so we created a shorthand to more succinctly specify both compartments and transitions between them. This works especially well for models where individuals are stratified by other properties (like age, vaccination status, etc.) in addition to their infection status.

The model is specified in two separate sections of the configuration file. In the compartments section, users define the possible states individuals can be categorized into. Then in the seir section, users define the possible transitions between states, the values of parameters that govern the rates of these transitions, and the numerical method used to simulate the model.

An example section of a configuration file defining a simple SIR model is below.

compartments:
  infection_stage: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 0.1
    gamma: 0.2
  integration:
     method: rk4
     dt: 1.00

Specifying model compartments (compartments)

The first stage of specifying the model is to define the infection states (variables) that the model will track. These "compartments" are defined first in the compartments section of the config file, before describing the processes that lead to transitions between them. The compartments are defined separately from the rest of the model because they are also used by the seeding section that defines initial conditions and importations.

For simple disease models, the compartments can simply be listed with whatever notation the user chooses. For example, for a simple SIR model, the compartments could be ["S", "I", "R"]. The config also requires that there be a variable name for the property of the individual that these compartments describe, which for example in this case could be infection_stage

compartments:
  infection_stage: ["S", "I", "R"]

Our syntax allows for more complex models to be specified without much additional notation. For example, consider a model of a disease that followed SIR dynamics but for which individuals could receive vaccination, which might change how they experience infection.

In this case we can specify compartments as the cross product of multiple states of interest. For example:

 compartments:
   infection_stage: ["S", "I", "R"]
   vaccination_status: ["unvaccinated", "vaccinated"]

Corresponds to 6 compartments, which the code internally converts to this data frame

infection_stage, vaccination_status, compartment_name
S,               unvaccinated,       S_unvaccinated
I,               unvaccinated,       I_unvaccinated
R,               unvaccinated,       R_unvaccinated
S,               vaccinated,         S_vaccinated
I,               vaccinated,         I_vaccinated
R,               vaccinated,         R_vaccinated

In order to more easily describe transitions, we want to be able to refer to a compartment by its components, but then use it by its compartment name.

If the user wants to specify a model in which some compartments are repeated across states but others are not, there will be pros and cons of how the model is specified. Specifying it using the cross product notation is simpler, less error prone, and makes config files easier to read, and there is no issue with having compartments that have zero individuals in them throughout the model. However, for very large models, extra compartments increase the memory required to conduct the simulation, and so having unnecessary compartments tracked may not be desired.

For example, consider a model of a disease that follows SI dynamics in two separate age groups (children and adults), but for which only adults receive vaccination, with one or two doses of vaccine. With the simplified notation, this model could be specified as:

 compartments:
   infection_stage: ["S", "I"]
   age_group: ["child", "adult"]
   vaccination_status: ["unvaccinated", "1dose", "2dose"]

corresponding to 12 compartments, 4 of which are unnecessary to the model

infection_stage, age_group, vaccination_status, compartment_name
S,		 child,	    unvaccinated,	S_child_unvaccinated	
I,		 child,	    unvaccinated,	I_child_unvaccinated
S,		 adult,	    unvaccinated,	S_adult_unvaccinated
I,		 adult,	    unvaccinated,	I_adult_unvaccinated
S,		 child,	    1dose,		S_child_1dose
I,		 child,	    1dose,		I_child_1dose
S,		 adult,     1dose,		S_adult_1dose
I,		 adult,     1dose,		I_adult_1dose
S,		 child,     2dose,		S_child_2dose	
I,		 child,     2dose,		I_child_2dose
S,		 adult,	    2dose,		S_adult_2dose
I,		 adult,	    2dose,		I_adult_2dose

Or, it could be specified with the less concise notation

compartments:
   overall_state: ["S_child", "I_child", "S_adult_unvaccinated", "I_adult_unvaccinated", "S_adult_1dose", "I_adult_1dose", "S_adult_2dose", "I_adult_2dose"]

which does not result in any unnecessary compartments being included.

These compartments are referenced in multiple different subsequent sections of the config. In the seeding (LINK TBA) section the user can specify how the initial (or later imported) infections are distributed across compartments; in the seir section the user can specify the form and rate of the transitions between these compartments encoded by the model; in the outcomes section the user can specify how the observed variables are generated from the underlying model states.

Notation must be consistent between these sections.

Specifying compartmental model transitions (seir::transitions)

The way we specify transitions between compartments in the model is a bit more complicated than how the compartments themselves are specified, but allows users to specify complex stratified infectious disease models with minimal code. This makes checking, sharing, and updating models more efficient and less error-prone.

We specify one or more transition globs, each of which corresponds to one or more transitions. Since transition globs are shorthand for collections of transitions, we will first explain how to specify a single transition before discussing transition globs.

A transition has 5 pieces of associated information that a user can specify:

• source

• destination

• rate

• proportional_to

• proportion_exponent

For more details on the mathematical forms possible for transitions in our models, read the Model Description section.

We first consider a simple example of an SI model where individuals may either be vaccinated (v) or unvaccinated (u), but the vaccine does not change the susceptibility to infection nor the infectiousness of infected individuals.

We will focus on describing the first transition of this model, the rate at which unvaccinated individuals move from the susceptible to infected state.

Specifying a single transition

Source

The compartment the transition moves individuals out of (e.g., the source compartment) is an array. For example, to describe a transition that moves unvaccinated susceptible individuals to another state, we would write

[S,unvaccinated]

which corresponds to the compartment S_unvaccinated

Destination

The compartment the transition moves individuals into (e.g. the destination compartment) is an array. For example, to describe a transition that moves individuals into the unvaccinated but infected state, we would write

[I,unvaccinated]

which corresponds to the compartment I_unvaccinated

Rate

The rate constant specifies the probability per time that an individual in the source compartment changes state and moves to the destination compartment. For example, to describe a transition that occurs with rate 5/time, we would write:

5

instead, we could describe the rate using a parameter beta, which can be given a numeric value later:

beta

The interpretation and unit of the rate constant depend on the model details, as the rate may potentially also be per number (or proportion) of individuals in other compartments (see below).

Proportional to

A vector of groups of compartments (each of which is an array) that modify the overall rate of transition between the source and destination compartment. Each separate group of compartments in the vector are first summed, and then all entries of the vector are multiplied to get the rate modifier. For example, to specify that the transition rate depends on the product of the number of unvaccinated susceptible individuals and the total infected individuals (vaccinated and unvaccinated), we would write:

[[[S,unvaccinated]], [[I,unvaccinated], [I, vaccinated]]]

To understand this term, consider the compartments written out as strings

[[S_unvaccinated], [I_unvaccinated, I_vaccinated]]

and then sum the terms in each group

[S_unvaccinated, I_unvaccinated + I_vaccinated]

From here, we can say that the transition we are describing is proportional to S_unvaccinated and I_unvaccinated + I_vaccinated, i.e., the rate depends on the product S_unvaccinated * (I_unvaccinated + I_vaccinated).

Proportion exponent

This is an exponent modifying each group of compartments that contribute to the rate. It is equivalent to the "order" term in chemical kinetics. For example, if the reaction rate for the model above depends linearly on the number of unvaccinated susceptible individuals but on the total infected individuals sub-linearly, for example to a power 0.9, we would write:

[1, 0.9]

or a power parameter alpha, which can be given a numeric value later:

[1, alpha]

The (top level) length of the proportion_exponent vector must be the same as the (top level) length of the proportional_to vector, even if the desire of the user is to have the same exponent for all terms being multiplied together to get the rate.

Summary

Putting it all together, the model transition is specified as

source: [S, unvaccinated]
destination: [I, unvaccinated]
proportional_to: [[[S,unvaccinated]], [[I,unvaccinated], [I,vaccinated]]]
rate: [5]
proportion_exponent: [1, 0.9]

would correspond to the following model if expressed as an ordinary differential equation

with parameter and parameter (we will describe how to use parameter symbols in the transitions and specify their numeric values separately in the section Specifying compartmental model parameters).

Transition globs

We now explain a shorthand we have developed for specifying multiple transitions that have similar forms all at once, via transition globs. The basic idea is that for each component of the single transitions described above where a term corresponded to a single model compartment, we can instead specify one or more compartment. Similarly, multiple rate values can be specified at once, for each involved compartment. From one transition glob, multiple individual transitions are created, by broadcasting across the specified compartments.

For transition globs, any time you could specify multiple arguments as a list, you may instead specify one argument as a non-list, which will be used for every broadcast. So [1,1,1] is equivalent to 1 if the dimension of that broadcast is 3.

We continue with the same SI model example, where individuals are stratified by vaccination status, but expand it to allow infection to occur at different rates in vaccinated and unvaccinated individuals:

Source

We allow one or more arguments to be specified for each compartment. So to specify the transitions out of both susceptible compartments (S_unvaccinated and S_unvaccinated), we would use

[[S], [unvaccinated,vaccinated]]

Destination

The destination variable should be the same shape as the source, and in the same relative order. So to specify a transition from S_unvaccinated to I_unvaccinated and S_vaccinated to I_vaccinated, we would write the destination as:

[[I], [unvaccinated,vaccinated]]

If instead we wrote:

[[I], [vaccinated,unvaccinated]]

we would have a transition from S_unvaccinated to I_vaccinated and S_vaccinated to I_unvaccinated.

Rate

The rate vector allows users to specify the rate constant for all the source -> destination transitions that are defined in a shorthand way, by instead specifying how the rate is altered depending on the compartment type. For example, the rate of transmission between a susceptible (S) and an infected (I) individual may vary depending on whether the susceptible individual is vaccinated or not and whether the infected individual is vaccinated or not. The overall rate constant is constructed by multiplying together or "broadcasting" all the compartment type-specific terms that are relevant to a given compartment.

For example,

rate: [[3], [0.6,0.5]]

This would mean our transition from S_unvaccinated to I_unvaccinated would have a rate of 3 * 0.6 while our transition from S_vaccinated to I_vaccinated would have a rate of 3 * 0.5.

The rate vector should be the same shape as source and destination and in the same relative order.

Note that if the desire is to make a model where the difference in the rate constants varies in a more complicated than multiplicative way between different compartment types, it would be better to specify separate transitions for each compartment type instead of using this shorthand.

Proportional to

The broadcasting here is a bit more complicated. In other cases, each broadcast is over a single component. However, in this case, we have a broadcast over a group of components. We allow a different group to be chosen for each broadcast.

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I,unvaccinated],[I, vaccinated]]
]

Again, let's unpack what it says. Since the broadcast is over groups, let's split the config back up

into those groups

[
  [S,unvaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]
[
  [S,vaccinated],
  [[I,unvaccinated],[I, vaccinated]]
]

From here, we can say that we are describing two transitions. Both occur proportionally to the same compartments: S_unvaccinated and the total number of infections (I_unvaccinated+I_vaccinated).

If, for example, we want to model a situation where vaccinated susceptibles cannot be infected by unvaccinated individuals, we would instead write:

[
  [[S,unvaccinated], [S,vaccinated]],
  [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
]

Proportion exponent

Similarly to rate and proportional_to, we provide an exponent for each component and every group across the broadcast. So we could for example use:

[[1,1], [0.9,0.8]]

The (top level) length of the proportion_exponent vector must be the same as the (top level) length of the proportional_to vector, even if the desire of the user is to have the same exponent for all terms being multiplied together to get the rate. Within each vector entry, the arrays must have the same length as the source and destination vectors.

Summary

Putting it all together, the transition glob

seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[3], [0.6,0.5]]
    proportion_exponent: [[1,1], [0.9,0.8]]

is equivalent to the following transitions

seir:
  transitions:
    - source: [S,unvaccinated]
      destination: [I,unvaccinated]
      proportional_to: [[[S,unvaccinated]], [[I,unvaccinated],[I, vaccinated]]]
      proportion_exponent: [1 * 0.9]
      rate: [3*0.6]
    - source: [S,vaccinated]
      destination: [I,vaccinated]
      proportional_to: [[[S,vaccinated]], [[I, vaccinated]]]
      proportion_exponent: [1 * 0.8]
      rate: [3*0.5]

Warning

We warn the user that with this shorthand, it is possible to specify large models with few lines of code in the configuration file. The more compartments and transitions you specify, the longer the model will take to run, and the more memory it will require.

Specifying compartmental model parameters (seir::parameters)

When the transitions of the compartmental model are specified as described above, they can either be entered as numeric values (e.g., 0.1) or as strings which can be assigned numeric values later (e.g., beta). We recommend the latter method for all but the simplest models, since parameters may recur in multiple transitions and so that parameter values may be edited without risk of editing the model structure itself. It also improves readability of the configuration files.

Parameters can take on three types of values:

• Fixed values

• Value drawn from distributions

• Values read from timeseries specified in a data file

Specifying fixed parameter values

seir:
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

The full model section of the config could then read

compartments:
  infection_state: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: [1,1]
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2

For the stratified SI model described above, this portion of the config would read

compartments:
  infection_stage: ["S", "I", "R"]
  vaccination_status: ["unvaccinated", "vaccinated"]
  
seir:
  transitions:
    source: [[S],[unvaccinated,vaccinated]]
    destination: [[I],[unvaccinated,vaccinated]]
    proportional_to: [
                       [[S,unvaccinated], [S,vaccinated]],
                       [[I,unvaccinated],[I, vaccinated]], [[I, vaccinated]]
                     ]
    rate: [[beta], [theta_u,theta_v]]
    proportion_exponent: [[1,1], [alpha_u,alpha_v]]
  parameters:
    beta: 
      value: 0.1
    theta_u: 
      value: 0.6
    theta_v: 
      value: 0.5
    alpha_u: 
      value: 0.9
    alpha_v: 
      value: 0.8

If there are no parameter values that need to be specified (all rates given numeric values when defining model transitions), the seir::parameters section of the config can be left blank or omitted.

Specifying parameters values from distributions

seir:
  parameters:
    beta: 
      value:
        distribution: fixed
        value: 0.1
    gamma: 
      value:
        distribution: lognorm
        logmean: -1.6
        logsd: 0.2

Details on the possible distributions that are currently available, and how to specify their parameters, is provided in the Distributions section.

Note that understanding when a new parameter values from this distribution is drawn becomes more complicated when the model is run in Inference mode. In Inference mode, we distinguish model runs as occurring in different "slots" – i.e., completely independent model instances that could be run on different processing cores in a parallel computing environment – and different "iterations" of the model that occur sequentially when the model is being fit to data and update fitted parameters each time based on the fit quality found in the previous iteration. A new parameter values is only drawn from the above distribution once per slot. Within a slot, at each iteration during an inference run, the parameter is only changed if it is being fit and the inference algorithm decides to perturb it to test a possible improved fit. Otherwise, it would maintain the same value no matter how many times the model was run within a slot.

Specifying parameter values as timeseries from data files

Sometimes, we want to be able to specify model parameters that have different values at different timepoints. For example, the relative transmissibility may vary throughout the year based on the weather conditions, or the rate at which individuals are vaccinated may vary as vaccine programs are rolled out. One way to do this is to instead specify the parameter values as a timeseries.

This can be done by providing a data file in .csv or .parquet format that has a list of values of the parameter for a corresponding timepoint and subpopulation name. One column should be date, which should have an entry for every calendar day of the simulation, with the first and last date corresponding to the start_date and end_date for the simulation specified in the header of the config. There should be another column for each subpopulation, where the column name is the subpop name used in other files and the values are the desired parameter values for that subpopulation for the corresponding day. If any day or subpopulation is missing, an error will occur. However, if you want all subpopulations to have the same parameter value for every day, then only a single column in addition to date is needed, which can have any name, and will be applied to every subpop ;

For example, for an SIR model with a simple two-province population structure where the relative transmissibility peaks on January 1 then decreases linearly to a minimal value on June 1 then increases linearly again, but varies more in the small province than the large province, the theta parameter could be constructed from the file seasonal_transmission_2pop.csv with contents including

date,        small_province,    large_province
2022-01-01,  1.5,               1.3
.....
2022-05-01,  0.5,               0.7 
....
2022-12-31,  1.5,               1.3

as a part of a configuration file with the model sections:

compartments:
  infection_stage: ["S", "I", "R"]

seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta*theta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 
      value: 0.1
    gamma: 
      value: 0.2
    theta:
       timeseries: data/seasonal_transmission.csv

Note that there is an alternative way to specify time dependence in parameter values that is described in the Specifying time-varying parameter modifications section. That method allows the user to define intervention parameters that apply specific additive or multiplicative shifts to other parameter values for a defined time interval. Interventions are useful if the parameter doesn't vary frequently and if the values of the shift is unknown and it is desired to either sample over uncertainty in it or try to estimate its value by fitting the model to data. If the parameter varies frequently and its value or relative value over time is known, specifying it as a timeseries is more efficient.

Compartmental model parameters can have an additional attribute beyond value or timeseries, which is called stacked_modifier_method. This value is explained in the section on coding time-dependent parameter modifications (also known as "modifiers") as it determines what happens when two different modifiers act on the same parameter at the same time (are they combined additively or multiplicatively?) ;

Specifying model simulation method (seir::integration)

A compartmental model defined using the notation in the previous sections describes rules for classifying individuals in the population based on infection state dynamically, but does not uniquely specify the mathematical framework that should be used to simulate the model.

Our framework allows for two major methods for implementing compartmental models of disease transmission:

• ordinary differential equations, which are completely deterministic, operate in continuous time (consider infinitesimally small timesteps), and allow for arbitrary fractions of the population (i.e., not just discrete individuals) to move between model compartments

• discrete-time stochastic process, which tracks discrete individuals and produces random variation in the number of individuals transitioning between states for any given rate, and which allows transitions between states only to occur at discrete time intervals

The mathematics behind each implementation is described in the Model Description section

For example, to simulate a model deterministically using the 4th order Runge-Kutta algorithm for numerical integration with a timestep of 1 day:

seir:
  integration:
     method: rk4
     dt: 1.00

Alternatively, to simulate a model stochastically with a timestep of 0.1 days

seir:
  integration:
     method: stochastic
     dt: 0.1

For any method, the results of the model will be more accurate when the timestep is smaller (i.e., output will more precisely match the mathematics of the model description and be invariant to the choice of timestep). However, the computing time required to simulate the model for a certain time range of interest increases with the number of timesteps required (i.e., with smaller timesteps). In our experience, the 4th order Runge-Kutta algorithm (for details see Advanced section) is a very accurate method of numerically integrating such models and can handle timesteps as large as roughly a day for models with the maximum per capita transition rates in this same order of magnitude. However, the discrete time stochastic model or the legacy method for integrating the model in deterministic mode require smaller timesteps to be accurate (around 0.1 for COVID-19-like dynamics in our experience.

Modifiers are a powerful feature in flepiMoP to enable users to modify any of the parameters being specified in the model during particular time periods. They can be used, for example, to mirror public health control interventions, like non-pharmaceutical interventions (NPIs) or increased access to diagnosis or care, or annual seasonal variations in disease parameters. Modifiers can act on any of the transmission model parameters or observation model parameters ;

In the seir_modifiers and outcome_modifiers sections of the configuration file the user can specify several possible types of modifiers which will then be implemented in the model. Each modifier changes a parameter during one or multiple time periods and for one or multiple specified subpopulations.

We currently support the following intervention types. Each of these is described in detail below:

• "SinglePeriodModifier" – Modifies a parameter during a single time period

• "MultiPeriodModifier" – Modifies a parameter by the same amount during a multiple time periods

• "ModifierModifier" – Modifies another intervention during a single time period

• "StackedModifier" – Combines two or more interventions additively or multiplicatively, and is used to be able to turn on and off groups of interventions easily for different runs ;

Within flepiMoP, modifiers can be run as "scenarios". With scenarios, we can use the same configuration file to run multiple versions of the model where only the modifiers applied differ.

The modifiers section contains two sub-sections: modifiers::scenarios, which lists the name of the modifiers that will run in each separate scenario, and modifiers::modifiers, where the details of each modifier are specified (e.g., the parameter it acts on, the time it is active, and the subpopulation it is applied to). An example is outlined below

seir_modifiers:
  scenarios:
    -NameOfIntervention1
    -NameofIntervention2
  modifiers:
    NameOfIntervention1:
      ...
    NameOfIntervention2:
      ...

In this example, each scenario runs a single intervention, but more complicated examples are possible. ;

The major benefit of specifying both "scenarios" and "modifiers" is that the user can use "StackedModifier" option to combine other modifiers in different ways, and then run either the individual or combined modifiers as scenarios. This way, each scenario may consist of one or more individual parameter modifications, and each modification may be part of multiple scenarios. This provides a shorthand to quickly consider multiple different versions of a model that have different combinations of parameter modifications occurring. For example, during an outbreak we could evaluate the impact of school closures, case isolation, and masking, or any one or two of these three measures. An example of a configuration file combining modifiers to create new scenarios is given below

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs
  modifiers:
    SchoolClosures:
      method:SinglePeriodModifier
      ...
    CaseIsolation:
      method:SinglePeriodModifier
      ...
    Masking:
      method:SinglePeriodModifier
      ....
    AllNPIs
      method: StackedModifier
      modifiers: ["SchoolClosures","CaseIsolation","Masking"]

Example

[Give a configuration file that tries to use all the possible option available. Based on simple SIR model with parameters beta and gamma in 2 subpopulations. Maybe a SinglePeriodModifier on beta for a lockdown and gamma for isolation, one having a fixed value and one from a distribution, MultiPeriodModifier for school year in different places, ModifierModifer for ..., StackedModifier for .... ]

modifiers::scenarios

A optional list consisting of a subset of the modifiers that are described in modifiers::settings, each of which will be run as a separate scenario. For example

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs

or

outcome_modifiers
  scenarios:
    -BaselineTesting
    -TestShortage

modifiers::settings

A formatted list consisting of the description of each modifier, including its name, the parameter it acts on, the duration and amount of the change to that parameter, and the subset of subpopulations in which the parameter modification takes place. The list items are summarized in the table below and detailed in the sections below.

SinglePeriodModifier

SinglePeriodModifier interventions enable the user to specify a multiplicative reduction to a parameter of interest. It take a parameter, and reduces it's value by value (new = (1-value) * old) for the subpopulations listed insubpop during the time interval [period_start_date, period_end_date]

For example, if you would like to create an SEIR modifier called lockdown that reduces transmission by 70% in the state of California and the District of Columbia between two dates, you could specify this with a SinglePeriodModifier, as in the example below

Example

seir_modifiers:
  modifiers:
    lockdown: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: ['06000', '11000']
      value: 0.7

Or to create an outcome variable modifier called enhanced_testing during which the case detection rate double ;

outcome_modifiers:
  modifiers:
    enhanced_testing: 
      method: SinglePeriodModifier
      parameter: incidC::probability
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: ['06000', '11000']
      value: -1.0

Configuration options

method: SinglePeriodModifier

parameter: The name of the parameter that will be modified. This could be a parameter defined for the transmission model in seir::parameters or for the observational model in outcomes. If the parameter is used in multiple transitions in the model then all those transitions will be modified by this amount ;

period_start_date: The date when the modification starts, in YYYY-MM-DD format. The modification will only reduce the value of the parameter after (inclusive of) this date.

period_end_date: The date when the modification ends, in YYYY-MM-DD format. The modification will only reduce the value of the parameter before (inclusive of) this date.

subpop:A list of subpopulation names/ids in which the specified modification will be applied. This can be a single subpop, a list, or the word "all" (specifying the modification applies to all existing subpopulations in the model). The modification will do nothing for any subpopulations not listed here.

value:The fractional reduction of the parameter during the time period the modification is active. This can be a scalar number, or a distribution using the notation described in the Distributions section. The new parameter value will be

new_parameter_value = old_parameter_value * (1 - value)

subpop_groups: An optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. See subpop_groups section below for more details ;

MultiPeriodModifier

MultiPeriodModifier interventions enable the user to specify a multiplicative reduction to the parameter of interest by value (new = (1-value) * old) for the subpopulations listed in subpop during multiple different time intervals each defined by a start_date and end_date.

For example, if you would like to describe the impact that transmission in schools has on overall disease spread, you could create a modifier that increases transmission by 30% during the dates that K-12 schools are in session in different regions (e.g., Massachusetts and Florida):

Example

school_year:
  method: MultiPeriodModifier
  parameter: beta
  groups:
    - subpop: ["25000"] 
      periods:
        - start_date: 2021-09-09
          end_date: 2021-12-23
        - start_date: 2022-01-04
          end_date: 2022-06-22
    - subpop: ["12000"]
      periods:
        - start_date: 2021-08-10
          end_date: 2021-12-17
        - start_date: 2022-01-04
          end_date: 2022-05-27
  value: -0.3

Configuration options

method: MultiPeriodModifier

parameter: The name of the parameter that will be modified. This could be a parameter defined for the transmission model in seir::parameters or for the observational model in outcomes. If the parameter is used in multiple transitions in the model then all those transitions will be modified by this amount ;

groups: A list of subpopulations (subpops) or groups of them, and time periods the modification will be active in each of them

• groups:subpop A list of subpopulation names/ids in which the specified modification will be applied. This can be a single subpop, a list, or the word "all" (specifying the modification applies to all existing subpopulations in the model). The modification will do nothing for any subpopulations not listed here.

• groups: periods A list of time periods, each defined by a start and end date, when the modification will be applied

  • groups:periods:start_date The date when the modification starts, in YYYY-MM-DD format. The modification will only reduce the value of the parameter after (inclusive of) this date.

  • groups:periods:end_date The date when the modification ends, in YYYY-MM-DD format. The modification will only reduce the value of the parameter before (inclusive of) this date.

value:The fractional reduction of the parameter during the time period the modification is active. This can be a scalar number, or a distribution using the notation described in the Distributions section. The new parameter value will be

new_parameter_value = old_parameter_value * (1 - value)

subpop_groups: An optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. See subpop_groups section below for more details ;

ModifierModifier

ModifierModifier interventions allow the user to specify an intervention that acts to modify the value of another intervention, as opposed to modifying a baseline parameter value. The intervention multiplicatively reduces the modifier of interest by value (new = (1-value) * old) for the subpopulations listed in subpop during the time interval [period_start_date, period_end_date].

Example

For example, ModifierModifier could be used to describe a social distancing policy that is in effect between two dates and reduces transmission by 60% if followed by the whole population, but part way through this period, adherence to the policy drops to only 50% of in one of the subpopulations population:

seir_modifiers:
  modifiers:
    social_distancing: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-06-30
      subpop: ['all']
      value: 0.6
    fatigue: 
      method: ModifierModifier
      baseline_scenario: social_distancing
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['large_province']
      value: 0.5

Note that this configuration is identical to the following alternative specification

seir_modifiers:
  modifiers:
    social_distancing_initial: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-04-31
      subpop: ['all']
      value: 0.6
    social_distancing_fatigue_sp: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['small_province']
      value: 0.6
    social_distancing_fatigue_lp: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-05-01
      period_end_date: 2020-06-30
      subpop: ['large_province']
      value: 0.3

However, there are situations when the ModiferModifier notation is more convenient, especially when doing parameter fitting. ;

Configuration options

method: ModifierModifier

baseline_modifier: The name of the original parameter modification which will be further modified.

parameter: The name of the parameter in the baseline_scenario that will be modified ;

period_start_date: The date when the intervention modifier starts, in YYYY-MM-DD format. The intervention modifier will only reduce the value of the other intervention after (inclusive of) this date.

period_end_date: The date when the intervention modifier ends, in YYYY-MM-DD format. The intervention modifier will only reduce the value of the other intervention before (inclusive of) this date.

subpop:A list of subpopulation names/ids in which the specified intervention modifier will be applied. This can be a single subpop, a list, or the word "all" (specifying the interventions applies to all existing subpopulations in the model). The intervention will do nothing for any subpopulations not listed here.

value:The fractional reduction of the baseline intervention during the time period the modifier intervention is active. This can be a scalar number, or a distribution using the notation described in the Distributions section. The new parameter value will be

new_intervention_value = old_intervention_value * (1 - value)

and so the value of the underlying parameter that was modified by the baseline intervention will be

new_parameter_value = original_parameter_value * (1 - baseline_intervention_value * (1 - value) )

subpop_groups: An optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. See subpop_groups section below for more details ;

StackedModifier

Combine two or more modifiers into a scenario, so that they can easily be singled out to be run together without the other modifiers. If multiply modifiers act during the same time period in the same subpopulation, their effects are combined multiplicatively. Modifiers of different types (i.e. SinglePeriodModifier, MultiPeriodModifier, ModifierModifier, other StackedModifiers) can be combined ;

Examples

seir_modifiers:
  scenarios:
    -SchoolClosures
    -AllNPIs
  modifiers:
    SchoolClosures:
      method:SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.7
    CaseIsolation:
      method:SinglePeriodModifier
      parameter: gamma
      period_start_date: 2020-04-01
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -1.0
    Masking:
      method:SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-04-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.5
    AllNPIs
      method: StackedModifier
      modifiers: ["SchoolClosures","CaseIsolation","Masking"]

or

outcome_modifiers:
  scenarios:
    - ReducedTesting
    - AllDelays
  modifiers:
    DelayedTesting
      method:SinglePeriodModifier
      parameter: incidC::probability
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: 0.5
    DelayedHosp
      method:SinglePeriodModifier
      parameter: incidD::delay
      period_start_date: 2020-04-01
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -1.0
    LongerHospStay
      method:SinglePeriodModifier
      parameter: incidH::duration
      period_start_date: 2020-04-15
      period_end_date: 2020-05-01
      subpop: 'all'
      value: -0.5

Configuration options

method: StackedModifier

modifiers: A list of names of the other modifiers (specified above) that will be combined to create the new modifier (which we typically refer to as a "scenario")

modifiers::modifiers::groups

subpop_groups: For any of the modifier types, subpop_groups is an optional list of lists specifying which subsets of subpopulations in subpop should share parameter values; when parameters are drawn from a distribution or fit to data. All other subpopulations not listed will have unique intervention values unlinked to other areas. If the value is 'all', then all subpopulations will be assumed to have the same modifier value. When the subpop_groups option is not specified, all subpopulations will be assumed to have unique values of the modifier ;

For example, for a model of disease spread in Canada where we want to specify that the (to be varied) value of a modification to the transmission rate should be the same in all the Atlantic provinces (Nova Scotia, Newfoundland, Prince Edward Island, and New Brunswick), the same in all the prairie provinces (Manitoba, Saskatchewan, Alberta), the same in the three territories (Nunavut, Northwest Territories, and Yukon), and yet take unique values in Ontario, Quebec, and British Columbia, we could write

seir_modifiers:
  modifiers:
    lockdown: 
      method: SinglePeriodModifier
      parameter: beta
      period_start_date: 2020-03-15
      period_end_date: 2020-05-01
      subpop: 'all'
      subpop_groups: [['NS','NB','PE','NF'],['MB','SK','AB'],['NV','NW','YK']]
      value: 
        distribution: uniform
        low: 0.3
        high: 0.7
        

About configuration files

flepiMop is set up so that all parameters and other options for running the pipeline can be specified in a single "configuration" file (aka "config"). Users do not need to edit any other code files, or even be aware of their contents, to create and run complex model scenarios. Configuration files also provide a convenient record of model options and promote reproducibility of model results.

We use the YAML language syntax to write config files, which are typically named something like config.yml. The file has simple plain text contents and follows a tabbed outline structure. When config files are read by the model code, a data structure encoding the model options is created.

Comments can be added to the config file by starting with the hash key (#) then a space. Comments can start anywhere on a line and continue until the end, but if they run over to a new line, a new # must be used at the start of the new line.

Example

(give a simple configuration for a toy model with two subpopulations, SEIR, single "cases" outcome, single seeded infection, single NPI that starts after some time? this page is currently under development, please see our example repo _for some simple configurations) ;

When referring to config items (individual parameters), we use their full position in the outline. For example, in the sample config file above, we denote

subpop_setup:
  ...
  geodata: minimal

as subpop_setup::geodata having a value of minimal

Notation

Parameters and other options specified in the configuration files can take on a variety of types of values, using the following notations:

• dates are specified as [year]-[month]-[day]. (e.g., 2020-01-31)

• boolean values are either "TRUE" or "FALSE"

• files names are strings

• probability is a float between 0 and 1

• distribution is a probability distribution from which a random value for the parameter is drawn each time a new simulation is run (or chain, if doing inference). See here for the require schema.

Configuration files sections

Global header

These global configuration options typically sit at the top of the configuration file.

For example, for a configuration file to simulate the spread of COVID-19 in the US during 2020 and compare to data from March 1 onwards, with 1000 independent simulations, the header of the config might read:

name: USA_covid19_2020
model_output_dirname: model_output
start_date: 2020-01-01
end_date: 2020-12-31
start_date_groundtruth: 2020-03-01
end_date_groundtruth: 2020-12-31
nslots: 1000

subpop_setup section

This section specifies the population structure on which the model will be simulated, including the names and sizes of each subpopulation and the connectivity between them. More details here.

compartments section

This section is where users can specify the variables (infection states) that will be tracked in the infectious disease transmission model. More details can be found here. The other details of the model are specified in the seir section, including transitions between these compartments (seir::transitions), the names of the parameters governing the transitions (seir::parameters), and the numerical method used to simulate the equations over time (seir::integration). The initial conditions of the model can be specified in the initial_conditions section, and any other inputs into the model from external populations or instantaneous transitions between states that occur at later times can be specified in the seeding section. ;

seir section

This section is where users can specify the details of the infectious disease transmission model they wish to simulate (e.g., SEIR). This model describes the allowed transitions (seir::transitions) between the compartments that were specified in the compartments section, the values of the parameters involved in these transitions (seir::parameters), and the numerical method used to simulate the equations over time (seir::integration). More details here. The initial conditions of the model can be specified in the separate initial_conditions section, and any other inputs into the model from external populations or instantaneous transitions between states that occur at later times can be specified in the seeding section. ;

initial_conditions section

This section is used to specify the initial conditions of the model, which define how individuals are distributed between the model compartments at the time the model simulation begins. Importantly, the initial conditions specify the time and location where infection is first introduced. If this section is omitted, default values are used. If users want to add infections to the population at later times, or add or remove individuals from compartments separately from the model rules, they can do so via the related seeding section. More details here ;

seeding section

This section is used to specify how individuals are instantaneously "seeded" from one compartment to another, where they then continue to be governed by the model equations. For example, this seeding could be used to represent importations of infected individuals from an outside population, mutation events that create new strains, or vaccinations that alter disease susceptibility. Seeding events can occur at any time in the simulation. The seeding section specifies the numeric values added to or removed from any compartment of the model. More details here ;

outcomes section

This section is where users can define new variables representing the observed quantities and how they are related to the underlying state variables in the model (e.g., the fraction of infections that are detected as cases). More details here ;

interventions section

This section is where users can specify time-varying changes to parameters governing either the infectious disease model or the observational model. More details here ;

inference section

This section is where users can specify the details of how the model is fit to data, including what data streams they will be included and which outcome variables they represent and the likelihood functions describing the probability of the data given the model. More details here. ;

Thinking about outcomes variables

Our pipeline allows users to encode state variables describing the infection status of individuals in the population in two different ways. The first way is via the state variables and transitions of the compartmental model of disease transmission, which are specified in the compartments and seir sections of the config. This model should include all variables that influence the natural course of the epidemic (i.e., all variables that feed back into the model by influencing the rate of change of other variables). For example, the number of infected individuals influences the rate at which new infections occur, and the number of immune individuals influences the number of individuals at risk of acquiring infection.

However, these intrinsic model variables may be difficult to observe in the real world and so directly comparing model predictions about the values of these variables to data might not make sense. Instead, the observable outcomes of infection may include only a subset of individuals in any state, and may only be observed with a time delay. Thus, we allow users to define new outcome variables that are functions of the underlying model variables. Commonly used examples include detected cases or hospitalizations ;

Variables should not be included as outcomes if they influence the infection trajectory. The choice of what variables to include in the compartmental disease model vs. the outcomes section may be very model specific. For example, hospitalizations due to infection could be encoded as an outcome variable that is some fraction of infections, but if we believe hospitalized individuals are isolated from the population and don't contribute to onward infection, or that the number of hospitalizations feeds back into the population's perception of risk of infection and influences everyone's contact behavior, this would not be the best choice. Similarly, we could include deaths due to infection as an outcome variable that is also some fraction of infections, but unless death is a very rare outcome of infection and we aren't worried about actually removing deceased individuals from the modeled populations, deaths should be in the compartmental model instead.

The outcomes section is not required in the config. However, there are benefits to including it, even if the only outcome variable is set to be equivalent to one of the infection model variables. If the compartmental model is complicated but you only want to visualize a few output variables, the outcomes output file will be much easier to work with. Outcome variables always occur with some fixed delay from their source infection model variable, which can be more convenient than the exponential distribution underlying the infection model. Outcome variables can be created to automatically sum over multiple compartments of the infection model, removing the need for post-processing code to do this. If the model is being fit to data, then the outcomes section is required, as only outcome variables can be compared to data.

As an example, imagine we are simulating an SIR-style model and want to compare it to real epidemic data in which cases of infection and death from infection are reported. Our model doesn't explicitly include death, but suppose we know that 1% of all infections eventually lead to hospitalization, and that hospitalization occurs on average 1 week after infection. We know that not all infections are reported as cases, and assume that only 50% are detected and are reported 2 days after infection begins. The model and outcomes section of the config for these outcomes, which we call incidC (daily incidence of cases) and incidH (daily incidence of hospital admission) would be

compartments:
  infection_stage: ["S", "I", "R"]
  
seir:
  transitions:
    # infection
    - source: [S]
      destination: [I]
      proportional_to: [[S], [I]]
      rate: [beta]
      proportion_exponent: 1
    # recovery
    - source: [I]
      destination: [R]
      proportional_to: [[I]]
      rate: [gamma]
      proportion_exponent: 1
  parameters:
    beta: 
      value: 0.2
    gamma: 
      value: 0.1

outcomes:
  settings:
    method: delayframe
  outcomes:
    incidC:
      source:
        incidence:
          infection_stage: "I"
      probability: 
        value: 0.5
      delay: 
        value: 2
    incidH:
      source:
        incidence:
          infection_stage: "I"
      probability: 
        value: 0.01
      delay: 
        value: 21 

in the following sections we describe in more detail how this specification works

Specifying outcomes in the configuration file

The outcomes config section consists of a list of defined outcome variables (observables), which are defined by a user-created name (e.g., "incidH"). For each of these outcome variables, the user defines the source compartment(s) in the infectious disease model that they draw from and whether they draw from the incidence (new individuals entering into that compartment) or prevalence (total current individuals in that compartment). Each new outcome variable is always associated with two mandatory parameters ;

• probability of being counted in this outcome variable if in the source compartment

• ;delay between when an individual enters the source compartment and when they are counted in the outcome variable

and one optional parameter

• duration after entering that an individual is counted as part of the outcome variable

The value of the probability, delay, and duration parameters can be a single value or come from distribution ;

Outcome model parameters probability, delay, and distribution can have an additional attribute beyond value called modifier_key. This value is explained in the section on coding time-dependent parameter modifications (also known as "modifiers") as it provides a way to have the same modifier act on multiple different outcomes ;

Example

// Some code

`source ;

Required, unless sum option is used instead. This sub-section describes the compartment(s) in the infectious disease model from which this outcome variable is drawn. Outcome variables can be drawn from the incidence of a variable - meaning that some fraction of new individuals entering the infection model state each day are chosen to contribute to the outcome variable - or from the prevalence, meaning that each day some fraction of individuals currently in the infection state are chosen to contribute to the outcome variable. Note that whatever the source type, the named outcome variable itself is always a measure of incidence ;

To specify which compartment(s) contribute the user must specify the state(s) within each model stratification. For stratifications not mentioned, the outcome will sum over that states in all strata ;

For example, consider a configuration in which the compartmental model was constructed to track infection status stratified by vaccination status and age group. The following code would be used to create an outcome called incidH_child (incidence of hospitalization for children) and incidH_adult (incidence of hospitalization for adults) where some fraction of infected individuals would become hospitalized and we wanted to track separately track pediatric vs adult hospitalizations, but did not care about tracking the vaccination status of hospitalized individuals as in reality it was not tracked by the hospitals ;

 compartments:
   infection_state: ["S", "I", "R"]
   age_group: ["child", "adult"]
   vaccination_status: ["unvaxxed", "vaxxed"]
   
outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    ...
  incidH_adult:
    source:
      incidence:
        infection_state: "I"
        age_group: "adult"
    ...
  incidH_all:
    source:
      incidence:
        infection_state: "I"
    ...

to instead create an outcome variable for cases where on each day of infection there is some probability of testing positive (for example, for the situation of an asymptomatic infection where testing is administered totally randomly), the following code would be used

 compartments:
   infection_state: ["S", "I", "R"]
   age_group: ["child", "adult"]
   vaccination_status: ["unvaxxed", "vaxxed"]
   
outcomes:
  incidC:
    source:
      prevalence:
        infection_state: "I"
    ...

The source of an outcome variable can also be a previous defined outcome variable. For example, t to create a new variable for the number of individuals recruited to be part of a contact tracing program (incidT), which is just some fraction of diagnosed cases ;

outcomes:
  incidC:
    source:
      prevalence:
        infection_state: "I"
    ...
  incidT:
    source: incidC
    ...

`probability ;

Required, unless sum option is used instead. Probability is the fraction of individuals in the source compartment who are counted as part of this outcome variable (if the source is incidence; if the source is prevalence it is the fraction of individuals per day). It must be between 0 and 1 ;

Specifying the probability creates a parameter called outcome_name::probability that can be referred to in the outcome_modifiers section of the config. The value of this parameter can be changed using the probability::intervention_param_name option ;

For example, to track the incidence of hospitalization when 5% of children but only 1% of adults infected require hospitalization, and to create a modifier_key such that both of these rates could be modified by the same amount during some time period using the outcomes_modifier section:

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 
      value: 0.05
      modifier_key: hosp_rate
  incidH_adult:
    source:
      incidence:
        infection_state: "I"
        age_group: "adult"
    probability: 
      value: 0.01
      modifier_key: hosp_rate

To track the incidence of diagnosed cases iterating over uncertainty in the case detection rate (ranging 20% to 30%), and naming this parameter "case_detect_rate"

outcomes:
  incidC:
    source:
      prevalence:
        infection_state: "I"
    probability:
      value:
        distribution: uniform
        low: 
          value: 0.2
        high: 
          value: 0.3
      intervention_param_name: "case_detect_rate"

Each time the model is run a new random value for the probability of case detection will be chosen ;

Delay

Required, unless sum option is used instead. delay is the time delay between when individuals are chosen from the source compartment and when they are counted as part of this outcome variable ;

For example, to track the incidence of hospitalization when 5% of children are hospitalized and hospitalization occurs 7 days after infection:

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 
      value: 0.05
    delay: 
      value: 7

To iterate over uncertainty in the exact delay time, we could include some variation between simulations in the delay time using a normal distribution with standard deviation of 2 (truncating to make sure the delay does not become negative). Note that a delay distribution here does not mean that the delay time varies between individuals - it is identical) ;

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 
      value: 0.05
    delay: 
      value: 
        distribution: truncnorm
        mean: 7
        sd: 2
        a: 0
        b: Inf

Duration

By default, all outcome variables describe incidence (new individuals entering each day). However, they can also track an associated "prevalence" if the user specifies how long individuals will stay classified as the outcome state the outcome variable describes. This is the duration parameter ;

When the duration parameter is set, a new outcome variable is automatically created and named with the name of the original outcome variable + "_curr". This name can be changed using the duration::name option ;

For example, to track the incidence and prevalence of hospitalization when 5% of children are hospitalized, hospitalization occurs 7 days after infection, and the duration of hospitalization is 3 days:

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 
      value: 0.05
    delay: 
      value: 7
    duration: 
      value: 3

which creates the variable "incidH_child_curr" to track all currently hospitalized children. Since it doesn't make sense to call this new outcome variable an incidence, as it is a prevalence, we could instead rename it:

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 
      value: 0.05
    delay: 
      value: 7
    duration: 
      value: 3
      name: "hosp_child_curr"

Sum

Optional. sum is used to create new outcome variables that are sums over other previously defined outcome variables ;

If sum is included, source, probability, delay, and duration will be ignored ;

For example, to track new hospital admissions and current hospitalizations separately for children and adults, as well as for all ages combined

outcomes:
  incidH_child:
    source:
      incidence:
        infection_state: "I"
        age_group: "child"
    probability: 0.05
    delay: 6
    duration: 
      value: 14
      name: "hosp_child_curr"
  incidH_adult:
    source:
      incidence:
        infection_state: "I"
        age_group: "adult"
    probability: 0.01
    delay: 8
    duration:
      value: 7
      name: "hosp_adult_curr"
  incidH_total: 
    sum: ["incidH_child","incidH_adult"]
  hosp_curr_total:   
    sum: ["hosp_child_curr","hosp_adult_curr"]

outcomes::settings

There are other required and optional configuration items for the outcomes section which can be specified under outcomes::settings:

method: delayframe.This is the mathematical method used to create the outcomes variable values from the transmission model variables. Currently, the only model supported is delayframe, which .. ;

param_from_file: Optional, TRUE or FALSE. It is possible to allow any of the outcomes variables to have values that vary across the subpopulations. For example, disease severity rates or diagnosis rates may differ by demographic group. In this case, all the outcome parameter values defined in outcomes::outcomes will represent baseline values, and then you can define a relative change from this baseline for any particular subpopulation using the paths section. If params_from_file: TRUE is specified, then these relative values will be read from the params_subpop_file. Otherwise, if params_from_file: FALSE or is not listed at all, all subpopulations will have the same values for the outcome parameters, defined below ;

param_subpop_file: Required if params_from_file: TRUE. The path to a .csv or .parquet file that contains the relative amount by which a given outcome variable is shifted relative to baseline in each subpopulation. File must contain the following columns:

• subpop: The subpopulation for which the parameter change applies. Must be a subpopulation defined in the geodata file. For example, small_province

• parameter: The outcomes parameter which will be altered for this subpopulation. For example, incidH_child: probability

• value: The amount by which the baseline value will be multiplied, for example, 0.75 or 1.1

Examples

Consider a disease described by an SIR model in a population that is divided into two age groups, adults and children, which experience the disease separately. We are interested in comparing the predictions of the model to real world data, but we know we cannot observe every infected individual. Instead, we have two types of outcomes that are observed.

First, via syndromic surveillance, we have a database that records how many individuals in the population are experiencing symptoms from the disease at any given time. Suppose careful cohort studies have shown that 50% of infected adults and 80% of infected children will develop symptoms, and that symptoms occur in both age groups around 3 days after infection (following a log-normal distribution with log mean X and log standard deviation of Y). The duration that symptoms persist is also a variable, following a ...

Secondly, via laboratory surveillance we have a database of every positive test result for the infection. We assume the test is 100% sensitive and specific. Only individuals with symptoms are tested, and they are always tested exactly 1 day after their symptom onset. We are unsure what portion of symptomatic individuals are seeking out testing, but are interested in considering two extreme scenarios: 95% of symptomatic individuals are tested, or only 75% of individuals are tested.

The configuration file we could use to model this situation includes

// Some code

COVIDScenarioPipeline

/R/scripts

• filter_MC.R

• full_filter.R

• build_US_setup.R

• build_nonUS_setup.R

/R/pkgs

• inference

  • groundtruth.R

  • functions.R

  • filter_MC_runner_funcs.R

• MCMC step size

• Numerical integration step size

• Mobility proportion

Command line inputs

flepiMoP allows some input parameters/options to be specified in the command line at the time of model submission, in addition to or instead of in the configuration file. This can be helpful for users who want to quickly run different versions of the model – typically a different number of simulations or a different intervention scenario from among all those specified in the config – without having to edit or create a new configuration file every time. In addition, some arguments can only be specified via the command line.

In addition to the configuration file and the command line, the inputs described below can also be specified as environmental variables.

In all cases, command line arguments override configuration file entries which override environmental variables. The order of command line arguments does not matter.

Details on how to run the model, including how to add command line arguments or environmental variables, are in the section How to Run.

Command-line only inputs

Command-line versions of configuration file inputs

Example

As an example, consider running the following configuration file

name: sir
setup_name: minimal
start_date: 2020-01-31
end_date: 2020-05-31
data_path: data
nslots: 1

subpop_setup:
  geodata: geodata_sample_1pop.csv
  mobility: mobility_sample_1pop.csv
  popnodes: population
  nodenames: name

seeding:
  method: FromFile
  seeding_file: data/seeding_1pop.csv

compartments:
  infection_stage: ["S", "I", "R"]

seir:
  integration:
    method: stochastic
    dt: 1 / 10
  parameters:
    gamma:
      value:
        distribution: fixed
        value: 1 / 5
    Ro:
      value:
        distribution: uniform
        low: 2
        high: 3
  transitions:
    - source: ["S"]
      destination: ["I"]
      rate: ["Ro * gamma"]
      proportional_to: [["S"],["I"]]
      proportion_exponent: ["1","1"]
    - source: ["I"]
      destination: ["R"]
      rate: ["gamma"]
      proportional_to: ["I"]
      proportion_exponent: ["1"]

interventions:
  scenarios:
    - None
    - Lockdown
  modifiers:
    None:
      method: SinglePeriodModifier
      parameter: r0
      period_start_date: 2020-04-01
      period_end_date: 2020-05-15
      value:
        distribution: fixed
        value: 0
        settings:
    Lockdown:
      method: SinglePeriodModifier
      parameter: r0
      period_start_date: 2020-04-01
      period_end_date: 2020-05-15
      value:
        distribution: fixed
        value: 0.7

To run this model directly in Python (it can alternatively be run from R, for all details see section How to Run), we could use the command line entry

> gempyor-seir -c sir_control.yml

Alternatively, to run 100 simulations using only 4 of the available processors on our computer, but only running the "" scenario with a deterministic model, and to save the files as .csv (since the model is relatively simple), we could call the model using the command line entry

/> gempyor-seir -c sir_control.yml -n 100 -j 4 -npi_scenario None --non_stochastic --write_csv

Environmental variables

TBA

US-specific configuration file options

global: smh_round, setup_name, disease

spatial_setup: census_year, modeled_states, state_level

For US-specific population structures

For creating US-based population structures using the helper script build_US_setup.R which is run before the main model simulation script, the following extra parameters can be specified

Example 2

To simulate an epidemic across all 50 states of the US or a subset of them, users can take advantage of built in machinery to create geodata and mobility files for the US based on the population size and number of daily commuting trips reported in the US Census.

Before running the simulation, the script build_US_setup.R can be run to get the required population data files from online census data and filter out only states/territories of interest for the model. More details are provided in the How to Run section.

This example simulates COVID-19 in the New England states, assuming no transmission from other states, using 2019 census data for the population sizes and a pre-created file for estimated interstate commutes during the 2011-2015 period.

subpop_setup:
  census_year: 2010
  state_level: TRUE
  geodata: geodata_2019_statelevel.csv
  mobility: mobility_2011-2015_statelevel.csv
  modeled_states:
    - CT
    - MA
    - ME
    - NH
    - RI
    - VT
  

geodata.csv contains

USPS	subpop	population
AL	01000	4876250
AK	02000	737068
AZ	04000	7050299
AR	05000	2999370
CA	06000	39283497
.....

mobility_2011-2015_statelevel.csv contains

ori	dest	amount
01000	02000	198
01000	04000	292
01000	05000	570
01000	06000	1030
01000	08000	328
.....

importation section (optional)

This section is optional. It is used by the covidImportation package to import global air importation data for seeding infections into the United States.

If you wish to include it, here are the options.

importation::param_list

importation:
  census_api_key: "fakeapikey00000"
  travel_dispersion: 3
  maximum_destinations: Inf
  dest_type: state
  dest_county: USA
  aggregate_to: airport
  cache_work: TRUE
  update_case_data: TRUE
  draw_travel_from_distribution: FALSE
  print_progress: FALSE
  travelers_threshold: 10000
  airport_cluster_distance: 80
  param_list:
    incub_mean_log: log(5.89)
    incub_sd_log: log(1.74)
    inf_period_nohosp_mean: 15
    inf_period_nohosp_sd: 5
    inf_period_hosp_mean_log: 1.23
    inf_period_hosp_sd_log: 0.79
    p_report_source: [0.05, 0.25]
    shift_incid_days: -10
    delta: 1

report section

The report section is completely optional and provides settings for making an R Markdown report. For an example of a report, see the Supplementary Material of our preprint

If you wish to include it, here are the options.

report:
  data_settings:
    pop_year: 2018
  plot_settings:
    plot_intervention: TRUE
  formatting:
    scenario_labels_short: ["UC", "S1"]
    scenario_labels:
      - Uncontrolled
      - Scenario 1
    scenario_colors: ["#D95F02", "#1B9E77"]
    pdeath_labels: ["0.25% IFR", "0.5% IFR", "1% IFR"]
    display_dates: ["2020-04-15", "2020-05-01", "2020-05-15", "2020-06-01", "2020-06-15"]
    display_dates2: ["2020-04-15", "2020-05-15", "2020-06-15"]

inference settings

iterations_per_slot

do_inference

gt_data_path

With inference model runs, the number of simulations nsimulations refers to the number of final model simulations that will be produced. The filtering$simulations_per_slot setting refers to the number of iterative simulations that will be run in order to produce a single final simulation (i.e., number of simulations in a single MCMC chain).

f

inference::statistics options

required options

name

aggregator

period

sim_var

data_var

likelihood

The statistics specified here are used to calibrate the model to empirical data. If multiple statistics are specified, this inference is performed jointly and they are weighted in the likelihood according to the number of data points and the variance of the proposal distribution.

f

optional options ?

remove_na

add_one

gt_start_date

gt_end_date

Optional sections

inference::hierarchical_stats_geo

The hierarchical settings specified here are used to group the inference of certain parameters together (similar to inference in "hierarchical" or "fixed/group effects" models). For example, users may desire to group all counties in a given state because they are geograhically proximate and impacted by the same statewide policies. The effect should be to make these inferred parameters follow a normal distribution and to observe shrinkage among the variance in these grouped estimates.

inference::priors

It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.

Ground truth data

name

module

geo_group_col

transform

inference:::priors

inference::

Communication Between Iterations

The pipeline uses files to communicate between different iterations. Currently, the following file types exist:

• seed

• init

• snpi

• spar

• seir

• hpar

• hnpi

• hosp

• llik

During each iteration, inference uses these files to communicate with the compartmental model and outcomes. The intent is that inference should only need to read and write these files, and that the compartmental model can handle everything else. In addition to the global versions of these files actually passed to the compartmental/reporting model, there exist chimeric versions used internally by inference and stored in memory. These copies are what inference interacts with when it needs to perturb values. While this design was chosen primarily to support modularity (a fixed communication boundary makes it easy to swap out the compartmental model), it has had a number of additional benefits.

Bootstrapping

The first iteration of an MCMC algorithm is a special case, because we need to pull initial conditions for our parameters. We originally developed the model without inference in mind, so the compartmental model is already set up to read parameter distributions from the configuration file, and to draw values from those distributions, and record those parameters to file. We take advantage of this to bootstrap our MCMC parameters by running the model one time, and reading the parameters it generated from file.

Resume from previous run

We can, instead of bootstrapping our first iteration, read in final values of a previous iteration. This allows us to resume from runs to save computational time and effectively continue iterating on the same chain. We call these resumes: inferred parameters are taken from a previous run and allowed to continue being inferred ;

Resumes take the following files (if they exist) from previous runs and uses them as the starting point of a new run:

• hnpi

• snpi

• seed

Continue from previous run

Updates to other files

\LLIK (inference runs only)

During inference runs, an additional file type, llik, is created, which is described in the section ;

These files contain the log-likelihoods of the model simulation for each subpopulation, as well as some diagnostics on acceptance.

The meanings of the columns are:

ll - These values are the log-likelihoods of data given the model and parameter values for a single subpopulation (in subpop column) ;

filename - ...

subpop - The values of this column are the names of the nodes from the geodata file.

accept - Either 0 or 1, depending on whether the parameters during this iteration of the simulation were accepted (1) or rejected (0) in that subpopulation ;

accept_avg ;

accept_prob ;

For inference runs, ... flepiMoP produces one file per parallel slot, for both global and chimeric outputs...

Need to add MultiPeriodModifier and hospitalization interventions

Overview

This documentation describes the new YAML configuration file options that may be used when performing inference on model runs. As compared to previous model releases, there are additions to the seeding and interventions sections, the outcomes section replaces the hospitalization section, and the filtering section added to the file.

Importantly, we now name our pipeline modules: seeding, seir, hospitalization and this becomes relevant to some of the new filtering specifications.

Models may be calibrated to any available time series data that is also an outcome of the model (COVID-19 confirmed cases, deaths, hospitalization or ICU admissions, hospital or ICU occupancy, and ventilator use). Our typical usage has calibrated the model to deaths, confirmed cases, or both. We can also perform inference on intervention effectiveness, county-specific baseline R0, and the risk of specific health outcomes.

We describe these options below and present default values in the example configuration sections.

Modifications to seeding

The model can perform inference on the seeding date and initial number of seeding infections in each subpop. An example of this new config section is:

The method for determining the proposal distribution for the seeding amount is hard-coded in the inference package (R/pkgs/inference/R/functions/perturb_seeding.R). It is pertubed with a normal distribution where the mean of the distribution 10 times the number of confirmed cases on a given date and the standard deviation is 1.

Modifications to interventions

The model can perform inference on the effectiveness of interventions as long as there is at least some calibration health outcome data that overlaps with the intervention period. For example, if calibrating to deaths, there should be data from time points where it would be possible to observe deaths from infections that occurred during the intervention period (e.g., assuming 10-18 day delay between infection and death, on average).

An example configuration file where inference is performed on scenario planning interventions is as follows:

interventions::settings::[setting_name]

Interventions may be specified in the same way as before, or with an added perturbation section that indicates that inference should be performed on a given intervention's effectiveness. As previously, interventions with perturbations may be specified for all modeled locations or for explicit subpop. In this setup, both the prior distribution and the range of the support of the final inferred value are specified by the value section. In the configuration above, the inference algorithm will search 0 to 0.9 for all subpops to estimate the effectiveness of the stayhome intervention period. The prior distribution on intervention effectiveness follows a truncated normal distribution with a mean of 0.6 and a standard deviation of 0.3. The perturbation section specifies the perturbation/step size between the previously-accepted values and the next proposal value.

New outcomes section

This section is now structured more like the interventions section of the config, in that it has scenarios and settings. We envision that separate scenarios will be specified for each IFR assumption.

outcomes::settings::[setting_name]

The settings for each scenario correspond to a set of different health outcome risks, most often just differences in the probability of death given infection (Pr(incidD|incidI)) and the probability of hospitalization given infection (Pr(incidH|incidI)). Each health outcome risk is referenced in relation to the outcome indicated in source. For example, the probability and delay in becoming a confirmed case (incidC) is most likely to be indexed off of the number and timing of infection (incidI).

Importantly, we note that incidI is automatically defined from the SEIR transmission model outputs, while the other compartment sources must be defined in the config before they are used.

Users must specific two metrics for each health outcome, probability and delay, while a duration is optional (e.g., duration of time spent in the hospital). It is also optional to specify a perturbation section (similar to perturbations specified in the NPI section) for a given health outcome and metric. If you want to perform inference (i.e., if perturbation is specified) on a given metric, that metric must be specified as a distribution (i.e., not fixed) and the range of support for the distribution represents the range of parameter space explored in the inference.

New filtering section

This section configures the settings for the inference algorithm. The below example shows the settings for some typical default settings, where the model is calibrated to the weekly incident deaths and weekly incident confirmed cases for each subpop. Statistics, hierarchical_stats_geo, and priors each have scenario names (e.g., sum_deaths, local_var_hierarchy, and local_var_prior, respectively).

filtering settings

With inference model runs, the number of simulations nsimulations refers to the number of final model simulations that will be produced. The filtering$simulations_per_slot setting refers to the number of iterative simulations that will be run in order to produce a single final simulation (i.e., number of simulations in a single MCMC chain).

filtering::statistics

The statistics specified here are used to calibrate the model to empirical data. If multiple statistics are specified, this inference is performed jointly and they are weighted in the likelihood according to the number of data points and the variance of the proposal distribution.

filtering::hierarchical_stats_geo

The hierarchical settings specified here are used to group the inference of certain parameters together (similar to inference in "hierarchical" or "fixed/group effects" models). For example, users may desire to group all counties in a given state because they are geograhically proximate and impacted by the same statewide policies. The effect should be to make these inferred parameters follow a normal distribution and to observe shrinkage among the variance in these grouped estimates.

filtering::priors

It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.

Overview

In order for the models specified previously to be dynamically simulated, the user must provide initial conditions, in addition to the model structure and parameter values. Initial conditions describe the value of each variable in the model at the time point that the simulation is to start. For example, on day zero of an outbreak, we may assume that the entire population is susceptible except for one single infected individual. Alternatively, we could assume that some portion of the population already has prior immunity due to vaccination or previous infection. Different initial conditions lead to different model trajectories.

The initial_conditions section of the configuration file is detailed below. Note that in some cases, can replace or complement the initial condition, the table below provides a quick comparison of these sections.

Specifying model initial conditions

The configuration items in the initial_conditions section of the config file are

initial_conditions:method Must be either "Default", "SetInitialConditions", or "FromFile".

initial_conditions:initial_conditions_fileRequired for methods “SetInitialConditions” and “FromFile” . Path to a .csv or .parquet file containing the list of initial conditions for each compartment.

initial_conditions:initial_file_type Only required for method: “FolderDraw”. Description TBA

initial_conditions::allow_missing_subpops Optional for all methods, determines what will happen if initial_conditions_file is missing values for some subpopulations. If FALSE, the default behavior, or unspecified, an error will occur if subpopulations are missing. If TRUE, then for subpopulations missing from the initial_conditions file, it will be assumed that all individuals begin in the first compartment (the “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group), unless another compartment is designated to hold the rest of the individuals ;

initial_conditions::allow_missing_compartments Optional for all methods. If FALSE, the default behavior, or unspecified, an error will occur if any compartments are missing for any subpopulation. If TRUE, then it will be assumed there are zero individuals in compartments missing from the initial_conditions file.

initial_conditions::proportional If TRUE, assume that the user has specified all input initial conditions as fractions of the population, instead of numbers of individuals (the default behavior, or if set to FALSE). Code will check that initial values in all compartments sum to 1.0 and throw an error if not, and then will multiply all values by the total population size for that subpopulation ;

Details on implementing each initial conditions method and the options that go along with it are below.

initial_conditions::method

Default

The default initial conditions are that the initial value of all compartments for each subpopulation will be zero, except for the first compartment, whose value will be the population size. The “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group.

For example, a model with the following compartments

with the accompanying geodata file

will be started with 1000 individuals in the S_child_unvaxxed in the "small province" and 10,000 in that compartment in the "large province".

SetInitialConditions

With this method users can specify arbitrary initial conditions in a convenient formatted input .csv or .parquet file.

For example, for a model with the following compartments and initial_conditions sections

with the accompanying geodata file

where initial_conditions.csv contains

the model will be started with half of the population of both subpopulations, consisting of children and the other half of adults, everyone unvaccinated, and 5 infections (in exposed-but-not-yet-infectious class) among the unvaccinated adults in the large province, with the remaining individuals susceptible (4995). All other compartments will contain zero individuals initially ;

initial_conditions::initial_conditions_file must contain the following columns:

• subpop – the name of the subpopulation for which the initial condition is being specified. By default, all subpopulations must be listed in this file, unless the allow_missing_subpops option is set to TRUE.

• mc_name – the concatenated name of the compartment for which an initial condition is being specified. The order of the compartment groups in the name must be the same as the order in which these groups are defined in the config for the model, e.g., you cannot say unvaccinated_S.

• amount – the value of the initial condition; either a numeric value or the string "rest".

For each subpopulation, if there are compartments that are not listed in SetInitialConditions, an error will be thrown unless allow_missing_compartments is set to TRUE, in which case it will be assumed there are zero individuals in them. If the sum of the values of the initial conditions in all compartments in a location does not add up to the total population of that location (specified in the geodata file), an error will be thrown. To allocate all remaining individuals in a subpopulation (the difference between the total population size and those allocated by defined initial conditions) to a single pre-specified compartment, include this compartment in the initial_conditions_file but instead of a number in the amount column, put the word "rest" ;

If allow_missing_subpops is FALSE or unspecified, an error will occur if initial conditions for some subpopulations are missing. If TRUE, then for subpopulations missing from the initial_conditions file, it will be assumed that all individuals begin in the first compartment. (The “first” compartment depends on how the model was specified, and will be the compartment that contains the first named category in each compartment group.)

FromFile

For example, for an input configuration file containing

with the accompanying geodata file

where initial_conditions_from_previous.csv contains

The simulation would be initiated on 2021-06-01 with these values in each compartment (no children vaccinated, only adults in the small province vaccinated, some past and current infection in both compartments but ).

initial_conditions::initial_conditions_file must contain the following columns:

• mc_value_type – in model output files, this is either prevalence or incidence. Prevalence values only are selected to be used as initial conditions, since compartmental models described the prevalence (number of individuals at any given time) in each compartment. Prevalence is taken to be the value measured instantaneously at the start of the day

• mc_name – The name of the compartment for which the value is reported, which is a concatenation of the compartment status in each state type, e.g. "S_adult_unvaxxed" and must be in the same order as these groups are defined in the config for the model, e.g., you cannot say unvaxxed_S_adult.

• subpop_1, subpop_2, etc. – one column for each different subpopulation, containing the value of the number of individuals in the described compartment in that subpopulation at the given date. Note that these are named after the nodenames defined by the user in the geodata file.

• date – The calendar date in the simulation, in YYYY-MM-DD format. Only values with a date that matches to the simulation start_date will be used ;

SetInitialConditionsFolderDraw, FromFileFolderDraw

The way that initial conditions is specified with SetInitialConditions and FromFile results in a single value for each compartment and does not easily allow the user to instead specify a distribution (like is possible for compartmental or outcome model parameters). If a user wants to use different possible initial condition values each time the model is run, the way to do this is to instead specify a folder containing a set of file with initial condition values for each simulation that will be run. The user can do this using files with the format described in initial_conditions::method::SetInitialConditions using instead method::SetInitialConditionsFolder draw. Similarly, to provide a folder of initial condition files with the format described in initial_conditions::method:FromFile using instead method::FromFileFolderDraw ;

Each file in the folder needs to be named according to the same naming conventions as the model output files: run_number.runID.file_type.[csv or parquet] where ....[DESCRIBE] as it is now taking the place of the seeding files the model would normally outpu ;

Only one additional config argument is needed to use a FolderDraw method for initial conditions:

initial_file_type: either seir or seed

When using FolderDraw methods, initial_conditions_file should now be the path to the directory that contains the folder with all the initial conditions files. For example, if you are using output from another model run and so the files are in an seir folder within a model_output folder which is in within your project directory, you would use initial_conditions_file: model_outpu ;

The model needs the configurations file to run (described in previous sections). These configs become lengthy and sometimes difficult to type manually. The config writer helps to generate configs provided the relevant files are present.

Print Functions:

These functions are used to print specific sections of the configuration files.

print_header

Used to generate the global header. For more information on global headers click .

print_spatial_setup

Used to generate the spatial setup section of the configuration. For more information on spatial setup click .

print_compartments

Used to generate the compartment list for each way a population can be divided.

Parts of the configuration files that are printed but not needed for FlepiMop runs (need to be mentioned for US or COVID-19 specific runs??)

Spatial Setup:

• census year: year of geodata files

• modeled states (sim_states): This has US state abbreviations. Do we include the names of the sub-populations in the geodata file? Eg: small_province, large_province

• state_level: Specifies if the runs are run for US states

filtering section

The filtering section configures the settings for the inference algorithm. The below example shows the settings for some typical default settings, where the model is calibrated to the weekly incident deaths and weekly incident confirmed cases for each subpop. Statistics, hierarchical_stats_geo, and priors each have scenario names (e.g., sum_deaths, local_var_hierarchy, and local_var_prior, respectively).

filtering settings

With inference model runs, the number of simulations nsimulations refers to the number of final model simulations that will be produced. The filtering$simulations_per_slot setting refers to the number of iterative simulations that will be run in order to produce a single final simulation (i.e., number of simulations in a single MCMC chain).

filtering::statistics

The statistics specified here are used to calibrate the model to empirical data. If multiple statistics are specified, this inference is performed jointly and they are weighted in the likelihood according to the number of data points and the variance of the proposal distribution.

filtering::hierarchical_stats_geo

The hierarchical settings specified here are used to group the inference of certain parameters together (similar to inference in "hierarchical" or "fixed/group effects" models). For example, users may desire to group all counties in a given state because they are geograhically proximate and impacted by the same statewide policies. The effect should be to make these inferred parameters follow a normal distribution and to observe shrinkage among the variance in these grouped estimates.

filtering::priors

It is now possible to specify prior values for inferred parameters. This will have the effect of speeding up model convergence.

Ground truth data

Likelihood function

Fitting parameters

Ground truth data

flepiMoP

Current branch: main

This repository contains all the code underlying the mathematical model and the data fitting procedure, as well as ...

To actually run the model, this repository folder must be located inside a location folder (e.g. COVID19_USA) which contains additional files describing the specifics of the model to be run (i.e. the config file), all the necessary input data (i.e. the population structure), and any data to which the model will be fit (i.e. cases and death counts each day)

/gempyor_pkg

This directory contains the core Python code that creates and simulates generic compartmental models and additionally simulates observed variables. This code is called gempyor for General Epidemics Modeling Pipeline with Yterventions and Outcome Reporting. The code in gempyor is called from R scripts (see /main_scripts and /R sections below) that read the config, run the model simulation via gempyor as required, read in data, and run the model inference algorithms.

• pyproject.toml - contains the build system requirements and dependencies for the gempyor package; used during package installation

• setup.cfg - contains information used by Python's setuptools to build the gempyor package. Contains the definitions of command line shortcuts for running simulations directly from gempyor (bypassing R interface) if desired

/gempyor_pkg/src/gempyor/

• seir.py - Contains the core code for simulating the mathematical model. Takes in the model definition and parameters from the config, and outputs a file with a timeseries of the value of each state variable (# of individuals in each compartment)

• simulate_seir.py -

• steps_rk.py -

• steps_source.py -

• outcomes.py - Contains the core code for generating the outcome variables. Takes in the output of the mathematical model and parameters from the config, and outputs a file with a timeseries of the value of each outcome (observed) variable

• simulate_outcomes.py -

• setup.py

• file_paths.py -

• compartments.py

• parameters.py

• results.py

• seeding_ic.py

• /NPI/

  • base.py -

  • SinglePeriodModifier.py -

  • MultiPeriodModifier.py -

  • SinglePeriodModifierInterven.py -

• /dev - contains functions that are still in development

• /data - ?

/gempyor_pkg/docs

Contains notebooks with some gempyor-specific documentation and examples

• Rinterface.Rmd - And R notebook that provides some background on gempyor and describes how to run it as a standalone package in python, without the R wrapper scripts or the Docker.

• Rinterface.html - HTML output of Rinterface.Rmd

/R

/main_scripts

This directory contains the R scripts that takes the specifications in the configuration file and sets up the model simulation, reads the data, and performs inference.

• inference_main.R - This is the master R script used to run the model. It distributes the model runs across computer cores, setting up runs for all the scenarios specified in the config, and for each model iteration used in the parameter inference. Note that despite the name "inference" in this file, this script must be used to run the model even if no parameter inference is conducted

• inference_slot.R - This script contains the main code of the inference algorithm.

• create_seeding.R -

/R_packages

This directory contains the core R code - organized into functions within packages - that handle the model setup, data pulling and processing, conducting parameter inference for the model, and manipulating model output.

• flepicommon

  • config.R

  • DataUtils.R

  • file_paths.R

  • safe_eval.R

  • compartments.R

• inference - contains code to

  • groundtruth.R - contains functions for pulling ground truth data from various sources. Calls functions in the flepicommon package

  • functions.R - contains many functions used in running the inference algorithm

  • inference_slot_runner_funcs.R - contains many functions used in running the inference algorithm

  • inference_to_forecast.R -

  • documentation.Rmd - Summarizes the documentation relevant to the inference package, including the configuration file options relevant to model fitting

  • InferenceTest.R -

  • /tests/ -

• config.writer

  • create_config_data.R

  • process_npi_list.R

  • yaml_utils.R

• report.generation

  • DataLoadFuncs.R

  • ReportBuildUtils.R

  • ReportLoadData.R

  • setup_testing_environment.R

/test

/data

Depreciated? Should be removed

/vignettes

Depreciated? Should be removed

/doc

Depreciated? Should be removed

/batch

/slurm_batch

COVID19_USA Repository

Current branch: main

/R

Contains R scripts for generating model input parameters from data, writing config files, or processing model output. Most of the files in here are historic (specific to a particular model run) and not frequently used. Important scripts include:

• get_vacc_rate_and_outcomes_R13.R - this pulls vaccination coverage and variant prevalence data specific to rounds (either empirical, or specified by the scenario), and adjusts these data to the formats required for the model. Several data files are created in this process: variant proportions for each scenario, vaccination rates by age and dose. A file is also generated that defines the outcome ratios (taking in to account immune escape, cross protection and VE).

/R/scripts/config_writers

Scripts to generate config files for particular submissions to the Scenario Modeling Hub. Most of this functionality has now been replaced by the config writer package ()

R/scripts/postprocess

Scripts to process the output of model runs into data formats and plots used for Scenario Modeling Hub and Forecast Hub. These scripts pull runs from AWS S3 buckets and processes and formats them to specifications for submissions to Scenario Modeling Hubs, Forecast Hubs and FluSight. These formatted files are saved and the results visualized. This script uses functions defined in /COVIDScenarioPipeline/R/scripts/postprocess.

• run_sum_processing.R

/data

Contains data files used in parameterizing the model for COVID-19 in the US (such as creating the population structure, describing vaccine efficacy, describing parameter alterations due to variants, etc). Some data files are re-downloaded frequently using scripts in the pipeline (us_data.csv) while others are more static (geodata, mobility)

Important files and folders include

• geodata.csv

• geodata_2019_statelevel.csv

• mobility.csv

• mobility_territories_2011-2015_statelevel.csv

• outcomes_ratios.csv

• US_CFR_shift_dates_v3.csv

• US_hosp_ratio_corrections.cs

• seeding_agestrat_RX.csv

/data/shp

"Shape-files" (.shp) that .....

/data/outcomes

• usa-subpop-params-output_V2.parquet

/data/intervention_tracking

Data files containing the dates that different non pharmaceutical interventions (like mask mandates, stay-at-home orders, school closures) were implemented by state

/data/vaccination

Files used to create the config elements related to vaccination, such as vaccination rates by state by age and vaccine efficacy by dose

/data/variant

Files created in the process of downloading and analyzing data on variant proportions

/manuscripts

Contains files for scientific manuscripts using results from the pipeline. Not up to date

/config

Contains an archive of configuration files used for previous model runs

/old_configs

Same as above. Contains an archive of configuration files used for previous model runs

/scripts

Depreciated - to be removed? - contains rarely used scripts

/notebook

Depreciated - to be removed? - contains rarely used notebooks to check model input. Might be used in some unit tests?

/NPI

empty?

/ScenarioHub

Depreciated - to be removed?

Some information to consider if you'd like your script to be run automatically after an inference run ;

• Most R/python packages are installed already installed. Try to run your script on the conda environment defined on the (or easier if you are not set up on MARCC, ask me)

• There will be some variables set in the environment. These variables are:

  • $CONFIG_PATH the path to the configuration fil ;

  • $FLEPI_RUN_INDEX the run id for this run (e.g `CH_R3_highVE_pesImm_2022_Jan29`

  • $JOB_NAME this job name (e.g USA-20230130T163847_inference_med)

  • $FS_RESULTS_PATH the path where lies the model results. It's a folder that contains the model_ouput/ as a subfolder

  • $FLEPI_PATH path of the flepiMoP repository.

  • $DATA_PATH path of the Data directory (e.g Flu_USA or COVID19_USA).

  • Anything you ask can theoretically be provided here.

• The script must run without any user intervention.

• The script is run from $DATA_PATH.

• Your script lies in the flepiMoP directory (preferably) or it's ok if it is in a data directory if it makes sense ;

• It is run on a 64Gb of RAM multicore machine. All scripts combined must complete under 4 hours, and you can use multiprocessing (48 cores)

• Outputs (pdf, csv, html, txt, png ...) must be saved in a directory named pplot/ (you can assume that it exists) in order to be sent to slack by FlepiBot 🤖 after the run.

• an example postprocessing script (in python) is .

• You can test your script on MARCC on a run that is already saved in /data/struelo1/flepimop-runs or I can do it for you.

• Once your script works, add (or ask to add) the command line to run in file batch/postprocessing_scripts.sh between the START and END lines, with a little comment about what your script does.

Config changes w.r.t classical inference

You need, under inference, to add method: emcee and modify the statistics: as shown in the diff below (basically: all resampling goes to one subsection, with some minor changes to names).

To see which llik options and regularization (e.g do you want to weigh more the last weeks for forecasts, or do you want to add the sum of all subpop) see files statistics.py.

Test on your computer

Install gempyor from branch emcee_batch . Test your config by running:

on your laptop. If it works, it should produce:

• plots of simulation directly from your config

• plots after the fits with the fits and the parameter chains

• and h5 file with all the chains

• and in model_output, the final hosp/snpi/seir/... files in the flepiMoP structure.

It will output something like

```

Here, it says the config fits 92 parameters, we'll keep that in mind and choose a number of walkers greater than (ideally 2 times) this number of parameters.

Run on cluster

Install gempyor on the cluster. test it with the above line, then modify this script:

so you need to have:

• -c (number of core) equal to roughly half the number of walkers (slots/parallel chains)

• mem to be around two times the number of walkers. Look at the computes nodes you have access to and make something that can be prioritized fast enough.

• nsamples is the number of final results you want, but it's fine not to care about it, I rerun the sampling from my computer.

• To resume from an existing run, add the previous line --resume and it 'll start from the last parameter values in the h5 files.

Postprocess EMCEE

To analyze run postprocessing/emcee_postprocess.ipynb First, this plots the chains and then it runs nsamples (you can choose it) projection with the end of the chains and does the plot of the fit, with and without projections

⇅ Get set up to use Github

You need to interact with Github to run and edit flepimop code. is a web platform for people to share and manage software, and it is based on a 'version control' software called git that helps programmers keep track of changes to code. Flepimop core code as well as example projects using flepimop code are all stored on Github, and frequently being updated. The first step to using flepimop for your own project is making sure you're set up to interact with code shared on Github.

If you are totally new to Github, navigate to and Sign Up for a new account. Read about the .

To work with flepimop code, you can do some tasks from the Github website, but you'll also need a way to 'clone' the code to your own local computer and keep it up to date with versions hosted online. You can do this either using a user interface like , or, using commands from the command line. Make sure you have one or both installed.

If you are a veteran user, make sure you're signed in on Github.com and through whatever method you use locally on your computer to interact with Github.

🧺 Organizing your folders

In order to run any model with flepiMoP, you need access to two separate directories: One containing the flepiMoP code (code directory), and another containing the specific input files to run the model of your choosing (and to save the output from that model) (project directory). The is available in a public repository on Github which can be pulled locally to serve as the code directory. We highly recommend also using Github to create your project directory. To get familiar with the code, we recommend starting with our example configurations by making a fork of . If you need to create your own project repository from scratch, see instructions .

For both the project repository and flepiMoP code repository, make sure you're on the correct branch, then pull updates from Github. Take note of the local file path to each directory.

🔐 Access the flepiMoP model code

To get access to the flepiMoP model code - stored as a - you need to copy it to your local computer (called "" in git lingo).

To clone the flepimop code repository

• If you're using the command line in a terminal, first navigate to your local directory you'll use as the parent directory for all these files. Then, use the command: git clone https://github.com/HopkinsIDD/flepiMoP

• If you're using Github Desktop, go File -> Clone Repository, switch to the "URL" tab and copy the URL https://github.com/HopkinsIDD/flepiMoP there. For the "Local Path" option, make sure you choose your desired parent directory.

📂 Create a project repository from flepimop_sample

The flepimop_sample repository contains example configuration files you can use to run simple models, as well as structured model inputs (e.g., population sizes, initial conditions) organized as we recommend organizing your own project repositories. A good way to start out with flepimop is to try running these simple models, or use them as a template to make more complex ones ;

2. Clone the sample project repository locally.

   • If you're using the command line in a terminal, first navigate to your local directory you'll use as the parent directory for all these files. Then, use the command: git clone <my flepimop_sample repository URL>

   • If you're using Github Desktop, go File -> Clone Repository and either find the repository name under the Github.com list, or switch to the "URL" tab and copy the URL there. For the "Local Path" option, make sure you choose your desired parent directory.

   • In either case, make you are cloning your forked version of the repository, not the version owned by HopkinsIDD

3. Make sure the sample project repository files are up to date

   • To make sure that changes in the core flepimop code are always in sync with our example configuration files, we keep the newest versions of the sample code in the flepimop repository (in the examples directory). We try to keep flepimop_sample repository up to date, but in case we haven't kept up, it's best to copy the most up to date files over yourself

   • Copy all the contents of the examples/tutorials directory into your local version of flepimop_sample . You can do this by copying and pasting the files, or, by running the following at the command line in your terminal (assuming you have navigated to the parent directory that contains both your repositories):

     • cp -a ./flepiMoP/examples/tutorials/. ./flepimop_sample/

   • You will see if there are any changes to flepimop_sample files by looking at Github's change tracking feature locally ;

📂 Create a project repository from scratch

Create a repository for your project on Github, naming it something other than flepiMoP. This repository will eventually contain the configuration file specifying your model, any other input files or data, and will be where the model output will be stored. These files are always kept completely separately from the universal flepimop code that runs the model.

Clone the repository locally (as above), either before or after populating it with files described below.

👨‍👩‍👧‍👦 Populate the repository

Add config file

Put your model configuration file(s) directly in this repository.

Create a folder for model inputs

🏃🏽‍♀️ Running the code

🤔 Deciding how to run

Final notes

Since you will be editing files in your project repository frequently, get in the habit of using the git workflow - committing those changes when you've completed a concrete task, pushing your changes to the remote (online) version of your repository on Github.com, and making sure to fetch changes from the remote version to download to your local version before you start a new task (if you share your project with others or switch computers).

Access model files

See the section to ensure you have access to the correct files needed to run. On your local machine, determine the file paths to:

• the directory containing the flepimop code (likely the folder you cloned from Github), which we'll call <dir1>

• the directory containing your project code including input configuration file and population structure (again likely from Github), which we'll call <dir2>

🧱 Set up Docker

is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime. This means you can run and install software without installing the dependencies in the local operating system.

A Docker container is an environment which is isolated from the rest of the operating system i.e. you can create files, programs, delete and everything but that will not affect your OS. It is a local virtual OS within your OS ;

For flepiMoP, we have a Docker container that will help you get running quickly ;

Make sure you have the Docker software installed, and then open your command prompt or terminal application ;

Run the Docker image

First, make sure you have the latest version of the flepimop Docker (hopkinsidd/flepimop) downloaded on your machine by opening your terminal application and entering:

Next, run the Docker image by entering the following, replace <dir1> and <dir2> with the path names for your machine (no quotes or brackets, just the path text):

In this command, we run the Docker container, creating a volume and mounting (-v) your code and project directories into the container. Creating a volume and mounting it to a container basically allocates space in Docker for it to mirror - and have read and write access - to files on your local machine ;

The folder with the flepiMoP code <dir2> will be on the path flepimop within the Docker environment, while the project folder will be at the path `drp. ;

Define environment variables

Create environmental variables for the paths to the flepimop code folder and the project folder:

Go into the code directory and do the installation the R and Python code packages

Each installation step may take a few minutes to run.

Run the code

Everything is now ready 🎉 The next step depends on what sort of simulation you want to run: One that includes inference (fitting model to data) or only a forward simulation (non-inference). Inference is run from R, while forward-only simulations are run directly from the Python package gempyor.

In either case, navigate to the project folder and make sure to delete any old model output files that are there

Inference run

An inference run requires a configuration file that has the inference section. Stay in the $DATA_PATH folder, and run the inference script, providing the name of the configuration file you want to run (ex. config.yml ;

This will run the model and create a lot of output files in $DATA_PATH/model_output/ ;

The last few lines visible on the command prompt should be:

[[1]]

[[1]][[1]]

[[1]][[1]][[1]]

NULL

If you want to quickly do runs with options different from those encoded in the configuration file, you can do that from the command line, for example

where:

• n is the number of parallel inference slots,

• j is the number of CPU cores to use on your machine (if j > n, only n cores will actually be used. If j <n, some cores will run multiple slots in sequence)

• k is the number of iterations per slots.

You can put all of this together into a single script that can be run all at once ;

Non-inference run

Stay in the $DATA_PATH folder, and run a simulation directly from forward-simulation Python package gempyor,call gempyor-simulate providing the name of the configuration file you want to run (ex. config.yml ;

You can put all of this together into a single script that can be run all at once ;

Finishing up

You can avoid repeating all the above steps every time you want to run the code. When the docker run command creates an container, it is stored locally on your computer with all the installed packages/variables/etc you created. You can leave this container and come back to it whenever you want, without having to redo all this set up ;

When you're in the Docker container, figure out the name Docker has given to the container you created by typing

the output will be something silly like

write this down for later reference. You can also see the container name in the Docker Desktop app's Containers tab ;

To "detach" from the Docker container and stop it, type CTLR + c

The command prompt for your terminal application is now just running locally, not in the Docker container ;

Next time you want to re-start and "attach" the container, type

at the command line or hit the play button ▶️ beside the container's name in the Docker app. Replace container_name with the name for your old container ;

Then "attach" to the container by typing

The reason that stopping/starting a container is separate from detaching/attaching is that technically you can leave a container (and any processes within it) running in the background and exit it. In case you want to do that, detach and leave it running by typing CTRL + p then quickly CTRL + q. Then when you want to attach to it again, you don't need to do the part about starting the container ;

If you the core model code within the flepimop repository (flepimop/flepimop/gempyor_pkg/ or flepimop/flepimop/R_packages) has been edited since you created the contained, or if the R or Python package requirements have changed, then you'll have to re-run the steps to install the packages, but otherwise, you can just start running model code!