Running with Docker locally 🛳
Short tutorial on running FlepiMop on your personal computer using a "Docker" container
Access model files
See the Before any run section to ensure you have access to the correct files needed to run. On your local machine, determine the file paths to:
the directory containing the flepimop code (likely the folder you cloned from Github), which we'll call
<dir1>
the directory containing your project code including input configuration file and population structure (again likely from Github), which we'll call
<dir2>
For example, if you clone your Github repositories into a local folder called Github and are using the flepimop_sample as a project repository, your directory names could be On Mac:
<dir1> = /Users/YourName/Github/flepiMoP
<dir2> = /Users/YourName/Github/flepimop_sample On Windows: <dir1> = C:\Users\YourName\Github\flepiMoP
<dir2> = C:\Users\YourName\Github\flepimop_sample
(hint: if you navigate to a directory like C:\Users\YourName\Github
using cd C:\Users\YourName\Github
, modify the above <dir1>
paths to be .\flepiMoP
and .\flepimop_sample)
Note that Docker file and directory names are case sensitive
🧱 Set up Docker
Docker is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime. This means you can run and install software without installing the dependencies in the local operating system.
A Docker container is an environment which is isolated from the rest of the operating system i.e. you can create files, programs, delete and everything but that will not affect your OS. It is a local virtual OS within your OS.
For flepiMoP, we have a Docker container that will help you get running quickly!
Make sure you have the Docker software installed, and then open your command prompt or terminal application.
Helpful tools
To understand the basics of Docker, refer Docker Basics. The following Docker Tutorial may also be helpful.
To install Docker for Mac, refer to the following link: Installing Docker for Mac. Pay special attention to the specific chip your Mac has (Apple Silicon vs Intel), as installation files and directions differ
To install Docker for Windows, refer to the following link: Installing Docker for Windows
To find the Windows Command Prompt, type “Command Prompt" in the search bar and open it. This Command Prompt Video Tutorial may be helpful for new users.
To find the Apple Terminal, type "Terminal" in the search bar or go to Applications -> Utilities -> Terminal.
Run the Docker image
First, make sure you have the latest version of the flepimop Docker (hopkinsidd/flepimop)
downloaded on your machine by opening your terminal application and entering:
Next, run the Docker image by entering the following, replace <dir1>
and <dir2>
with the path names for your machine (no quotes or brackets, just the path text):
On Windows: If you get an error, you may need to delete the "\" line breaks and submit as a single continuous line of code.
In this command, we run the Docker container, creating a volume and mounting (-v
) your code and project directories into the container. Creating a volume and mounting it to a container basically allocates space in Docker for it to mirror - and have read and write access - to files on your local machine.
The folder with the flepiMoP code <dir2>
will be on the path flepimop
within the Docker environment, while the project folder will be at the path drp.
You now have a local Docker container installed, which includes the R and Python versions required to run flepiMop with all the required packagers already installed!
You don't need to re-run the above steps every time you want to run the model. When you're done using Docker for the day, you can simply "detach" from the container and pause it, without deleting it from your machine. Then you can re-attach to it when you next want to run the model.
Define environment variables
Create environmental variables for the paths to the flepimop code folder and the project folder:
Go into the code directory and do the installation the R and Python code packages
Each installation step may take a few minutes to run.
Note: These installations take place in the Docker container and not the local operating system. They must be made once while starting the container and need not be done for every time you run a model, provided they have been installed once. You will need an active internet connection for pulling the Docker image and installing the R packages (since some are hosted online), but not for other steps of running the model
Run the code
Everything is now ready 🎉 The next step depends on what sort of simulation you want to run: One that includes inference (fitting model to data) or only a forward simulation (non-inference). Inference is run from R, while forward-only simulations are run directly from the Python package gempyor
.
In either case, navigate to the project folder and make sure to delete any old model output files that are there
Inference run
An inference run requires a configuration file that has the inference
section. Stay in the $DATA_PATH
folder, and run the inference script, providing the name of the configuration file you want to run (ex. config.yml
)
This will run the model and create a lot of output files in $DATA_PATH/model_output/
.
The last few lines visible on the command prompt should be:
[[1]]
[[1]][[1]]
[[1]][[1]][[1]]
NULL
If you want to quickly do runs with options different from those encoded in the configuration file, you can do that from the command line, for example
where:
n
is the number of parallel inference slots,j
is the number of CPU cores to use on your machine (ifj
>n
, onlyn
cores will actually be used. Ifj
<n
, some cores will run multiple slots in sequence)k
is the number of iterations per slots.
You can put all of this together into a single script that can be run all at once:
Non-inference run
Stay in the $DATA_PATH
folder, and run a simulation directly from forward-simulation Python package gempyor,
call gempyor-simulate
providing the name of the configuration file you want to run (ex. config.yml
)
It is currently required that all configuration files have an interventions
section. There is currently no way to simulate a model with no interventions, though this functionality is expected soon. For now, simply create an intervention that has value zero.
You can put all of this together into a single script that can be run all at once:
Finishing up
You can avoid repeating all the above steps every time you want to run the code. When the docker run
command creates an container, it is stored locally on your computer with all the installed packages/variables/etc you created. You can leave this container and come back to it whenever you want, without having to redo all this set up.
When you're in the Docker container, figure out the name Docker has given to the container you created by typing
the output will be something silly like
write this down for later reference. You can also see the container name in the Docker Desktop app's Containers tab.
To "detach" from the Docker container and stop it, type CTLR
+ c
The command prompt for your terminal application is now just running locally, not in the Docker container.
Next time you want to re-start and "attach" the container, type
at the command line or hit the play button ▶️ beside the container's name in the Docker app. Replace container_name with the name for your old container.
Then "attach" to the container by typing
The reason that stopping/starting a container is separate from detaching/attaching is that technically you can leave a container (and any processes within it) running in the background and exit it. In case you want to do that, detach and leave it running by typing CTRL
+ p
then quickly CTRL
+ q
. Then when you want to attach to it again, you don't need to do the part about starting the container.
If you the core model code within the flepimop repository (flepimop/flepimop/gempyor_pkg/
or flepimop/flepimop/R_packages
) has been edited since you created the contained, or if the R or Python package requirements have changed, then you'll have to re-run the steps to install the packages, but otherwise, you can just start running model code!
Last updated