Running your Jupyter notebooks from Docker means you can easily recreate your environment on another computer.
When you’re executing several long running jobs, you can sneakily run an additional server on a co-workers machine!
If you are new to Docker, you can checkout my introduction to docker, as this tutorial will skip the basics. I have included the complete Dockerfile at the bottom of the post in case you don’t want to read the step by step instructions.
Installing Jupyter with pip
First I am going to show you a basic Jupyter setup. For this we are going to install Jupyter notebook, NumPy, SciPy, Pandas, Scikit-learn and Matplotlib.
FROM ubuntu:16.04 RUN apt-get update \ && apt-get install -y python-pip build-essential \ && apt-get -y autoremove \ && apt-get -y clean \ && rm -rf /var/lib/apt/lists/* RUN pip install pip -U \ && pip install jupyter numpy scipy pandas sklearn matplotlib \ && rm -r /root/.cache/pip CMD jupyter-notebook --ip="*" --no-browser
The first line states which base image you want to start from, in this case I’ve chosen Ubuntu 16.04 Xenial Xerus. The second command (starting with
build-essential which is a set of tools that are commonly required for installing software from source. The additional lines in this
RUN command clean up any excess data that was added to the image due to using
RUN command updates pip and installs the required pip packages. The final part of this statement removes the pip cache to reduce the size of the image.
CMD line launches the Jupyter notebook server. The server will be accessed from outside of the container, which is why you wantt to allow all IP addresses to connect, and don’t want to launch a browser.
To build and run the container, you can use the following commands.
docker build -t <imagename> . docker run -ti -v /your/notebook/directory:/notebooks -p 7777:8888 <imagename>
/your/notebook/directory with your local (host) notebook directory, this will then be mounted at
/notebooks/ within the container.
Following this you will be able to access Jupyter by navigating to
localhost:7777 in your browser.
Installing the same packages as your current environment
There is a shortcut to install the same packages and package versions you currently have installed in your development environment (provided you use pip for package installation). If you run
pip freeze in the command line this outputs a list of installed packages and their version numbers, which will look something like this:
numpy==1.11.2 pandas==0.19.0 ...
If you write that to a requirements file (
pip freeze > requirements.txt), and put it in the same directory as the Dockerfile. We can swap the pip RUN command for the following and now all of your previous pip install commands will be replicated in the container.
# copy across pip files COPY requirements.txt /requirements.txt # install python packages RUN pip install pip -U \ && pip install -r /requirements.txt \ && rm -r /root/.cache/pip
Install Jupyter extensions
I wrote a post on Jupyter extensions here, and this is how to add them to out Jupyter Docker container.
We need to add additional package
jupyter_contrib_nbextensions to the pip list from before as shown below, and need to add an additional line to activate the extensions.
RUN pip install pip -U \ && pip install jupyter numpy scipy pandas sklearn matplotlib \ && pip install jupyter_contrib_nbextensions \ && rm -r /root/.cache/pip RUN jupyter contrib nbextension install
If you want to copy across your existing notebook extension settings, they are stored in
~/.jupyter/nbconfig/notebook.json. You can see an example starting point in my other post. If you copy that file into the same directory as your Dockerfile, and add the line below, the config file will be copied into your container and loaded into Jupyter.
COPY notebook.json /root/.jupyter/nbconfig/notebook.json
Now you know how to create a Docker container to run Jupyter notebooks. To make things easier, I’ve included a complete Dockerfile below. This file assumes that you have written a requirements.txt file, and that your directory contains a
notebook.json config file.
FROM ubuntu:16.04 RUN apt-get update \ && apt-get install -y python-pip build-essential \ && apt-get -y autoremove \ && apt-get -y clean \ && rm -rf /var/lib/apt/lists/* RUN pip install pip -U \ && pip install -r /requirements.txt \ && pip install jupyter_contrib_nbextensions \ && rm -r /root/.cache/pip RUN jupyter contrib nbextension install COPY notebook.json /root/.jupyter/nbconfig/notebook.json CMD jupyter-notebook --ip="*" --no-browser