If you have existing code, text, or other material in the form of Jupyter notebooks, you may be wondering if there’s a way to embed your .ipynb files directly into an Educative course. Our platform allows you to embed your Jupyter notebooks in your lessons within a live virtual machine (Live VM) by using our Live App widget.

Overview

To enable this advanced functionality, you will first need to perform a one-time Docker setup. The result will be a Docker environment that supports Jupyter notebooks in your course. You will then be able to select and embed your .ipynb files in individual lessons.

Here is an overview of the setup steps you will perform:

  1. Create a Dockerfile.

  2. Prepare a Dockerfile tarball (including all your .ipynb files) for upload.

  3. Upload the Dockerfile tarball to your course in the Docker Container section.

  4. Configure a Jupyter Docker job to activate an .ipynb file of your choice in the Live App widget.

  5. Embed a Live App in a lesson of your choice, linked to the Docker job.

Note: If you are unfamiliar with Docker, don’t worry. You can simply follow the instructions below. If you’d first like additional context, read our Custom Docker Environments section and then return to this lesson.

Let’s go over each of these steps in detail.

Prepare Your Dockerfile and Python Jupyter Files

To embed Python Jupyter notebooks in your lessons, you will first need to set up a custom Docker environment for use in your course. In order to do that, you will need to prepare a zipped tarball that includes three things:

  • A Dockerfile with instructions needed to install Python and Jupyter Labs in Docker
  • Your .ipynb files
  • A Python configuration file, config.py

These resources are combined into a zipped tarball, which is then used to build a Docker image later on.

First, let’s create our Dockerfile.

Create a Dockerfile for Python Jupyter

Note: You have to perform this step locally (on your computer).

  1. Create a Dockerfile by adding the following script to an empty text file saved as Dockerfile (with no extension):
# 20.04 version of ubuntu as base image
FROM    ubuntu:20.04
# Install the following packages
RUN     apt-get update && apt-get install software-properties-common -y &&\
        add-apt-repository ppa:deadsnakes/ppa && apt-get update &&\
        apt-get install python3.6 -y && apt install python3-pip -y &&\
        pip3 install --upgrade pip && pip3 install jupyter &&\
        mkdir /usr/local/notebooks
        
# Install any Python modules required in the notebook
RUN     pip3 install pandas && pip3 install numpy && pip3 install seaborn &&\
        pip3 install matplotlib && pip3 install sklearn
# Add configuration file
ADD     config.py /root/.jupyter/jupyter_notebook_config.py
# Add ipynb files
ADD     helloworld.ipynb /usr/local/notebooks/
The instructions in the Dockerfile are used to build a Docker image for your Jupyter notebooks

You can copy the above script into your Dockerfile.

Dockerfile Explained

Docker uses the instructions in your Dockerfile to build a Docker image. This image will be used to run your Jupyter notebooks. Let’s take a look at what our Dockerfile contains.

  • FROM ubuntu:20.04 - We are using the 20.04 version of ubuntu as base image of our Dockerfile.

  • RUN ... install software-properties-common -y

    Ubuntu uses the package manager apt. This allows you to easily install any packages or software required in your environment.

    • add-apt-repository ppa:deadsnakes/ppa

      1. add-apt-repository - This will add the specified apt repository to the list.
      2. ppa:deadsnakes/ppa - This will add the PPA deadsnakes repository, which contains the python3.6 package.

    • apt-get update - This will check the dependencies of the packages you want and install any that are needed.

    • apt-get install python3.6 - This will install Python 3.6.

    • install python3-pip -y - This will install the Python package manager pip which is needed in order to install JupyterLab.

    • pip3 install --upgrade pip - This will upgrade pip.

    • pip3 install jupyterlab - This will install the JupyterLab package for Python.

    • mkdir /usr/local/notebooks - This is the directory where you will add your ipynb files.

  • RUN pip3 ... install sklearn

    This will install any Python modules that you need in your Jupyter notebooks.

  • ADD config.py /root/.jupyter/jupyter_notebook_config.py

    This will add the configuration file for your notebooks. It contains preferences such as the port number (set to 8080) and the notebook directory (/usr/local/notebooks).

  • ADD helloworld.ipynb /usr/local/notebooks/

    This will add your ipynb files to your notebook directory. We have added helloworld.ipynb as an example.

We are now ready to compress our Dockerfile into a tarball, which will be used later on below.

Compress Your Files into a Tarball File

Next, you need to compress the Dockerfile, your .ipynb files, and the config.py file into a single tarball file. Here’s how:

  1. Use your command line terminal to navigate to the directory where your Dockerfile is
    • Move all the .ipynb files your course needs into this directory as well.
    • Also move the config.py file into this directory.
  1. In your terminal, use the following command to turn these assets into a tarball:
    tar -czvf jupyter.tar.gz Dockerfile helloworld.ipynb config.py

This command will take your Dockerfile, .ipynb files, and config.py file as inputs and create a single tarball file as output. The tarball file will be named jupyter.tar.gz.

Note: In our example, we are only using one .ipynb file: helloworld.ipynb.

For your convenience, the files are provided below. Please download them.

c.NotebookApp.port = 8080
c.NotebookApp.ip = '0.0.0.0' # listen on all IPs
c.NotebookApp.token = ''
c.NotebookApp.allow_origin = '*' #allow all origins
c.NotebookApp.notebook_dir = '/usr/local/notebooks'
c.NotebookApp.disable_check_xsrf = True
c.NotebookApp.tornado_settings = {
    'headers': {
        'Content-Security-Policy': "frame-ancestors https://*.appspot.com https://*.educative.io 'self' ;",
    }
}

Now that you have the files, create the tarball, upload it and configure your Docker job!

Upload the Tarball

Go to your course editor page. Navigate to the Docker Container section. It looks like this:

Click on Change Docker File (Beta). Then select the tarball jupyter.tar.gz.

Click on the Save button to start the docker image building process.

Wait for the Image to Build

Building an image will take a few minutes. If your image is built successfully, you will see the following message:

However, if your message build has failed, you will see the Image build failed message displayed instead.

If the build fails, check the build logs:

The logs will indicate where the building process failed. It is a great way to debug your Dockerfile.

If all else fails, send us a message at authors@educative.io.

Create the Docker Job

The next step is to configure a Jupyter Docker job—think of this as some extra Jupyter-specific settings that are needed for the LiveApp widget to work properly later.

After the image successfully builds, click the + sign in the Docker Container section. The screen will expand to show this:

Use the following settings:

Job Name

You can include as many Docker jobs in a course as you need. The Job Name acts as an ID to each job. In this example, we are setting up a job named jupyter.

Input File Name

In our Live VM functionality, this field can have any value in it. For demonstration purposes, let’s call it helloworld.

Run Script

The Run Script is the most important part of the job. This script is run every time the Live App is launched. In our case, we want to launch the helloworld.ipynb in Jupyter. Add the lines given below in your Run Script:

nohup jupyter notebook /usr/local/notebooks/helloworld.ipynb --allow-root --no-browser > /dev/null 2>&1 &

Run in Live Container

Check this box to enable LiveVM functionality.

Application Port

This port must match the port specified in config.py. In our case, it is 8080.

Start Script

The Start Script is not required for the Live App widget, but it cannot be left empty. Hence, a simple echo command should be fine:

echo "hello world"

Putting it All Together

Your final Jupyter job will look like this:

Our Docker setup is now ready! Let’s use it in a Live App

Use Jupyter Notebook in Live App

Everything we’ve done up to this point has been to embed a Python Jupyter Notebook in our lessons. As our last step, we need to insert a Live App widget in our lesson and choose the Docker job.

A blank Live App widget will now appear. You will also see two fields: App Entrypoint and Live Docker Job:

App Entrypoint

The App Entrypoint displays the URL of your container. From this URL, we must navigate to the ipynb file in our notebooks directory.

In our case, the required file is /notebooks/helloworld.ipynb. Add this at the end of the URL.

Now the URL will https://jy9lnm-live-app.educative.run/notebooks/helloworld.ipynb.

Note: For each ipynb file in your tarball, you will have to create a separate Docker job, and specify the file’s path in the App Entrypoint field, e.g, URL/notebooks.filename.ipynb.

Live Docker Job

In the Live Docker Job field, select the Docker job we created: jupyter.

Click the “Click to launch app!” 🚀 button and you will now see helloworld.ipynb running as a Python Jupyter notebook.

A Working Jupyter Notebook

A demo of our working Jupyter notebook can be found here.

Kernel Error

In your Jupyter Notebook setup, you may encounter a kernel error:

To fix this, you can follow the steps below.

  1. Update the config.py. Assign empty values to token and password. Set disable_check_xsrf equal to true:
c.NotebookApp.port = 8080
c.NotebookApp.ip = '0.0.0.0' # listen on all IPs
c.NotebookApp.token = ''
c.NotebookApp.password = ''
c.NotebookApp.allow_origin = '*' #allow all origins
c.NotebookApp.notebook_dir = '/usr/local/notebooks'
c.NotebookApp.disable_check_xsrf = True
c.NotebookApp.tornado_settings = {
    'headers': {
        'Content-Security-Policy': "frame-ancestors https://*.appspot.com https://*.educative.io 'self' ;",
    }
}
  1. Lock the JupyterLab version in the Dockerfile:
RUN pip3 install jupyterlab==3.0.1

Conclusion

Congratulations! You have now embedded a Python Jupyter Notebooks into your course by using our Live App widget and selecting a specific .ipynb file, which you uploaded during the Docker setup process.

Although we used the helloworld.ipynb notebook as an example, you can embed your own .ipynb notebook files in your lessons. Simply include the Jupyter notebook files you need in your tarball.

Then you can embed your notebooks, one per Live App widget, by running it in the Docker job’s Run Script and adding its path to the App Entrypoint URL (/notebooks/filename.ipynb).

Miscellaneous Notes

📝 Note: The Live App session is active only for 15 minutes. During that time, all files that were added in the tarball can be accessed. However, a new file can be created at run time, but this will reset once the session becomes inactive. The following activities on the file are session-dependent:

  1. The existence of a new file created at run time.
  2. Any change (update or delete) to the existing files in the tar, or to a new file that was created at run time.

If you have any trouble setting up your Python Jupyter notebooks, send us a message at authors@educative.io.