Using Jupyter Notebook

Firstly, let us encourage you not to use Jupyter Notebook on the supercomputers. We suggest you use Jupyter Notebook on your own system during the development stage, then, once you’re ready to run it on big data, convert it to straight Python and run it on the supercomputers in the traditional manner. We are happy to help you get this setup.

However, we realize some of you will want to use Jupyter Notebooks despite our warnings, so below are instructions on how to do so.

First Time

First, send an email to assist@mcsr.olemiss.edu requesting permission to use Jupyter Notebooks. This can take up to one business day.

Configure jupyter notebook

Security

Each Time

We will actually be running the Jupyter Notebook server on Catalpa.

Each time Catalpa

qsub -I -l walltime=8:00:00 -l ncpus=1 -l mem=10gb

If you need more than 10gb of memory, or more than 1 CPU, modify the command as needed.

The walltime parameter means that your PBS session will be automatically killed after 8 hours. This is to insure that you don’t accidentally leave your Jupyter Notebook session running when it’s not in use. If you need it to run longer than eight hours, modify this parameter.

module load python

Start jupyter-notebook

jupyter-notebook --no-browser --ip=130.74.110.5

Example output

Note the port number

SSH Tunnel

From your local system:
ssh -N -L 8888:catalpa:8888 username@hpcwoods.olemiss.edu

Connect via browser: localhost:8888

Shutdown

When you’re done with your Jupyter Notebook session, start the shutdown process by closing your browser tab or window. Then

Then, in your Catalpa terminal window, press Control-C twice in quick succession to shutdown the Jupyter Notebook server.

Then, type “exit” to exit the interactive PBS session to release the CPUs and memory to be used by other users.

Finally, in your local terminal window, press Control-C to stop the SSH tunnel.