Using Jupyter Notebook
Firstly, let us encourage you not to use Jupyter Notebook on the supercomputers. We suggest you use Jupyter Notebook on your own system during the development stage, then, once you’re ready to run it on big data, convert it to straight Python and run it on the supercomputers in the traditional manner. We are happy to help you get this setup.
However, we realize some of you will want to use Jupyter Notebooks despite our warnings, so below are instructions on how to do so.
First, send an email to firstname.lastname@example.org requesting permission to use Jupyter Notebooks. This can take up to one business day.
Configure jupyter notebook
We will actually be running the Jupyter Notebook server on Catalpa.
Each time Catalpa
qsub -I -l walltime=8:00:00 -l ncpus=1 -l mem=10gb
If you need more than 10gb of memory, or more than 1 CPU, modify the command as needed.
The walltime parameter means that your PBS session will be automatically killed after 8 hours. This is to insure that you don’t accidentally leave your Jupyter Notebook session running when it’s not in use. If you need it to run longer than eight hours, modify this parameter.
module load python
jupyter-notebook --no-browser --ip=22.214.171.124
Note the port number
From your local system:
ssh -N -L 8888:catalpa:8888 email@example.com
Connect via browser: localhost:8888
When you’re done with your Jupyter Notebook session, start the shutdown process by closing your browser tab or window. Then
Then, in your Catalpa terminal window, press Control-C twice in quick succession to shutdown the Jupyter Notebook server.
Then, type “exit” to exit the interactive PBS session to release the CPUs and memory to be used by other users.
Finally, in your local terminal window, press Control-C to stop the SSH tunnel.