Accessing Jupyter notebooks in the cluster

From a (Linux/OSX) terminal ssh to snorlax-login.cs:

ssh user@snorlax-login.cs.uwaterloo.ca

Preliminaries: Create a conda environment that includes the jupyter server

Also add required conda packages for your working environment e.g. pytorch:

conda init
source ~/.bashrc
conda create -y -n jupyter-server
conda activate jupyter-server
conda install -c conda-forge pytorch-gpu
pip install jupyter
conda deactivate

Using jupyter notebooks with your environment

Make an interactive reservation with the SLURM scheduler:

salloc --gres=gpu:1 --cpus-per-task=4 --mem=32G --time=1:00:00

Once the reservation starts, ssh to the allocated compute system e.g. snorlax-2:

ssh snorlax-2

Activate your jupyter-server environment and start a jupyter notebook:

jupyter notebook --ip=$PUBLIC_IP --no-browser
...
[I 19:07:49.062 NotebookApp] Serving notebooks from local directory: /u1/tcauduro
[I 19:07:49.063 NotebookApp] Jupyter Notebook 6.5.4 is running at:
[I 19:07:49.063 NotebookApp] http://129.97.26.146:8888/?token=ffd03731fff0a09fe01eb3b536a3076127439be446095a62
[I 19:07:49.063 NotebookApp]  or http://127.0.0.1:8888/?token=ffd03731fff0a09fe01eb3b536a3076127439be446095a62
[I 19:07:49.063 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 19:07:49.072 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///u1/tcauduro/.local/share/jupyter/runtime/nbserver-105349-open.html
    Or copy and paste one of these URLs:
        http://129.97.26.146:8888/?token=ffd03731fff0a09fe01eb3b536a3076127439be446095a62
     or http://127.0.0.1:8888/?token=ffd03731fff0a09fe01eb3b536a3076127439be446095a62

Copy the link with the 8888 port to your broswer. Your juptyer notebook will be available.

Be sure to shut down the server when done with Control-c.

Custom kernels

There is a single default kernel at the moment: "Python 3". You can also create your own kernels by opening a Terminal inside the notebook:

Once you've opened the terminal you can create your own kernel. Below is an example:

conda create --name myenv # create a custom conda environment which will install packages to, and add to the notebook as a kernel

conda install --yes numpy # install a package you want

conda install -c anaconda ipykernel #install ipykernel which we will use to add kernel to notebook

python -m ipykernel install --user --name=myenv # add the conda environment as a kernel