Codespace unable to start in Jupyter using python feature with option "installJupyterLab": true

[prabhakk-mw]

Listed below is the devcontainer.json file that I am using to start a codespace on GitHub.

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/base:ubuntu",
  "hostRequirements": {
    "cpus": 4
  },
  "features": {
    "ghcr.io/devcontainers/features/python": {
      "version": "3.10",
      "installJupyterlab": true,
      "configureJupyterlabAllowOrigin": "*"
    }
  },
  "containerUser": "vscode"
}

When attempting to open in Jupyter as shown in the image below:

I get the following error:

The documentation for opening-your-codespace-in-jupyterlab seems to suggest that this should be all I need to do.

Restarting the codespace as the option suggests has no effect.

I can however, open in VSCode, and start jupyter lab from the terminal and access it in a separate browser tab.

Any help would be much appreciated!

[prabhakk-mw]

Tagging for visibility: @jungaretti @samruddhikhandale

[samruddhikhandale]

Hi 👋

Can you try with the following .devcontainer.json?

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/universal:2",
  "hostRequirements": {
    "cpus": 4
  }
}

I vaguely remember there needs another configuration change (when using Python Feature) which doesn't seem to be documented.

[prabhakk-mw]

Thank you @samruddhikhandale,

Yes, the "mcr.microsoft.com/devcontainers/universal:2" image opens with the Jupyterlab editor. Probably because it already has Jupyter installed within it.

This image works with JupyterLab, with or without the python feature.

However, my goal is to use features to introduce the Python & JupyterLab into an image that didn't already have it; and then use that image to start in Codespace's Jupyter Editor.

The "mcr.microsoft.com/devcontainers/universal:2" image is over 13.7GB, way more than what I need in my containers.

Any other thoughts?

Could it be because of the containerUser specification?

[jungaretti]

Is this a problem with the $PATH variable in the base image?

JupyterLab is installed in the $HOME/.local/bin/ directory. If I remember correctly, this directory is only included in $PATH for login shells.

We hardcode this value into the non-login $PATH in the universal image, but not in other images: https://github.com/devcontainers/images/blob/9bb5c4f0540a9a54b0d4ea4151b737c7af2f8aec/src/universal/.devcontainer/local-features/setup-user/install.sh#L92

[prabhakk-mw]

Surprisingly, These problematic containers do have jupyter available on path As shown below:

@prabhakk-mw ➜ /workspaces/matlab-codespaces (mybranch) $ which jupyter
/home/vscode/.local/bin/jupyter

@prabhakk-mw ➜ /workspaces/matlab-codespaces (mybranch) $ python -m pip show jupyterlab | grep -i location
Location: /home/vscode/.local/lib/python3.10/site-packages

I can even start JuptyerLab in the terminal and click on the link to view the Lab console, albeit with a refresh of the vscode interface:

prabhakk-mw ➜ /workspaces/matlab-codespaces (deleteme) $ jupyter lab
[I 2024-02-23 09:45:25.096 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-02-23 09:45:25.100 ServerApp] jupyter_server_mathjax | extension was successfully linked.
[I 2024-02-23 09:45:25.102 ServerApp] jupyter_server_terminals | extension was successfully linked.
[W 2024-02-23 09:45:25.104 LabApp] 'allow_origin' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2024-02-23 09:45:25.106 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-02-23 09:45:25.107 ServerApp] jupyterlab_git | extension was successfully linked.
[I 2024-02-23 09:45:25.107 ServerApp] nbdime | extension was successfully linked.
[I 2024-02-23 09:45:25.108 ServerApp] Writing Jupyter server cookie secret to /home/vscode/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-02-23 09:45:25.322 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-02-23 09:45:25.336 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-02-23 09:45:25.338 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-02-23 09:45:25.338 ServerApp] jupyter_server_mathjax | extension was successfully loaded.
[I 2024-02-23 09:45:25.339 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-02-23 09:45:25.340 LabApp] JupyterLab extension loaded from /home/vscode/.local/lib/python3.10/site-packages/jupyterlab
[I 2024-02-23 09:45:25.341 LabApp] JupyterLab application directory is /home/vscode/.local/share/jupyter/lab
[I 2024-02-23 09:45:25.341 LabApp] Extension Manager is 'pypi'.
[I 2024-02-23 09:45:25.370 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-02-23 09:45:25.374 ServerApp] jupyterlab_git | extension was successfully loaded.
[I 2024-02-23 09:45:25.412 ServerApp] nbdime | extension was successfully loaded.
[I 2024-02-23 09:45:25.413 ServerApp] Serving notebooks from local directory: /workspaces/matlab-codespaces
[I 2024-02-23 09:45:25.413 ServerApp] Jupyter Server 2.12.5 is running at:
[I 2024-02-23 09:45:25.413 ServerApp] http://localhost:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
[I 2024-02-23 09:45:25.413 ServerApp]     http://127.0.0.1:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
[I 2024-02-23 09:45:25.413 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-02-23 09:45:25.417 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///home/vscode/.local/share/jupyter/runtime/jpserver-2104-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
        http://127.0.0.1:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959

Clicking on the last link does open up JuptyerLab, but still requires me to copy-paste the token into the authentication page that is presented.

Pasting the token from VSCode terminal lets me through.

The exact same image fails when you attempt to open it through

Does this error message at the bottom of the presented screen mean anything to you guys?
Do you know of anyone on the codespace team that could help?

[samruddhikhandale]

Is this a problem with the $PATH variable in the base image?

JupyterLab is installed in the $HOME/.local/bin/ directory. If I remember correctly, this directory is only included in $PATH for login shells.

We hardcode this value into the non-login $PATH in the universal image, but not in other images: https://github.com/devcontainers/images/blob/9bb5c4f0540a9a54b0d4ea4151b737c7af2f8aec/src/universal/.devcontainer/local-features/setup-user/install.sh#L92

Thanks @jungaretti for pointing it out, and yes ^ this is the dev container configuration change needed for a codespace to be opened with JupyeterLab.

@prabhakk-mw Can you add a similar change with :/home/${USERNAME}/.local/bin added to PATH in your Dockerfile? Let us know if that helps.

Looping in Codespaces PM @craiglpeters, this is a known gap ^ in our documents where the Python Feature don't work without the PATH change. We should update our docs to reflect it until this gets fixed in the code. (Last we check, we ran into multiple issues to fix it on Codespaces end, hence, the doc updates should help avoid such confusions in the meanwhile)

[jungaretti]

Surprisingly, These problematic containers do have jupyter available on path

@prabhakk-mw It looks like you are using a login shell that adds the $HOME/.local/bin/ to your PATH variable. This directory is scoped to a certain user (with $HOME) and relies on a login shell. To accurately reproduce this bug, you need to use a non-login shell.

You can use docker run in a Codespace to test this.

base Image

Command: docker run --rm mcr.microsoft.com/devcontainers/base:ubuntu /bin/sh -c 'echo $PATH'

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

/home/vscode/.local/bin is missing from this PATH variable.

universal:2 image

Command: docker run --privileged --rm mcr.microsoft.com/devcontainers/universal:2 /bin/sh -c 'echo $PATH'

/home/codespace/.dotnet:/home/codespace/nvm/current/bin:/home/codespace/.php/current/bin:/home/codespace/.python/current/bin:/home/codespace/java/current/bin:/home/codespace/.ruby/current/bin:/home/codespace/.local/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/oryx:/usr/local/go/bin:/go/bin:/usr/local/sdkman/bin:/usr/local/sdkman/candidates/java/current/bin:/usr/local/sdkman/candidates/gradle/current/bin:/usr/local/sdkman/candidates/maven/current/bin:/usr/local/sdkman/candidates/ant/current/bin:/usr/local/rvm/gems/default/bin:/usr/local/rvm/gems/default@global/bin:/usr/local/rvm/rubies/default/bin:/usr/local/share/rbenv/bin:/usr/local/php/current/bin:/opt/conda/bin:/usr/local/nvs:/usr/local/share/nvm/current/bin:/usr/local/hugo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/share/dotnet:~/.dotnet/tools

This PATH variable contains /home/codespace/.local/bin and works as expected.

[prabhakk-mw]

@samruddhikhandale , Thank you for your suggestions.
Unfortunately, my usecase doesn't have a Dockerfile that I can update.

@jungaretti, Thank you for the clarification.
It does feel very quirky that the Codespaces infrastructure does not check against the specified containerUsers PATH before declaring that it is inaccessible.
Having users place binaries from their HOME folders for all possible users of a container also feels uncomfortable.

However, I've taken a pass at fixing the issue at the feature level in #887
Thoughts and suggestions are welcomed!