JupiniumStartAndAttachToServer doesn't open Jupyter in Firefox; Jupynium not found but is installed

[squishram]

The Bug
I apologise if I've made an obvious mistake here but I can't get Jupynium working and it looks like such a fantastic solution. When I run :JupiniumStartAndAttachToServer nothing happens; when I run :JupiniumStartAndAttachToServerInTerminal Jupynium is not found, yet I'm pretty sure I have all the dependencies. I am using Linux.

To Reproduce
Steps to reproduce the behavior:

1. Install Jupynium with lazy.nvim:

return {
  "kiyoon/jupynium.nvim",
  build = "pip3 install --user .",
  ft = { "python" },
}

Loading on python files including .ju.py files works as expected

2. Install jupyter-notebook, jupyper-nbclassic, firefox, geckodriver

sudo pacman -S jupyter-notebook jupyper-nbclassic firefox geckodriver

3. open main.ju.py; run

:JupiniumStartAndAttachToServer

4. nothing happens; :JupyniumStartSync not available

Expected behaviour
Expected Jupyter to open and :JupyniumStartSync to be an available option

Logs in /tmp/jupynium/logs/
/tmp/jupynium does not exist
:JupyniumStartAndAttachToServerInTerminal returns:

No module named 
jupynium

Output of jupynium --version

command not found: jupynium

Output of nvim --version

NVIM v0.9.4
Build type: Release
LuaJIT 2.1.1696795921

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/share/nvim"

Run :checkhealth for more info

[kiyoon]

Go to the jupynium installed directory and run the build command (pip3 ...) manually, and can you let me know if jupynium command exists?

[squishram]

Do you mean go to ~/.local/share/nvim/lazy/jupynium.nvim/ and run pip3 install --user .`?
I get

error: externally-managed-environment

I get the same problems whether I attempt to use jupynium in a virtual environment (with jupyter notebook installed via pip) or outside of one (where jupyter notebook is installed with pacman)

Thanks for getting back to me so quickly!

[kiyoon]

Yes, that's what I meant.

So you have a problem with pip3 command, and you need to solve that first. I don't use Arch linux so I can't be much help, but there seems to be lots of solutions on StackOverflow.

[fecet]

I'm using arch linux but I think that's not a problem related to jupynium.
I think the error: externally-managed-environment means you are trying to install jupynium with the system python, is this the full error output?

[squishram]

To others with this problem, the solution is to use

sudo mv /usr/lib/python3.11/EXTERNALLY-MANAGED /usr/lib/python3.11/EXTERNALLY-MANAGED.old

which removes the limitations on using externally managed packages -- this will likely be an issue for people that use their system package manager for python packages as opposed to pip with virtual environments (note: this is somewhat ill-advised because of breaking changes to packages as they evolve, but it's so less faff and wasted hard disk space that I don't mind)

[squishram]

I'm using arch linux but I think that's not a problem related to jupynium. I think the error: externally-managed-environment means you are trying to install jupynium with the system python, is this the full error output?

Yes I think this is the problem. If you know of any more sensible solutions than the one I posted above I'd be keen to hear! :)

[fecet]

I'm using arch linux but I think that's not a problem related to jupynium. I think the error: externally-managed-environment means you are trying to install jupynium with the system python, is this the full error output?

Yes I think this is the problem. If you know of any more sensible solutions than the one I posted above I'd be keen to hear! :)

you dont have to install jupynium in the system python，to make jupynium work，all you need is make it appear in PATH. So you can install it with pipx or install in any venv and append the venv in PATH.

For me, I install it on the base conda env.

[BillGatesPriv]

I have got it working using some bash scripting:
I didn't want to install conda or remove the pip venv restriction on arch.

The relevant code is in server.lua:

  args = utils.table_concat({ "-m", "jupynium", "--nvim_listen_addr", vim.v.servername }, args)

  local cmd
  if type(options.opts.python_host) == "string" then
    cmd = options.opts.python_host
  elseif type(options.opts.python_host) == "table" then
    cmd = options.opts.python_host[1]
    args = utils.table_concat({ unpack(options.opts.python_host, 2) }, args)
  else
    error "Invalid python_host type."
  end

You can build a bash string that skips the $-m$ argument to call jupynium directly.
To do this, you pass the following bash in the python_host setup variable:

python_host = {"bash", "-c", "shift; \"$@\"", "_"},

If your package(s) are called differently than what is expected, you may still need create a link with the required name and put it somewhere in the PATH env.

Hope that works not only on my system 👍

[kiyoon]

@BillGatesPriv Thanks for sharing. If you want to install it in venv, I think simply you can provide the path to the venv python, or am I getting something wrong?

But I guess some people use pipx etc. and want to call jupynium command directly.

[BillGatesPriv]

@kiyoon Yeah no, you are right. I overread that this was also proposed.

I think its a good solution to do it with the venv. It is less intertwined with the system packages and probably can be setup quite well from the nvim config.

I guess the only argument against it would be that it still feels like a workaround to the restrictions placed on pip in Arch Linux to avoid conflicts between system packages and pip-installed packages. (Altough I never had any problems with it)