You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm having the same issue here with virt-manager hanging on creating domain.
dmesg is reporting: NVRM: Attempting to remove device 0000:01:00.0 with non-zero usage count!
I have changed the graphics mode to compute as recommended in issue #12. nvidia-smi indicates that there are no processes running on the Nvidia GPU, but still this issue with a non-zero usage count.
The IO information for the GPU is IOMMU Group 2 01:00.0 3D controller [0302]: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] [10de:1f95] (rev a1)
I have added the scripts bind_vfio.sh & unbind_vfio.sh, and made them executable.
The contents of bind_vfio.sh are:
#!/bin/bash
## Load the config file
source "/etc/libvirt/hooks/kvm.conf"
## Load vfio
modprobe vfio
modprobe vfio_iommu_type1
modprobe vfio_pci
## Unbind gpu from nvidia and bind to vfio
virsh nodedev-detach $VIRSH_GPU_VIDEO
The contents of unbind_vfio.sh are:
#!/bin/bash
## Load the config file
source "/etc/libvirt/hooks/kvm.conf"
## Unbind gpu from vfio and bind to nvidia
virsh nodedev-reattach $VIRSH_GPU_VIDEO
## Unload vfio
modprobe -r vfio_pci
modprobe -r vfio_iommu_type1
modprobe -r vfio
kvm.conf contents are simply:
VIRSH_GPU_VIDEO=pci_0000_01_00_0
Where to go from here?
Thanks
P.S. I saw that newer drivers could cause issues so downgraded drivers to nvidia-470 and tried again. Still hanging at the same point. dmesg:
NVRM: GPU at PCI:0000:01:00: GPU-832b7f23-880e-d656-20c0-331ca6c8873a
[ 109.630956] NVRM: Xid (PCI:0000:01:00): 79, pid=3465, GPU has fallen off the bus.
[ 109.630958] NVRM: GPU 0000:01:00.0: GPU has fallen off the bus.
[ 243.446680] VFIO - User Level meta-driver version: 0.3
[ 243.464970] NVRM: Attempting to remove minor device 0 with non-zero usage count!
P.P.S I have further information, by adding set -x at the beginning of bind_vfio.sh we have further logs in dmesg:
[ 26.485239] nvidia 0000:01:00.0: not ready 8191ms after resume; waiting
[ 34.933241] nvidia 0000:01:00.0: not ready 16383ms after resume; waiting
[ 52.597123] nvidia 0000:01:00.0: not ready 32767ms after resume; waiting
[ 87.412701] nvidia 0000:01:00.0: not ready 65535ms after resume; giving up
[ 87.412759] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 87.473074] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.473112] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.473413] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 87.473638] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.473669] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.473846] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 87.473949] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.473980] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.474025] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 87.474119] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.474148] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.474309] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 87.474426] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.474457] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.474603] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.474632] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.474912] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.474937] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 87.475108] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 87.475137] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 145.364413] nvidia 0000:01:00.0: not ready 1023ms after resume; waiting
[ 146.420419] nvidia 0000:01:00.0: not ready 2047ms after resume; waiting
[ 148.596426] nvidia 0000:01:00.0: not ready 4095ms after resume; waiting
[ 152.948405] nvidia 0000:01:00.0: not ready 8191ms after resume; waiting
[ 161.396346] nvidia 0000:01:00.0: not ready 16383ms after resume; waiting
[ 179.572264] nvidia 0000:01:00.0: not ready 32767ms after resume; waiting
[ 214.387968] nvidia 0000:01:00.0: not ready 65535ms after resume; giving up
[ 214.388023] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.410562] VFIO - User Level meta-driver version: 0.3
[ 214.429107] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.429128] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.429312] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.429326] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.429365] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 214.552809] audit: type=1400 audit(1706607632.349:51): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-aab4928c-15e6-4b1b-aab1-de4fcd2b24af" pid=4628 comm="apparmor_parser"
[ 214.681154] audit: type=1400 audit(1706607632.477:52): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-aab4928c-15e6-4b1b-aab1-de4fcd2b24af" pid=4631 comm="apparmor_parser"
[ 214.819618] audit: type=1400 audit(1706607632.613:53): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="libvirt-aab4928c-15e6-4b1b-aab1-de4fcd2b24af" pid=4640 comm="apparmor_parser"
[ 216.980015] vfio-pci 0000:01:00.0: not ready 1023ms after resume; waiting
[ 218.035803] vfio-pci 0000:01:00.0: not ready 2047ms after resume; waiting
[ 220.275818] vfio-pci 0000:01:00.0: not ready 4095ms after resume; waiting
[ 224.627829] vfio-pci 0000:01:00.0: not ready 8191ms after resume; waiting
[ 233.075664] vfio-pci 0000:01:00.0: not ready 16383ms after resume; waiting
[ 251.251461] vfio-pci 0000:01:00.0: not ready 32767ms after resume; waiting
[ 286.067103] vfio-pci 0000:01:00.0: not ready 65535ms after resume; giving up
[ 286.067158] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.067542] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.137034] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.137047] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.137052] vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.137195] nvidia 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[ 286.137281] NVRM: This is a 64-bit BAR mapped above 4GB by the system
NVRM: BIOS or the Linux kernel, but the PCI bridge
NVRM: immediately upstream of this GPU does not define
NVRM: a matching prefetchable memory window.
[ 286.137283] NVRM: This may be due to a known Linux kernel bug. Please
NVRM: see the README section on 64-bit BARs for additional
NVRM: information.
[ 286.137284] nvidia: probe of 0000:01:00.0 failed with error -1
Virt manager now exits cleanly with an error, without hanging:
Unable to complete install: 'internal error: Unknown PCI header type '127' for device '0000:01:00.0''
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/createvm.py", line 2008, in _do_async_install
installer.start_install(guest, meter=meter)
File "/usr/share/virt-manager/virtinst/install/installer.py", line 695, in start_install
domain = self._create_guest(
File "/usr/share/virt-manager/virtinst/install/installer.py", line 637, in _create_guest
domain = self.conn.createXML(initial_xml or final_xml, 0)
File "/usr/lib/python3/dist-packages/libvirt.py", line 4400, in createXML
raise libvirtError('virDomainCreateXML() failed')
libvirt.libvirtError: internal error: Unknown PCI header type '127' for device '0000:01:00.0'
The text was updated successfully, but these errors were encountered:
I had a similar issue and in my case i was using the graphic card for rendering in my host so it was not getting released, i had to use the IGPU for rendering and let my dGPU idle so that it can be available
Hi,
I'm having the same issue here with virt-manager hanging on
creating domain
.dmesg is reporting:
NVRM: Attempting to remove device 0000:01:00.0 with non-zero usage count!
I have changed the graphics mode to compute as recommended in issue #12.
nvidia-smi
indicates that there are no processes running on the Nvidia GPU, but still this issue with a non-zero usage count.The IO information for the GPU is
IOMMU Group 2 01:00.0 3D controller [0302]: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] [10de:1f95] (rev a1)
I have added the scripts
bind_vfio.sh
&unbind_vfio.sh
, and made them executable.The contents of
bind_vfio.sh
are:The contents of unbind_vfio.sh are:
kvm.conf
contents are simply:Where to go from here?
Thanks
P.S. I saw that newer drivers could cause issues so downgraded drivers to nvidia-470 and tried again. Still hanging at the same point. dmesg:
P.P.S I have further information, by adding
set -x
at the beginning of bind_vfio.sh we have further logs in dmesg:Virt manager now exits cleanly with an error, without hanging:
The text was updated successfully, but these errors were encountered: