protonscr

Nvidia Xid 31 and GPU timeouts in Remnant II

vkd3dclosed
HansKristian-Work/vkd3d-proton#1652 · opened 2023-08-12 by Saancreed · updated 2023-09-06 · 9 comments · github
SSaancreed 2023-08-12 github

After some random amount of time playing Remnant II the game suddenly freezes with messages like:

Aug 10 01:04:13 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=639388, name=GameThread, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_6b400000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Aug 10 01:09:36 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=641659, name=GameThread, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_b4000000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Aug 10 01:16:29 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=642795, name=GameThread, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_ce800000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Aug 11 00:08:36 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=950975, name=Remnant2-Win64-, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_aa1c0000. Fault is of type FAULT_PTE ACCESS_TYPE_VIRT_READ
Aug 11 20:46:15 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=1006235, name=Remnant2-Win64-, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_79a00000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Aug 12 15:13:10 strixhaven kernel: NVRM: Xid (PCI:0000:01:00): 31, pid=1049859, name=Remnant2-Win64-, Ch 0000000e, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_17a00000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ

… which then angers kwin and i915 to the point where my entire desktop session temporarily freezes (possibly due to VRR attempting to sync display's refresh rate to the game that just hung):

Aug 10 01:04:23 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:496b timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:04:34 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:496b timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:04:45 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:14b84e timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:04:45 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:496b timed out (hint:submit_notify [i915])
Aug 10 01:09:47 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:2b21 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:09:57 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:2b21 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:10:09 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:153668 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:10:09 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:2b22 timed out (hint:submit_notify [i915])
Aug 10 01:16:39 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:4c84 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:16:50 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:4c85 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:17:01 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:157dc8 timed out (hint:intel_atomic_commit_ready [i915])
Aug 10 01:17:01 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:4c85 timed out (hint:submit_notify [i915])
Aug 11 00:08:46 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:5147ca timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 00:08:46 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:3599 timed out (hint:submit_notify [i915])
Aug 11 00:08:57 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:5147d6 timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 00:08:57 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:359a timed out (hint:submit_notify [i915])
Aug 11 20:46:25 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:9dabfa timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 20:46:25 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:1a981 timed out (hint:submit_notify [i915])
Aug 11 20:46:36 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:9dabfc timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 20:46:36 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:1a981 timed out (hint:submit_notify [i915])
Aug 11 20:46:47 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:9dac0c timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 20:46:47 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:1a981 timed out (hint:submit_notify [i915])
Aug 11 20:47:31 strixhaven kernel: Asynchronous wait on fence 0000:00:02.0:kwin_wayland[1308]:9dc28a timed out (hint:intel_atomic_commit_ready [i915])
Aug 11 20:47:31 strixhaven kernel: Asynchronous wait on fence NVIDIA:nvidia.prime:1a981 timed out (hint:submit_notify [i915])

Software information

Remnant II, settings around High preset with DLSS Quality enabled, borderless fullscreen on KDE Wayland session and VKD3D_DISABLE_EXTENSIONS=VK_EXT_mesh_shader

System information

  • GPU: GeForce RTX 4080 Mobile
  • Driver: 535.98
  • Wine version: Proton experimental-8.0-20230810b
  • VKD3D-Proton version: 87cc08bf44ca4a1, debug build with breadcrumbs enabled

Log files

steam-1282100_1.log
steam-1282100_2.log

EDIT: and one more for good measure: steam-1282100_3.log

Rrunar-work 2023-08-15 github

Did you happen to get any breadcrumb traces from those freezes?

I had one Xid 31 with a GTX 1660 Ti after about four minutes the first time, but when I tried again with breadcrumbs I wasn't able to reproduce it after more than an hour of trying, so it does seem very random. I used the same vkd3d build, though without the mesh shader disabling.

NVRM: Xid (PCI:0000:08:00): 31, pid=14317, name=gstglcontext, Ch 000000b9, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_HOST faulted @ 0x0_97800000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Rrunar-work 2023-08-15 github

Oh, nevermind, I see they were part of the proton logs. I'll see if I can reproduce with breadcrumbs and with the extension disabled.

Rrunar-work 2023-08-15 github

It was a fast one this time, and I got the same region in my trace:

0228:err:vkd3d_breadcrumb_tracer_report_command_list:  ===== Potential crash region BEGIN (make sure RADV_DEBUG=syncshaders is used for maximum accuracy) =====
0228:err:vkd3d_breadcrumb_tracer_report_command_list:   Command: resolve_query
0228:err:vkd3d_breadcrumb_tracer_report_command_list:  ===== Potential crash region END =====

Full log: remnant2_nv_xid31_breadcrumbs.txt

HHansKristian-Work maintainer 2023-08-17 github

@runar-work Just merged something which should make it easier to figure out what could go wrong here.

VKD3D_DESCRIPTOR_QA_LOG=/tmp/qa.txt VKD3D_CONFIG=breadcrumbs should now report more information.

E.g.:

1be73:info:d3d12_command_queue_submission_worker_main: === Executing command list 1 (context 2) on VkQueue 0x56339ad25b60, queue family 0 ===
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:      Tag: QueryResolve [Type, StartIndex, QueryCount, DstOffset, QueryHeapCookie, DstBuffer]
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 0 (#0)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 0 (#0)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 1 (#1)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 0 (#0)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 9 (#9)  <-- Cookie for heap
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:      Tag: BufferDesc [VkBuffer VA, SuballocatedOffset, Cookie, GlobalCookie, Size, Flags]
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 18446603340537135104 (#ffff800101400000)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 0 (#0)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 10 (#a)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 11 (#b)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 8 (#8)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:  Set arg: 8 (#8)
1be73:err:vkd3d_breadcrumb_tracer_report_command_list:   Command: resolve_query
1be73:info:d3d12_command_queue_submission_worker_main: ============================

and in QA log.

VIEW CREATE #1 <- RESOURCE #0
VIEW CREATE #2 <- RESOURCE #0
COOKIE DESTROY #1
COOKIE DESTROY #2
VIEW CREATE #3 <- RESOURCE #0
VIEW CREATE #4 <- RESOURCE #0
REGISTER HEAP 5 || COUNT = 1 || RTV
RESOURCE CREATE #6 || Tex2D || Format = DXGI_FORMAT_R8G8B8A8_UNORM || Levels = 1 || Layers = 1 || Width = 32 || Height = 32 || RTV
RESOURCE CREATE #7 || Buffer || Size = 0x1000000 bytes
VIEW CREATE #8 <- RESOURCE #6
VIEW CREATE #8 <- RESOURCE #6
QUERY HEAP CREATE #9 ||  || OCCLUSION || Count = 1
RESOURCE CREATE [#10](/issue/HansKristian-Work/vkd3d-proton/10) || Buffer || Size = 0x8 bytes
RESOURCE CREATE [#11](/issue/HansKristian-Work/vkd3d-proton/11) || Buffer || Size = 0x1000000 bytes
RESOURCE CREATE [#12](/issue/HansKristian-Work/vkd3d-proton/12) || Buffer || Size = 0x100000 bytes
COOKIE DESTROY #9
COOKIE DESTROY [#10](/issue/HansKristian-Work/vkd3d-proton/10)
COOKIE DESTROY [#11](/issue/HansKristian-Work/vkd3d-proton/11)
DESTROY HEAP 5
COOKIE DESTROY #8
COOKIE DESTROY #6
COOKIE DESTROY #7
COOKIE DESTROY [#12](/issue/HansKristian-Work/vkd3d-proton/12)
COOKIE DESTROY #3
COOKIE DESTROY #4
Rrunar-work 2023-08-17 github

A quick repro for once:

NVRM: Xid (PCI:0000:08:00): 31, pid=36996, name=gstglcontext, Ch 000000f9, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_HOST faulted @ 0x0_3f800000. Fault is of type FAULT_PTE ACCESS_TYPE_VIRT_READ

Log files with the extra debug info:
remnant2_xid31_breadcrumbs_qa.txt
qa_xid31.txt

HHansKristian-Work maintainer 2023-08-21 github

Should be fixed now on master.

Rrunar-work 2023-08-22 github

Got another Xid 31 just now with https://github.com/HansKristian-Work/vkd3d-proton/commit/f7ee2a0aaaad0c22fa7132180db5b4c131d3af7e

NVRM: Xid (PCI:0000:08:00): 31, pid=5438, name=gstglcontext, Ch 00000101, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_HOST faulted @ 0x0_74c00000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Rrunar-work 2023-08-22 github

Here's with the RTX 4070, built from https://github.com/HansKristian-Work/vkd3d-proton/commit/72c01d738b1f0d046133e8a53daa772423cfd6ca:

NVRM: Xid (PCI:0000:08:00): 31, pid=20728, name=gstglcontext, Ch 00000129, intr 00000000. MMU Fault: ENGINE HOST0 HUBCLIENT_ESC faulted @ 0x0_41400000. Fault is of type FAULT_PTE ACCESS_TYPE_VIRT_READ

remnant2_Xid31_2.txt
qa_xid31_2.txt

SSaancreed 2023-09-06 github

Should be fixed, for real this time, in #1689. I'll reopen if I encounter this again.

Proton versions

Launch options