protonscr

[DXVK][Unigine Valley Benchmark DX11] Benchmark silently crashes due to memory leak

dxvkclosed
doitsujin/dxvk#3899 · opened 2024-03-08 by wensunvidia · updated 2024-04-09 · 6 comments · github
Wwensunvidia 2024-03-08 github

Please describe your issue as accurately as possible.

If you use Windows, please check the following page: https://github.com/doitsujin/dxvk/wiki/Windows

System information

Test PC configuration :
CPU: Intel 12th Gen i7-12700K 12 Core Processer 3.60 Ghz
System Memory: 32 GB DDR 4
GPU: RTX 4070
Drivers: 551.76 https://www.nvidia.com/download/driverResults.aspx/221876/en-us/
OS: WIN 11 22H2 SV2
Mother Board: Asus Strix Z690-F Gaming WIFI
Monitor: Acer 4K GSYNC Monitor

Software information

Repro steps:

  1. Install Win 11 SV2.
  2. Install driver 551.76
  3. Get the latest DXVK dll files from https://github.com/doitsujin/dxvk/releases
  4. Install Unigine valley application https://benchmark.unigine.com/valley
  5. Now copy all DXVK dll files present under x32 folder to the path where application exe is present - C:\Program Files (x86)\Unigine\Valley Benchmark 1.0\bin
  6. Set the system environment variable: DXVK_HUD=1
  7. Now launch the app and run it in 'DirectX 11' mode.
  8. While benchmark is running minimize the benchmark with ALT+TAB & wait for a minute and observe the issue.

Observation:
Benchmark silently crashes pointing to nvoglv32.dll while minimized when DXVK dll's are loaded.

  • Wine version: N/A test on Windows
  • DXVK version: 2.3

Isolation

-no repro from dxvk-1.10.3

Apitrace file(s)

Cannot use trace file because the mermory leak takes time and too many frames.

For instructions on how to use apitrace, see: https://github.com/doitsujin/dxvk/wiki/Using-Apitrace

KK0bin maintainer 2024-03-08 github

This appears to be caused by descriptor sets. When you tab back in the number of descriptor sets is extremely high for a second (600k) before it is cut back down (20k). When the game is visible, the number of descriptor sets merely grows to 35k.
It also seems to keep growing when it has focus but at a much slower pace.

KK0bin maintainer 2024-03-08 github

https://drive.google.com/file/d/1rGDzerAZUvyLgOz8LiHX1GnLdgTykf7S/view?usp=sharing
Apitrace of the minimized game. It can reproduce some level of descriptor set growth but I don't know if that's just the usual growth or if it reproduces the problem here.

KK0bin maintainer 2024-03-08 github

This isn't actually leaking. It maxes out at 1.1GB on my system. The game keeps rendering when minimized but doesn't actually present.

The problem is that DXVKs heuristic of when to reset descriptor pools when the game isn't presenting aren't aggressive enough.

24576 sets * 8 pools * n active command lists

So if there are 3 command lists in flight, we get the 600k descriptor sets I saw earlier.

Wwensunvidia 2024-03-08 github

I agree it is not a leak, but the memory usage goes high enough to cause crashes. The peak memory usage varies per system. On the reported system the crash is due to memory allocation failure. At the time of the failure, msvc debugger reports 1.4GB used. The large allocations are from vkCreateDescriptorPool with the following.

  •    createInfo    {sType=VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO (0x00000021) pNext=0x00000000 flags=0x00000000 ...}    const VkDescriptorPoolCreateInfo &
      sType    VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO (0x00000021)    VkStructureType
      pNext    0x00000000    const void *
      flags    0x00000000    unsigned int
      maxSets    0x00006000    unsigned int
      poolSizeCount    0x00000008    unsigned int
    
Ddoitsujin maintainer 2024-03-09 github

The problem is that DXVKs heuristic of when to reset descriptor pools when the game isn't presenting aren't aggressive enough.

This is why I generally don't like relying on frame boundaries for anything, however for any sort of per-frame resource reuse it's the only thing we can rely on. We also need numbers to be high for real games to consistently hit the fast path, we've seen >100k descriptor sets used per frame in the wild.

It's not really surprising that this falls apart in a 32-bit app that a) doesn't present and b) likely runs into a CPU runaway scenario where even the number of queued frames might end up quite high, it's just a really dumb thing to do that makes things unnecessarily difficult for us. We could dramatically tune things down on 32-bit builds but there's always a risk that this slows real games down to a crawl because we're constantly creating and destroying new descriptor pools (think GTA 4 with >20k draw calls per frame).

Descriptor buffers aren't viable for DXVK unfortunately, so descriptor memory allocation is always going to be orders of magnitude higher than what the app would actually need.

Ddoitsujin maintainer 2024-04-08 github

I adjusted some numbers, and most importantly, removed the part of the heuristic that required us to spam 16 frames worth of descriptors into one pool before resetting. Wasn't that useful to begin with, but the effects on apps like this were rather problematic.

Hopefully this works better now, even though the underlying problem still exists.

Launch options

DLLs