Ori Will of the Wisps: low fps/GPU utilisation on NVIDIA

dxvkclosed performancenot our bug

doitsujin/dxvk#3153 · opened 2022-12-28 by w0utert · updated 2023-04-03 · 12 comments · github

Ww0utert 2022-12-28 github

Running Ori: Will of the Wisps through DXVK/Proton yields very low (below ~30) fps and low (~20%) GPU utilization on RTX3080/Ryzen 5950X.

I've tried many, many things to find and debug this, because the game runs perfectly smooth at a locked 60fps on my Steam Deck, which is much less powerful. In the end I turned to enabling the Vulkan validation layer and debug logging, and found the game will spam the log with 100s of validation errors similar to this one, before hard-locking the machine (probably because of the amount of logging):

err:   UNASSIGNED-CoreValidation-Shader-InterfaceTypeMismatch: 
err:   Validation Error: [ UNASSIGNED-CoreValidation-Shader-InterfaceTypeMismatch ] Object 0: VK_NULL_HANDLE, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0xb6cf33fe | Attribute type of `VK_FORMAT_R8G8B8A8_UINT` at location 4 does not match vertex shader input type of `ptr to Input vec4 of sint32`
err:   VUID-vkCmdDraw-None-02699: 
err:   Validation Error: [ VUID-vkCmdDraw-None-02699 ] Object 0: handle = 0x8639c40000000ab7, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0x1608dec0 | Descriptor set VkDescriptorSet 0x8639c40000000ab7[] encountered the following validation error at vkCmdDraw time: Descriptor in binding #0 index 0 requires UINT component type, but bound descriptor format is VK_FORMAT_R32G32B32A32_SFLOAT. The Vulkan spec states: Descriptors in each bound descriptor set, specified via vkCmdBindDescriptorSets, must be valid if they are statically used by the VkPipeline bound to the pipeline bind point used by this command (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkCmdDraw-None-02699)

I have attached a snipped version of the debug log for review.

The game otherwise looks fine while playing but I can imagine the validation errors are indicative of a vulkan issue that may affect performance?

Edit: some additional info:
To reproduce just start a new game. Framerate is still ok in the first few seconds of the intro sequence though you can already see stuttering/frame pacing issues when the 3 geese fly through the screen. The frame rate hovers between 50 and 60 fps until you get control, just walking left for a few seconds the frame rate progressively drops down to 30 where the spider thing is waving at you. Interestingly the frame graph in the HUD looks relatively clean, no big spikes.

When I enable the Vulkan validation layer + debug logging, the game starts up quite normally, HUD shows normal framerate during the shader compilation screen with the running Ori, but the moment the menu opens everything slows to a crawl immediately to the point it is almost impossible to start a game, eventually locking up the system.

I tried all kinds of configurations, all with the same result: Nvidia driver 525, start Steam BPM through steamcompmgr, from a KDE Plasma session, exclusive fullscreen, borderless, V-Sync ON/OFF, frame rate limit on/off, all with the same result.

This is on Arch x86-64 with kernel 6.1.1 by the way

Software information

Ori: Will of the Wisps

System information

GPU: Nvidia RTX 3080
Driver: 520.56.06
Wine version: Proton 7, Proton 6, Proton 5, GE-Proton 7, all same behaviour
DXVK version: v2.0

Log files

d3d9.log:
d3d11.log: oriwotw_d3d11.log
dxgi.log: oriwotw_dxgi.log

Ww0utert 2022-12-28 github

I tried to make an API trace, but running the game with the apitrace DLL's will crash Wine with an unhandled page fault before the game even starts. I tried to run the version installed directly through the Linux Steam client, as well as a copy installed through Windows Steam running under wine, but both crash in the same way. I can see the apitrace file being generated by the way, so at least that part is working ;-)

BBlisto91 2022-12-28 github

You can try to make one both with and without dxvk and see if it makes a difference. One made on Windows would also be fine if you have access to that and it works.

Ww0utert 2022-12-29 github

@Blisto91 how do I make the API trace using DXVK? If I just put the DLL’s with the game exe and launch through Steam the trace is not generated, i guess I need to do the equivalent of overriding them using winecfg?

Ddoitsujin maintainer 2023-01-02 github

I've very frequently tested (and to some degree, played) this game during DXVK 2.0 development on Nvidia and have never noticed performance being lower than expected (on a mobile 2060). It's also running perfectly fine on my AMD desktop.

The validation error is caused by a game bug and not something we can really fix, but it's also harmless.

Ww0utert 2023-01-02 github

Do the vulkan validation errors and the fact there are literally ~4000 emitted for just a few seconds of game runtime provide a clue? Surely they mean something is not quite right? Anything else I can do to debug further?

I know the game runs perfect on AMD, I’ve played it on steam deck. The performance I’m seeing on my desktop is terrible compared to that.

Ddoitsujin maintainer 2023-01-02 github

FWIW I'm running some older 5.19 kernel on my Nvidia machine right now. Building a 6.1 right now to test that, but given that even on my main desktop I've had some major problems with 6.1 (like the entire system stuttering like crazy every couple of hours, even in regular desktop use) it might just be that.

Ww0utert 2023-01-02 github

Yes the kernel is the only thing I did not try to downgrade yet (proton/dxvk or driver version did not make a difference).

if you need more info or know something else I can try just let me know

Ddoitsujin maintainer 2023-01-02 github

Looks like a kernel regression indeed, with 6.1 I'm getting periods of like 10 FPS for a few seconds in the game and then a bit over 30. With 5.19 it's sitting at a locked 60.

Adding tsc=reliable to the kernel command line kind of fixes this on my end though, for some reason it defaults to HPET which makes everything stupidly slow (CPU is a Ryzen 4900HS).

RADV is significantly less affected by this than Nvidia but the issue is there on both.

Ww0utert 2023-01-02 github

Ok, serves me right for using Arch with its bleeding edge kernels then, I guess 😆

Does this mean the issue is strictly on the kernel side and needs to be fixed there? And with the tsc=reliable boot option, does it run as before on a 5.x kernel or is it only 'kind of fixed' in the sense performance is better but still not great? If so I will just downgrade my kernel for now.

Thanks for debugging so far!

Ddoitsujin maintainer 2023-01-02 github

I'm GPU-bound now so performance is the same between the two kernels. I don't really have the time now to do extensive testing to compare the two, but with the boot parameter I'm not seeing any obvious red flags in perf top either, so it should be good.

As for who is at fault here, don't know, but I doubt it's something we can fix on the Proton/DXVK side.

Ww0utert 2023-01-02 github

Ok makes sense. I already observed that even with the 6.1 kernel with bad performance, both CPU and GPU utilisation were very low even despite the low framerates, so definitely something is fundamentally broken there. I guess I'll just add the boot option then.

Ww0utert 2023-01-13 github

I have since tried with kernel 6.1.1 and 6.1.5 with the tsc=reliable boot option, and with the Arch linux-lts (5.15) kernel, both without any success. This game still runs like crap under linux with DXVK no matter what I try.

My system is a Linux hypervisor running a QEMU Linux VM with GPU passthrough, which may or may not be related to this performance bug, but the fact of the matter is that a for example a game like GTA V runs perfectly in the same VM, and the Ori game itself also runs perfectly fine in a QEMU Windows 10 VM on the same machine. Which suggests there is something uniquely broken with the combination of running Ori WoTW through DXVK/Proton.

I've given up on this for now. I'm not fully convinced this issue really is 'not a bug' for DXVK though, considering I have no problems with other games. If there is anything I can do to further debug I'm open to that.

Edit: ok nevermind, I finally got the game working properly, the issue indeed was with the 6.x kernel, but I had to downgrade both the hypervisor (host) and VM (guest) kernel. Game is now running fine.

Nothing extracted yet.