Compatibility Report

Hey! Late response.

Have you tried any other steps?

Hey! Late response.

Have you tried any other steps?

I've tried with different OpenVR/OpenXR runtimes and it seems like no matter what I do it's just no dice. I've tried this game on ALVR (SteamVR), and WiVRn with both OpenComposite and xrizer and nothing seems to work. iirc this game is OpenXR natively so I don't think OpenVR -> XR translation layers would matter in this case

I had a look at the issue. Here are the two problems (or, well, one and a half) which are preventing the start:

1. When the engine searches for Vulkan physical device to use, it demands "VK_EXT_full_screen_exclusive" Vulkan extension. This Vulkan extension is available on Windows but is currently not supported on Linux natively and is not faked in Proton anyhow. Thing is, the game doesn't really use the extension. I found a call to vkAcquireFullScreenExclusiveModeEXT in the current engine's source code (https://github.com/uberpixel/Rayne) but it is commented out, that is, never called. Also, just faking the extension in the advertised Vulkan extension list in Proton fixes the issue and the game starts working (but see p. 2), the functions exposed by extension are not attempted to be called.

2. This relates only to setups with more than one GPU (e. g., integrated and discrete, or just more than one discrete GPUs). The engine seems to be finding the first Vulkan device which has required extension and attempts to use it for OpenXR with xrCreateSession. If the device actually bound for XR is different (like happened here) xrCreateSession will fail session creation. There is no real choice here with OpenXR, the Vulkan device to be used is the one which xrGetVulkanGraphicsDeviceKHR() returned, attempt to use a different one won't succeed. This part only relates to setups which have multiple GPUs (while it currently might be challenging to setup VR on such setups on Linux anyway). That can be worked around by running the game with VK_MESA_device_select (with AMD) or equivalent on NVidia, forcing the device actually used by VR.

So the main blocker here is 1. I am so far not planning adding any faking around that extension in Proton. Maybe if it is possible to report to the engine / game developers it can be fixed there by just not requiring unneeded and unused extension?

PS p. 2. would also be great to fix of course, only the physical device returned from xrGetVulkanGraphicsDeviceKHR() should be considered for selection.

Thanks for that insight gofman. I'll see if I can relay this into the developer's Discord server for the game, as they are somewhat active there

Developer said they'd look into it.

I just released an update fixing the first issue, no longer requiring that extension. I am aware of 2, it just hasn't been a big enough issue yet to be worth the effort, I do hope to find the time to fix it eventually, probably in combination with switching to XR_KHR_vulkan_enable2.

Game opens on a black screen and crashes after a few seconds.

steam-1701130.log

Can you please try with Proton Experimental (https://github.com/ValveSoftware/Proton/wiki/Proton-Versions#proton-experimental) and if it still happens with attach PROTON_LOG=+vrclient,+openxr,+module,+vulkan ?

Can you please try with Proton Experimental (https://github.com/ValveSoftware/Proton/wiki/Proton-Versions#proton-experimental) and if it still happens with attach PROTON_LOG=+vrclient,+openxr,+module,+vulkan ?

Hi again. Same problem. I don't see a log in my Home folder with that command line parameter

I also tried explicitly making a PROTON_LOG_DIR but still no log.

sorry, the full launch options are "PROTON_LOG=+vrclient,+openxr,+module,+vulkan %command%" without quotes.

sorry, the full launch options are "PROTON_LOG=+vrclient,+openxr,+module,+vulkan %command%" without quotes.

Thanks. Here's the new log:

steam-1701130.log

I looked a bit more into it. I could not reproduce first but then reproduced switching to KDE / XWayland from X11.

The crash is in VulkanSwapChain::Prepare(), here: https://github.com/uberpixel/Rayne/blob/209d42af358a31c613f458c42d77832e7fdda1d7/Modules/Vulkan/Sources/RNVulkanSwapChain.cpp#L440

_framebuffer->GetColorTexture(_frameIndex) returns NULL.

I think that is triggered by vkQueuePresentKHR returning VK_SUBOPTIMAL_KHR. There are various occasions during initial wiindow configuration how that can happen, depending on how and when swapchain is created, probably on [X]Wayland that is different from other cases. VK_SUBOPTIMAL_KHR triggers VulkanSwapChain::CreateSwapChain() call from VulkanSwapChain::PresentBackBuffer(). Then, the first thing CreateSwapChainDoes does is calling _framebuffer->WillUpdateSwapChain() (https://github.com/uberpixel/Rayne/blob/209d42af358a31c613f458c42d77832e7fdda1d7/Modules/Vulkan/Sources/RNVulkanSwapChain.cpp#L186). That clears _colorTargets() in VulkanFramebuffer. I suppose _colorTargets are always meant to be recreated with DidUpdateSwapChain(). But it doesn't happen: CreateSwapchain returns only since surface extens did not change: https://github.com/uberpixel/Rayne/blob/209d42af358a31c613f458c42d77832e7fdda1d7/Modules/Vulkan/Sources/RNVulkanSwapChain.cpp#L206 . So the framebuffer is left with zero color target count and next render crashes in Prepare because _framebuffer0->GetColorTexture() is going to return NULL for any frame index.

I suppose this is a bug in VulkanSwapChain::CreateSwapChain: it should probably not call WillUpdateSwapChain if it is not going to actually recreate it (while that would be also wrong I guess; it is best to recreate swapchain once receiving VK_SUBOPTIMAL_KHR even if the surface size didn't change, there are other reasons for suboptimal swapchain and failing to recreate it will keep VK_SUBOPTIMAL_KHR coming on each Acquire or Present). Or always call DidUpdateSwapChain(). What I'd actually suggest is to remove that check for (not) changed image extents, that should hopefully help.

As a separate note (not sure if it currently causes actual issue, maybe not), this looks a bit wrong:

https://github.com/uberpixel/Rayne/blob/209d42af358a31c613f458c42d77832e7fdda1d7/Modules/Vulkan/Sources/RNVulkanSwapChain.cpp#L209 , should probably be if(caps.maxImageCount > 0) imageCount = std::min(caps.maxImageCount, imageCount)?

Thanks for all the effort digging into it!
I made both changes to the engine and just released an update with those changes.
Lets hope no other problems show up now :)