[BUG] Arch kernel freeze on startup (6.4.1-arch1-1 to 6.4.5-arch1-1 seem broken, including core-testing's 6.4.5-arch1-1)
Hello @Danielduel, reading through this, it reads like a video driver regression which should also be mentioned to your video driver vendor.
If I'm reading your logs correctly, it looks like you're using a Valve Index at 144Hz with an AMD Radeon RX 7900 XTX. The video driver for this specific GPU has had a hard time with that combination in the past (#552). I think it would be an interesting data point to test what happens at 120Hz.
Hi @kisak-valve do you mean that while I have the working version on 6.3.9.arch1-1 I should change the headset refresh rate to 120Hz.
Then update packages to latest versions (so 6.4.4.arch1-1), reboot and test what happens there.
While I had latest packages with downgraded kernel (a setup that worked everytime on 2nd run)
I've changed the refresh rate of the headset to 120Hz. Quit SteamVR. Reopened, checked if it actually saved that - it did.
# pacman -Syyuu, reboot.
SteamVR initialization is freezing the system again.
This is how system report looks like on 6.4.4.arch1-1.
SteamVR-2023-07-23-PM_09_57_55.txt
Is there something that I can help with?
If this has to be forwarded to some amd bugtracker - is this the correct place to start? https://bugzilla.kernel.org/
Unfortunate, thanks for testing. The Linux kernel bugtracker makes sense, or maybe https://gitlab.freedesktop.org/drm/amd/-/issues/?
I've downgraded to 6.3.9.arch1-1 as a workaround, I will try to make a ticket in suggested https://gitlab.freedesktop.org/drm/amd/-/issues/. Thank you, I suggest adding info about latest kernel being broken to AMD section of readme if I am not the only case like that (so question to others - if you also have this problem).
I will try to keep this thread updated, wish me luck :)
@kisak-valve do you know if steamvr sources are available somewhere? I mean - opensource or available after you pass some process. I would like to do an executable for that bug so steps to reproduce wouldn't be "go to steam, get steamvr, buy index, update arch to latest kernel, look there's some bug".
... Actually since I have a stacktrace from amdgpu it shouldn't be necessary, but still I am curious if it is open/partially open somewhere.
Maybe I could figure out why #589 exists
I am also experiencing issues on kernel versions higher than 6.3.9.arch1-1 on Nvidia.
I get an Invalid libcef opcode in journalctl.
Problem appears worse on Nvidia as it also causes issues in the main Steam Client causing to crash at random.
The included screenshot from journalctl also applies to vrwebhelper, with the same error on anything newer than 6.3.9.arch1-1
System Information (please complete the following information):
Distribution: Arch Linux
SteamVR version: 1.26.6
Steam client version: 1689034492
Opted into Steam client beta?: No
Graphics driver version: 535.54.03
Gist for SteamVR System Information: (https://gist.github.com/megagtrwrath/0199b1f60b8ae91a02d39e41ee74536a)
The kernel freezes at start seem unrelated to SteamVR. I get them (7900XTX as GPU) upon logging in at times but it seems if I give it time to warm up at the display manager login screen (I use lightdm-gtk-greeter) it hasn't been an issue lately.
Have been using 6.3.9 for a while but recently been fine on 6.4.6 and 6.4.7 kernels.
@kzdixon could you explain what do you mean by "warming up"?
@kzdixon could you explain what do you mean by "warming up"?
Waiting a little bit to log in after startup rather than logging in immediately. Could be totally unrelated/happenstance that my latest logins I would let it sit for a small bit rather than immediately catch it right when it shows the display manager but the freeze doesn't happen every time at the very least (and my last attempts after waiting have thus far all avoided that freeze).
@kzdixon
Which kernel build are you using? Mainline or something custom?
Could you describe your setup? I would love to try upgrading if it solves issues, also the system report might have a hint about differences between our setups :)
@kzdixon Which kernel build are you using? Mainline or something custom? Could you describe your setup? I would love to try upgrading if it solves issues, also the system report might have a hint about differences between our setups :)
I'm using zen-sources technically (am on Gentoo) but both 6.4.6 and 6.4.7 have been my latest attempts without issue (low sample size of logins though).
I was seeing the freeze occur on 6.4.0 -> 6.4.4 and thus masked them with a warning in my ebuild repository.
Config isn't far off from standard arch zen kernel (I based my choices on lqx and a few system-specific things like znver4 & O3). Config also hasn't changed from 6.3.9 to these versions besides the new options that got added (most defaulting to FALSE).
I can confirm that SteamVR isn't working, beginning with kernel 6.4., downgrading to 6.3 fixes the issue. I'm on a 7900xtx with a Valve Index.
It might be interesting to check if this issue happens with a linux 6.5 release candidate kernel. If my suspicion is correct, then only 6.4.x is affected.
6.5.0 will probably release on sunday, as soon as it hits arch-testing I'll report back with my results.
I currently don't have the time to compile a -rc kernel.
I did a compilation of 6.5 rc, it is still there (consider this info to be a week old). After that I tried to comment a check if update is fast on async flip in DRM because kernel trace kinda lead to that place...
But no.
After I went digging - for some reason the stream that goes into update is NULLed or it is a NULL since the init.
Currently I am booting 6.3.13 (compiled tag from linux repo) when I want to use SteamVR.
I tried to debug it myself, but I have too many things going on in dayjob, hope it helps.
// also - I tried to find a commit that introduces this issue basing on changes related to amdgpu, but it is very time consuming
I can confirm that SteamVR isn't working, beginning with Kernel 6.4. Downgrading to 6.3 fixes the issue. I'm on a 7900xtx with a Valve Index.
SteamVR works on my end on kernel 6.4.10 and prior 6.4.X kernels. Also on a 7900 XTX.
I did a compilation of 6.5 rc, it is still there (consider this info to be a week old). After that I tried to comment a check if update is fast on async flip in DRM because kernel trace kinda lead to that place... But no. After I went digging - for some reason the stream that goes into update is NULLed or it is a NULL since the init. Currently I am booting 6.3.13 (compiled tag from linux repo) when I want to use SteamVR. I tried to debug it myself, but I have too many things going on in dayjob, hope it helps.
// also - I tried to find a commit that introduces this issue basing on changes related to amdgpu, but it is very time consuming
I can confirm that 6.5 does not fix this issue.
Kernel version 6.5.3 seems fixed(??) this problem
When first launch SteamVR, I still got a error of -203.
But after restart SteamVR, it can work correctly.
Replying to https://github.com/ValveSoftware/SteamVR-for-Linux/issues/598#issuecomment-1717907920
Indeed, they did fix this issue, I'm experiencing the same behavior as @Xanadu-Zen .
Will check,is it working on the 1st run of steamvr or like previously - you have to restart it? I reread the Xanadu comment
@Xanadu-Zen @bblacher
I have to restart it, same as @Xanadu-Zen. I had this behavior before kernel 6.4, so this is "normal" for me.
Another weird thing (probably subject of another issue and probably an issue with AMDGPU rather than SteamVR) is that I have to disable at least one of my screens in order to get SteamVR to use direct mode.
My setup is as follows:
7900xtx
DP-1 3840x2160@144Hz
DP-2 1920x1080@144Hz
HDMI-A-1 1920x1080@60Hz
In order to use SteamVR in direct mode I have to use either swaymsg output DP-2 power off or swaymsg output HDMI-A-1 power off. I guess there just isn't enough processing power to drive all those resolutions or this is a bug in AMDGPU.
1: initially I can tell that it seems to work, @kisak-valve I want to have 3-4 beatsaber sessions before we do victory laps with closing this one, could you ping me after 17 September if I forgot to communicate that it is solved?
2: @bblacher I use (1 display) DP 1.4 5120x1440@120(144?)Hz on the same gpu, I can check if I will have the same issue after connecting two FHD displays, but those are 60Hz displays. Maybe you have some :wood:s in journalctl?
@Danielduel Using my main 2160p@144Hz screen and my secondary 1080p@144Hz screen at the same time with my Index@144Hz is totally fine. Only when I connect a third screen it doesn't work anymore.
@Danielduel Here you go
This is me trying to start it twice with all screens connected.
If I disable at least one of the screens SteamVR works perfectly fine after the second try.
First BS session worked perfectly on 6.5.3.
Changing the title of this issue to communicate there's some development in this topic.
I now have the issue where SteamVR crashes my entire driver stack on AMD (7900XTX) on first boot of SteamVR; Killing it ASAP and starting a 2nd time seemed alright but I did not have this issue before.
Recently swapped HMDs to test out some stuff with the Bigscreen Beyond on Windows/Linux and it started doing this afterwards.
EDIT:
I can confirm kernel 6.5.3 has a broken first start for me that kills the AMD display stack; I used to never get these until I had swapped around an HMD and was testing across Windows/Linux for a bit so something got bungled in the process I guess.
1: initially I can tell that it seems to work, @kisak-valve I want to have 3-4 beatsaber sessions before we do victory laps with closing this one, could you ping me after 17 September if I forgot to communicate that it is solved? 2: @bblacher I use (1 display) DP 1.4 5120x1440@120(144?)Hz on the same gpu, I can check if I will have the same issue after connecting two FHD displays, but those are 60Hz displays. Maybe you have some 🪵s in journalctl?
Did you have the same issue with 3 screens?
@bblacher
Haven't tested it yet, I miss 2nd hdmi cable (the 2 DPs are taken by HMD and main display so I need 2 hdmi cables (I have 2dps and 2hdmis on gpu)).
https://github.com/ValveSoftware/SteamVR-for-Linux/issues/598#issuecomment-1718064260
Could you take this comment, do a separate issue and ping me there?
I would like to close current issue.
@kisak-valve
I've had few Beat Saber sessions on beta steamvr on latest kernel.
Updating kernel to 6.5.3 fixes the issue.
I've observed new unexpected behavior, but it is different noncritical issue - I will make another ticket for that.
I've tested https://liquorix.net/ on my latest session and it is stable.
I think we can close this.
@bblacher Haven't tested it yet, I miss 2nd hdmi cable (the 2 DPs are taken by HMD and main display so I need 2 hdmi cables (I have 2dps and 2hdmis on gpu)). [#598 (comment)](https://github.com/ValveSoftware/SteamVR-for-Linux/issues/598#issuecomment-1718064260) Could you take this comment, do a separate issue and ping me there?
@Danielduel done
Latest fully upgraded arch:
Clicking "launch" on SteamVR in Steam client makes system hang for few (2-10) seconds, then the gray color (same like in steamvr settings) is flashing on the screen, after this you get a broken SteamVR, sometimes UI is clickable, hmd is always unusable.
Journal seems to randomly crash vrmonitor, vrwebhelper and other "inner components", I can collect logs if it is supernecessary.
To Reproduce
On base arch system as superuser run
pacman -Syyuuthen accept everything, reboot system, run Steam, go to SteamVR (beta and latest have the same behavior).Expected behavior
SteamVR working.
System Information (please complete the following information):
QT_QPA_PLATFORM=xcb SDL_VIDEODRIVER=x11 %command%)Screenshots
Additional context
I've decided to tinker around and try getting kde-plasma with latest kernel and wayland working on my system and on top of that to switch Steam to the flatpak version.
I got Flatseal and steam-devices to solve some problems with capabilities and permissions.
I had a ton of problems, so I've started to dig into dependencies.
Aside of upgrading to testing - I've used https://wiki.archlinux.org/title/Downgrading_packages as a way to test archived versions.
Before this test I've assumed that it has something to do with mesa-related package (since graphical glitches and hangs, also I get gpu-coilwhine before hang) but it rendered no result, so I've deleted that log.
So.
My
downgrade-pacman.conflooks like this:mirrorlist-downgrade:Starting state is latest everything, steamvr beta and steam-devices -
# pacman -Syyuu, reboot.To downgrade (to use versions from archive) I was using a command
# pacman -Syy --config /home/shared/workspace/steam-steamvr-flatpak-setup/downgrade-pacman.conf --arch x86_64 linux linux-firmware linux-firmware-whence linux-headers.Let's start:
Assume that after each package upgrade/downgrade I've rebooted the system.
(this step was specific, I did a special pacman config to get this version)
6.4.5-arch1-1 (current core-testing) is broken.
(latest)
6.4.4-arch1-1 (current flagged core since 19-Jul-2023 21:30) is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/19/$repo/os/$arch6.4.3-arch1-2 is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/15/$repo/os/$arch6.4.3-arch1-1 is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/11/$repo/os/$arch6.4.2-arch1-1 is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/08/$repo/os/$archIt changes firmware, broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/07/$repo/os/$arch6.4.1-arch2-1 is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/04/$repo/os/$arch6.4.1-arch1-1 is broken.
(downgrade)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/02/$repo/os/$archWorks. (it fails in different way on the 1st run, like I had before, but on 2nd run (as before) it works)
(Fully downgrade to broken version)
sudo pacman -Syyuu --config /home/shared/workspace/steam-steamvr-flatpak-setup/downgrade-pacman.conf --arch x86_64Works as described above.
(Try a version above, the one that was broken just before)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/04/$repo/os/$archsudo pacman -Syyuu --config /home/shared/workspace/steam-steamvr-flatpak-setup/downgrade-pacman.conf --arch x86_64Broken.
(Fully upgrade to latest (broken version), my default pacman config is not using core-testing)
sudo pacman -SyyuuBroken.
(Downgrade selected packages to working version, but keep all other packages latest)
Change downgrade server to
https://archive.archlinux.org/repos/2023/07/02/$repo/os/$archTo be superprecise:
So it resulted in downgrade like follows (copypaste pacman output):
Works like described above.
Question to devs - is it some breaking change from kernel side or SteamVR just needs to be recompiled with new headers and pushed to superlatestbeta?
Note: Commenters who are also experiencing this issue are encouraged to include the "System Information" section in their replies.