protonscr

Latest Steam-Client beta won't start with mesa-git

steamclosed AMD driversSteam client
ValveSoftware/steam-for-linux#8202 · opened 2021-11-03 by kallestrop · updated 2021-11-11 · 35 comments · github
Kkallestrop 2021-11-03 github

System information

  • Steam client version (build number or date):
    Steam-Client
    Erstellt: Nov 2 2021 um 19:40:07
    Steam-API: v0200
    Steam-Paket-Version: 1635884102

  • Distribution: Debian 11

  • Mesa 20.3.5 / Mesa 22.0.0-devel (git-e15e3a8e86)

  • Opted into Steam client beta?: Yes

  • Have you checked for system updates?: Yes

Please describe your issue in as much detail as possible:

Since the last beta client update, Steam won't start anymore when mesa-git is active. There is a message with a segmentation fault in terminal output:

/home/user/.local/share/Steam/steam.sh: Zeile 794: 73645 Speicherzugriffsfehler "$STEAMROOT/$STEAMEXEPATH" "$@"

  • Steam Client starts normal when using Mesa 20.3.5 (Debian 11 repository version)
  • Steam Client was running without issues on mesa-git before latest beta update

https://gist.github.com/kallestrop/2ac9be55d074c5f02fc3e1a7b89a26b4#file-gistfile1-txt
terminal-log-mesa-git.txt
terminal-log-mesa-20_3_5.txt

Steps for reproducing this issue:

  1. Using Debian 11
  2. Compile and activate mesa-git from https://gist.github.com/baryluk/1041204eff4cc4fad6f1508afe67b562
  3. Start Steam Client Beta
Ddcbaker 2021-11-03 github

I can replicate this with Intel on NixOS using mesa 21.2.5, rolling back to 21.1.4 seems to fix the issue.

Dduartec3000 2021-11-10 github

Can confirm the same thing is happening on Arch Linux:

➜ ~ steam
Setting up Steam content in /home/g/.local/share/Steam
Running Steam on arch rolling 64-bit
STEAM_RUNTIME is enabled automatically
Updating Steam runtime environment...
Can't find 'steam-runtime-check-requirements', continuing anyway
WARNING: Using default/fallback debugger launch
/home/g/.local/share/Steam/ubuntu12_32/steam
WARNING: setlocale('en_US.UTF-8') failed, using locale: 'C'. International characters may not work.
[2021-11-10 10:30:01] Startup - updater built Jul 20 2021 22:25:51
ILocalize::AddFile() failed to load file "public/steambootstrapper_english.txt".
Installing breakpad exception handler for appid(steam)/version(0)
realloc(): invalid pointer
crash_20211110103001_1.dmp[10927]: Uploading dump (out-of-process)
/tmp/dumps/crash_20211110103001_1.dmp
/home/g/.local/share/Steam/steam.sh: line 772: 10908 Aborted (core dumped) $STEAM_DEBUGGER $DEBUGGER_ARGS "$STEAMROOT/$STEAMEXEPATH" "$@"
➜ ~ crash_20211110103001_1.dmp[10927]: Finished uploading minidump (out-of-process): success = yes
crash_20211110103001_1.dmp[10927]: response: CrashID=bp-46cd654d-3e09-4007-9f91-18e812211110
crash_20211110103001_1.dmp[10927]: file ''/tmp/dumps/crash_20211110103001_1.dmp'', upload yes: ''CrashID=bp-46cd654d-3e09-4007-9f91-18e812211110''

Ggardotd426 2021-11-10 github

Anyone compiling from source (and not a PPA or chaotic-aur/etc binary package) should try to bisect this so an actual bad commit can be pinpointed. Since this just happened today it shouldn't take long at all, especially if you have a decent CPU. I'm on Nvidia so I can't do it.

Also reporting that Nvidia doesn't experience the bug even with mesa-git, and yes Nvidia uses mesa components for stuff like GBM support so it's not stupid to point out that it still works with Nvidia and mesa-git.

Is everyone here on AMD? Because if so it might be a bug in radeonsi, if Intel GPU users can't reproduce.

Jjhu54 2021-11-10 github

Running Linux Mint with 5.15.1 kernel, rx 6700 xt, Steam beta (updated last night) and Mesa 22.0.0-devel (git-4dfb581 2021-11-10 focal-oibaf-ppa) - no issues today or in the last week. You might want to dig a bit deeper into his config...

Ggardotd426 2021-11-10 github

Running Linux Mint with 5.15.1 kernel, rx 6700 xt, Steam beta (updated last night) and Mesa 22.0.0-devel (git-4dfb581 2021-11-10 focal-oibaf-ppa) - no issues today or in the last week. You might want to dig a bit deeper into his config...

Well several people have reported this across three different distribution families (Nix, Arch, and Debuntu). So something is going on. There have been 3 commits since your current version. Either one of those commits is the problem, or the problem was fixed before or with 4dfb581, and these other users are all using builds that don't have the fix yet.

It's hard to tell, because none of the users included what git commit they're on, and for some reason @kallestrop's stated package version (git-e15e3a8e86) doesn't match any mesa commits I can find, so the script must version it some other way, and I can't figure out how (it's insanely long), and it doesn't use git rev-parse --short HEAD like any sane mesa-git package does.

@kallestrop, go into the mesa-git directory that the script clones the repo in, defaults to $HOME/mesa-git, and run git rev-parse --short HEAD to see what commit it was built against. That will go a long way into solving this. Or @duartec3000 can tell us with pacman -Q mesa-git and giving us the numbers after 22.0.0_devel.

I'm using the chaotic-aur package which is also on 4dfb581 same as you @jhu54.

JJonesGreen 2021-11-10 github
Ggardotd426 2021-11-10 github

Eh god, that's so old that a regular git clone doesn't even have it in the history you can tab through.

That's a week old. @jhu54's version is from today with only 3 commits since his package was compiled. @jhu54 doesn't have the issue.

@kallestrop, seems you haven't built mesa-git for like a week, maybe rebuild and see if the issue persists?

Kkallestrop 2021-11-10 github

@gardotd426 when i switch to mesa-git folder and type
rev-parse --short HEAD

output is
e15e3a8

Ggardotd426 2021-11-10 github

@gardotd426 when i switch to mesa-git folder and type rev-parse --short HEAD

output is e15e3a8

Yeah man your build is like a week old. Rebuild and see if it persists.

Kkallestrop 2021-11-10 github

Build mesa-git right now and tested again with latest steam beta client:

Steam-Client Nov 9 2021 21:29:11
Steam-Paket-Version: 1636496799

Mesa 22.0.0-devel (git-3a95e25e84)

But unfortunately, steam-client beta still won't start

steam-client-beta.txt

Ggardotd426 2021-11-10 github

That's bizarre then, since @jhu54 can't reproduce. @kallestrop your Steam build is incredibly old as well, mine is 1636496799 from Nov. 9 @ 21:29:11. Which is weird, since steam is supposed to automatically update.

@kisak-valve does steam push beta updates to different distributions differently? Since the steam package is the same regardless I would think that wouldn't be the case, and that they should all update at the same time.

Kkallestrop 2021-11-10 github

@gardotd426 I tested right now with the same Steam build as you have. The Steam build you refer to is probably the actual version 7 days ago when i opened this issue

?ghost 2021-11-10 github

Hi. I'm unaware of what Steam version I have since I can't open it anymore. However, I can confirm that I'm experiencing this bug on archlinux using AMDGPU. My mesa-git version says 1:22.0.0_devel.146973.ca7c748f45a-1 I just wanted to add this since I was previously unaware of the issue.

Kkisak-valve maintainer 2021-11-10 github

Hello @kallestrop, given the CDesktopCapturePipeWire: Opening DRM render node /dev/dri/renderD128 line just before the minidump gets generated at the end of your log, it might be interesting to test if steam -nopipewire has an effect.

Hello @duartec3000, @Develon5543, realloc(): invalid pointer in your terminal spew along with what's in #8220 hints towards a regression in llvm 14 git. Are you able to test mesa-git and lib32-mesa-git built against llvm 13?

Dduartec3000 2021-11-10 github

Or @duartec3000 can tell us with pacman -Q mesa-git and giving us the numbers after 22.0.0_devel.

I was using LordHeavy mesa-git repo, always worked flawlessly, updated to the current version today and the issue started:

mesa-git/lib32-mesa-git 1:22.0.0_devel.146973.ca7c748f45a-1 (mesagit)
    an open-source implementation of the OpenGL specification (32-bit) (git
    version)
mesa-git/mesa-git 1:22.0.0_devel.146973.ca7c748f45a-1 (mesagit)
    an open-source implementation of the OpenGL specification (git version)

Rolled back to mesa 21.2.4 and Steam Beta started launching just fine.

Ggardotd426 2021-11-10 github

@duartec3000 like kisak said it points to a regression in llvm-git.

mesa is compiled against llvm, and LordHeavy is compiling his builds against llvm-git (14.0.0-devel).

So you will need to compile mesa-git with llvm stable instead, to see if that fixes it.

You'll have to grab the PKGBUILDs for both mesa-git and lib32-mesa-git. https://aur.archlinux.org/packages/mesa-git and https://aur.archlinux.org/mesa-git. After cloning both those repos, remove llvm-git, llvm-libs-git, lib32-llvm-git, and lib32-llvm-libs-git in favor of the non-git versions of all those packages, then cd into the mesa-git directory, run makepkg -si, then do the same for the lib32-mesa-git package. By default the AUR package builds against stable llvm, not llvm-git. LordHeavy does it against llvm-git.

Alternatively just use mesa and not mesa-git.

Kkallestrop 2021-11-10 github

@kisak-valve starting with option -nopipewire has no effect

steam-client-beta.txt

Jjhu54 2021-11-10 github

Any other info you want from a working config? I can confirm I am on the same Steam. What llvm version is oibaf using? I always assumed he was using llvm stable

Ggardotd426 2021-11-10 github

If oibaf is using stable that would explain exactly why you aren't seeing the issue.

Jjhu54 2021-11-10 github

llvm-13 llvm-13-dev llvm-13-linker-tools llvm-13-runtime llvm-13-tools - guess that means llvm 13.

Ggardotd426 2021-11-10 github

Yep. Unfortunately Arch users aren't able to compile easily against LLVM 13, LLVM stable is at 12.0.0, 13.0.0 isn't in the Arch repos, and -git is at 14.0.0 (this is common with LLVM/clang/etc). But the AUR package can easily be built using llvm 12.0.0, so Arch users will need to do that, because as far as I'm aware the two options for precompiled mesa-git (LordHeavy and chaotic-aur) both compile against -git 14.0.0. But the AUR packages by default use stable LLVM 12.

Dduartec3000 2021-11-10 github

Hello @duartec3000, @Develon5543, realloc(): invalid pointer in your terminal spew along with what's in #8220 hints towards a regression in llvm 14 git. Are you able to test mesa-git and lib32-mesa-git built against llvm 13?

Even though I'm computer literate, I work in infrastructure and don't know that much about programming, I'm sorry.

I think you might be in the right path as llvm-libs 14.0.0 is installed as a dependency of current mesa-git:

mesa-git/lib32-llvm-libs-git 14.0.0_r404027.2caf85ad7ab8-1 (mesagit)
    Low Level Virtual Machine library (runtime library)(32-bit)(git version)
mesa-git/llvm-libs-git 14.0.0_r404021.ca47447952f1-1 (mesagit)
    LLVM runtime libraries (git version)

and this would explain why some people with mesa-git are not experiencing the issue, they probably have access to binaries compiled against llvm 13.

@gardotd426 Thank you for the instructions but I'm afraid of messing something up, rolling back to normal mesa already gave me enough trouble for a day.

Kkisak-valve maintainer 2021-11-10 github

@kallestrop, can you share the output of apt policy libegl1 libegl1:i386 libgbm1 libgbm1:i386 libegl-mesa0 libegl-mesa0:i386?

What I suspect is that part of Steam's new pipewire desktop capture method is being used in spite of it being expected to be disabled by default with users opting into the sub-component by running steam -pipewire. I think that wants to use new dependencies on libegl and libgbm, but it's encountering an incompatible library from the Steam runtime that is causing it to fail. The steam 1.0.0.73 package, which in the Valve-provided beta update channel should check for this and ask for the relevant system packages to be installed.

At this point, it's fairly clear that there's a separate regression with bleeding edge llvm14 which is unrelated to the opening post. That regression should be reported to the AMD driver devs and investigated on the video driver's side. @Develon5543, if you want to re-open #8220 to ponder it on this issue tracker, you can, but that is something which has to be evaluated and resolved by the video driver devs. Please report the llvm regression to the video driver vendor if it hasn't been already.

Ssmcv 2021-11-10 github

If @kallestrop is using a version of Mesa that has been compiled locally from source code, then OS-level packages are not necessarily what's important (or are not necessarily all of what's important); what's important is the versions of Mesa that Steam actually loads.

Knowing which packages are installed, and their versions, would still be useful information though.

Ssmcv 2021-11-10 github

The most useful piece of information to have here (from anyone who can reproduce it, but particularly from @kallestrop) would be: a configuration (combination of commits, packaging, build options, etc.) that does show this issue, and a configuration that is as similar as possible but does not show this bug.

What we have from @kallestrop is:

  • Mesa 20.3.5 (presumably 20.3.5-1) from Debian, presumably compiled against Debian's llvm-11 version 1:11.0.1-2, with Debian build options, with Steam beta Nov 2 2021: works
  • Mesa git commit e15e3a8e86 (594 commits after 21.3-branchpoint), compiled against unknown LLVM version, with baryluk's build options, with Steam beta Nov 2 2021: does not work
  • Mesa git commit e15e3a8e86 (594 commits after 21.3-branchpoint), compiled against unknown LLVM version, with baryluk's build options, with whatever was the Steam beta before Nov 2 2021 (?): works
  • Mesa git commit 3a95e25e84 (837 commits after 21.3-branchpoint), compiled against unknown LLVM version, with baryluk's build options, with Steam beta Nov 9 2021: does not work

That's quite a lot of changes between the working configurations and the non-working ones.

People who are not @kallestrop are not necessarily experiencing the same issue: they might be experiencing a different issue that has the same symptom (Steam not starting). Information from people who are not @kallestrop might help to solve their problems, but will not necessarily help to solve what @kallestrop reported.

@kallestrop writes:

user@debian-sid

You said Debian 11 before. Which is this: Debian 11, or Debian sid (unstable development branch)? Either one is a useful thing to investigate, but we should be clear about which one is the relevant one.

Ssmcv 2021-11-10 github

@kallestrop, can you share the output of apt policy libegl1 libegl1:i386 libgbm1 libgbm1:i386 libegl-mesa0 libegl-mesa0:i386?

From the system information in https://gist.github.com/kallestrop/2ac9be55d074c5f02fc3e1a7b89a26b4#file-gistfile1-txt, it looks as though all of those packages are installed.

Ssmcv 2021-11-10 github

@kallestrop writes:

Compile and activate mesa-git from https://gist.github.com/baryluk/1041204eff4cc4fad6f1508afe67b562

Please do whatever you do to activate mesa-git, but then instead of running steam in this environment, run this:

~/.steam/root/ubuntu12_32/steam-runtime/run.sh -- steam-runtime-system-info > srsi.txt

Then attach srsi.txt or send it as a Gist. This is the equivalent of Help -> System Information (which you can't use because you can't start Steam), and it will tell us more about how your mesa-git environment is set up.

Kkallestrop 2021-11-10 github

@smcv sorry for the confusion, i am on Debian 11 Stable branch. I haven't update hostname yet.

I use Baryluk's script for compiling mesa-git. As this script don't install mesa system-wide, i am not sure what version of libegl and libgbm is used when starting Steam beta with activated mesa-git.

Output of ~/.steam/root/ubuntu12_32/steam-runtime/run.sh -- steam-runtime-system-info > srsi.txt:

https://gist.github.com/kallestrop/92a5fbe4944283b25183f4f0d4931995#file-srsi-txt

Edit:
Don't know if it matters, but "activating mesa" means starting enable-new-mesa-opt.source which is generated after Baryluk's script is finished. I attached enable-new-mesa-opt.source :

https://gist.github.com/kallestrop/0d1e72b35891667ce706c9c93a907efe#file-gistfile1-txt

Ssmcv 2021-11-10 github

As this script don't install mesa system-wide, i am not sure what version of libegl and libgbm is used when starting Steam beta with activated mesa-git.

From the steam-runtime-system-info output, it looks like the answer for at least EGL is your locally-compiled Mesa:

  "architectures" : {
    "i386-linux-gnu" : {
...
        "egl_x11/gl" : {
          "renderer" : "AMD Radeon RX 6900 XT (sienna_cichlid, LLVM 11.0.1, DRM 3.40, 5.10.0-9-amd64)",
          "version" : "4.6 (Compatibility Profile) Mesa 22.0.0-devel (git-3a95e25e84)",
          "library-vendor" : "glvnd"
        },
        "egl_x11/glesv2" : {
          "renderer" : "AMD Radeon RX 6900 XT (sienna_cichlid, LLVM 11.0.1, DRM 3.40, 5.10.0-9-amd64)",
          "version" : "OpenGL ES 3.2 Mesa 22.0.0-devel (git-3a95e25e84)",
          "library-vendor" : "glvnd"
        }
      },
      "dri_drivers" : [
        {
          "library_path" : "/home/user/mesa-git/installdir/build-i386-opt/install/lib/dri/kms_swrast_dri.so"
        },
        {
          "library_path" : "/home/user/mesa-git/installdir/build-i386-opt/install/lib/dri/r600_dri.so"
        },
        {
          "library_path" : "/home/user/mesa-git/installdir/build-i386-opt/install/lib/dri/radeonsi_dri.so"
        },
...
    "x86_64-linux-gnu" : {
...
        "egl_x11/gl" : {
          "renderer" : "AMD Radeon RX 6900 XT (sienna_cichlid, LLVM 11.0.1, DRM 3.40, 5.10.0-9-amd64)",
          "version" : "4.6 (Compatibility Profile) Mesa 22.0.0-devel (git-3a95e25e84)",
          "library-vendor" : "glvnd"
        },
        "egl_x11/glesv2" : {
          "renderer" : "AMD Radeon RX 6900 XT (sienna_cichlid, LLVM 11.0.1, DRM 3.40, 5.10.0-9-amd64)",
          "version" : "OpenGL ES 3.2 Mesa 22.0.0-devel (git-3a95e25e84)",
          "library-vendor" : "glvnd"
        }
      },
      "dri_drivers" : [
        {
          "library_path" : "/home/user/mesa-git/installdir/build-amd64-opt/install/lib/dri/kms_swrast_dri.so"
        },
        {
          "library_path" : "/home/user/mesa-git/installdir/build-amd64-opt/install/lib/dri/r600_dri.so"
        },
        {
          "library_path" : "/home/user/mesa-git/installdir/build-amd64-opt/install/lib/dri/radeonsi_dri.so"
        },
...

Although, there does seem to be some confusion about your EGL driver, because the JSON manifest file we're seeing is the one from the system-wide Mesa installation:

  "egl" : {
    "icds" : [
      {
        "json_path" : "/usr/share/glvnd/egl_vendor.d/50_mesa.json",
        "library_path" : "libEGL_mesa.so.0",
        "issues" : [
        ]
      }
    ]
  },

Probably baryluk's script should be adjusting XDG_DATA_DIRS or __EGL_VENDOR_LIBRARY_FILENAMES or __EGL_VENDOR_LIBRARY_DIRS so that the JSON manifest from the newly-compiled Mesa is used? It seems to set that up successfully for Vulkan, but not for EGL.

Kkallestrop 2021-11-10 github

I found the cause of the error.

In the comments of baryluk's script is a hint regarding EGL:

# No OpenGL ES (GLES), or EGL will be built. It is simply to save a bit of time
# compiling Mesa. You can enable it in MESA_COMMON_OPTS below.

So after setting MESA_COMMON_OPTS from disabled to enabled

"-Dgles1=enabled"
"-Dgles2=enabled"
"-Degl=enabled"

Steam beta client is working again.

Aarrowgent 2021-11-10 github

7 days of "saving time"
hope you noted this issue upstream not to disable GLES/EGL since its ubiquitously used for many platforms and software

Ssmcv 2021-11-10 github

Right, good to know. So the root cause for @kallestrop's crash was something like this:

Steps to reproduce

Have an OS-level installation of Mesa 20.3.5, which is relatively complete (including EGL and GLES).

Have an installation of a newer Mesa from git, which is less complete (no EGL, no GLES).

Expected result

Steam runs successfully.

Actual result

Steam loads the GLVND loader libEGL.so.1, which loads Mesa's libEGL_mesa.so.0 from your older OS-level version of Mesa, which loads DRI modules like radeonsi_dri.so from your newer locally-installed Mesa; but libEGL_mesa.so.0 and radeonsi_dri.so were not designed to be used with mismatched versions, so this breaks their assumptions and Steam crashes.

Solution

Either use the OS-level stack for everything (so nothing from a locally-built Mesa is loaded), or use the locally-built stack for everything, enabling more build options if necessary (so nothing from the OS-level Mesa is loaded).

I don't think that's really a Steam bug.

Ssmcv 2021-11-10 github

@dcbaker, @duartec3000, @Develon5543: I suspect you are experiencing a different issue with the same high-level symptom (or maybe more than one different issue), since you are using non-Debian distributions, so presumably not using the same script @kallestrop did.

If you believe the issue you are experiencing is a bug in Steam, please open a separate issue with full details. It will be more likely to be feasible to solve if you are able to bisect the components you are using, or otherwise isolate the problem to something that has as few things varying as possible.

Pppascher 2021-11-11 github

Not sure if it helps the others having issues but steam works for me with llvm 14:

yesterday:
lib32-llvm 14.0.0 (2db66f8d48be)
llvm 14.0.0 (fd9b099906c6)
mesa 22.0.0 (e0de7aa4d77)

I just updated to latest git and steam still works fine:

lib32-llvm (2a0e77362e3a)
llvm 14.0.0 (9db2541d4c30)
mesa 22.0.0 (ee41e1bbd2a)

Using AUR mesa-minimal-git and llvm-minimal-git (and lib32-versions).

Ggardotd426 2021-11-11 github

@ppascher #8223 is the issue tracking the Arch stuff.

Launch options