I and a few other folks are also seeing this on GamerOS. It started happening with the June 1 Steam update and only with Nvidia GPU hardware. Also using the 440.82 drivers.

In case it helps anyone, I was able to partially restore the overlay functionality with the following patch to the SteamOS compositor:

diff --git a/src/steamcompmgr.c b/src/steamcompmgr.c
index aa3a520..0faf025 100644
--- a/src/steamcompmgr.c
+++ b/src/steamcompmgr.c
@@ -508,7 +509,7 @@ ensure_win_resources (Display *dpy, win *w)
        if (!w->pixmap)
        {
                w->pixmap = XCompositeNameWindowPixmap (dpy, w->id);
-               w->glxPixmap = glXCreatePixmap (dpy, w->fbConfig, w->pixmap, w->isOverlay ? tfpAttribsRGBA : tfpAttribs);
+               w->glxPixmap = glXCreatePixmap (dpy, w->fbConfig, w->pixmap, tfpAttribs);
                
                glBindTexture (GL_TEXTURE_2D, w->texName);
                __pointer_to_glXBindTexImageEXT (dpy, w->glxPixmap, GLX_FRONT_LEFT_EXT, NULL);

Essentially, I had to turn off the alpha channel of the overlay. This leads to various issues like no transparency, notifications with black borders, and the on-screen keyboard not being usable. But does make the overlay appear again.

I also noticed that there is an X11 error happening: BadDrawable (invalid Pixmap or Window parameter).
I haven't been able to narrow down what is causing it or even whether it is related to this issue though.

Overlay windows are depth 32 now, so this is likely related to that change. It should be more correct than before, but maybe there's a slight attribute mismatch in the glX operation to account for now.

Thanks for that info. I suspected there was some kind of pixel format change to the overlay, but what was throwing me off was that everything is working fine on AMD GPUs.

@Plagman I believe I know what the problem is. I think it is either an issue in the Steam client in the way that it selects X visuals for the overlay or possibly the Nvidia drivers in the way that it reports X visuals.

After some debugging of the SteamOS compositor, this is what I have found:

• In win_fbconfig the X visual attached to the overlay window has a reported visual depth of 32-bits, BUT the actual color buffer size, as reported by glxinfo for the X visual id, is 24-bits and it does not have an alpha channel.
• The selected X visual appears to be the first in the list that has a visual depth of 32, perhaps the Steam client is just picking the first X visual with a reported visual depth of 32, and assumes that is correct without checking the actual color buffer sizes that are allowed to be used with that visual.
• This results in win_fbconfig selecting a GLXFBConfig object without an alpha channel as there is no GLXFBConfig object with an alpha channel that is compatible with the X visual of the overlay window. This also explains the X11 BadDrawable error and explains why the overlay works (sans transparency) when I override the parameters for glXCreatePixmap to use RGB instead of RGBA, because the window is actually 24-bit and doesn't have alpha.
• On my AMD machine, all X visuals with a reported visual depth of 32-bits also have an actual color buffer that is 32-bits. Explaining why it works in the AMD environment.

TLDR: the way the Nvidia driver reports color depth is a bit wonky and I believe the Steam client is not accounting for it and inadvertently selecting the wrong X11 resource for a 32-bit color window.

I don't think there is anything that can be done in the SteamOS compositor to fix this. It most likely needs to be fixed in the Steam client. X11 doesn't like it when you select a GLXFBConfig with a different X visual id from the current window. It was worth a shot I guess.

Not sure how to proceed in getting this looked at and addressed. Please advise!
This is quite painful for all SteamOS compositor users with Nvidia GPUs.

Thanks!

Hello everybody!

I am seeing the same behaviour on SteamOS 2.0 with Nvidia GPU since the June 1st Steam client update.

Interestingly, this does only appear when I use the SteamOS compositor session. When I change to a Gnome session and run the Steam client in Big Picture more, the overlay works as expected.

Any progress on this would be highly appreciated since it blocks much of my and potentially others' SteamOS usage. I can play using the Gnome session but Gnome's heavy screen tearing issues make it quite unbearable for most content.

Thank you and kind regards.

Same thing for me. I'm using the Steamos-Compositor on top of Mint 20.

GTX 1660 Super with driver 440.100

Thanks for your support!

Same issues here:

GTX 1660ti / SteamOS & GamerOS

I found a work around.

For SteamOS/steamos-compositor:
Edit /usr/bin/steamos-session by removing the -steamos launch option to steam.

For GamerOS/steamos-compositor-plus:
Edit ~/.config/steamos-compositor-plus, adding the following line:
STEAMCMD="steam -tenfoot -fulldesktopres"

The downside of this change is that you will:

• lose access to audio, bluetooth and network settings
• have a bunch of irrelevant extra options in the shutdown menu
• lose access to OS version information
• likely lose access to OS update notifications

Since this is affecting SteamOS users I'm surprised valve haven't taken steps to fix this already. I'm not sure if there are any steps to be taken to wake them up.

@alkazar Thanks for the workaround. The loss of config is annoying, but it's better than no overlay. Looks like I still got a gameros update notification with this set like that. Hopefully someone at valve picks this up as this basically makes steamos unusable for nvidia users

Thanks @alkazar this was a massive annoyance

Without -steamos you also won't get the 1080p/60 overlay regardless of the game's resolution, and you won't get the overlay if the game hangs or locks up to be able to exit it, which can be pretty important. I'm looking into the issue now and hopefully we can resolve it soon.

@Plagman Thanks for looking into this! In case it helps, in addition to my findings posted above, which I think are still accurate as far as my understanding goes, I tried the following:

• I tried forcing use of an X visual with an alpha channel, but that just resulted in artifacting and general badness like games not displaying any graphics
• I tried cycling through dozens of different GLXFBConfigs, with no result

I also noticed the following:

• the X visual being used in Gnome was the same one used under the steamos compositor i.e. one without proper alpha support according to glxinfo
• the overlay seems to correctly have transparency when run with the -steamos option under Gnome, which seems to conflict with the above point
• in win_fbconfig there is only a single GLXFBConfig that matches the criteria of having the same visualid and it doesn't have an alpha channel

Fix confirmed on the following configurations:

• Arch Linux with steamos-compositor-plus and a GTX 670
• GamerOS 19 with steamos-compositor-plus and a GTX 1080

Also confirmed no regression to the overlay on the following configurations:

• Arch Linux with Big Picture Mode in Gnome and a GTX 670
• GamerOS 19 with steamos-compositor-plus and a RX 570

Thank you for your efforts @Plagman and @kisak-valve !

Confirmed working now on Arch with steamos-compositor-plus and GTX2070 (450.56.02). Thanks for the support !

Confirmed working on SteamOS beta using Nvidia GeForce GTX 960

Working on GamerOS 19 / GTX 1660 TI

THANKS!!

This happens to me with the latest Steam beta

...with an AMD GPU (5700xt / RX580)
...on Windows
🤦‍♂️