Replying to https://github.com/ValveSoftware/steam-for-linux/issues/10984#issue-2354952461

"sorry my english so bad"

This is also an issue for me, using the current stable 555 driver, on openSUSE. With 550, my machine encodes using NVENC, but Wayland is awful. On 555, it falls back to libx264 on both Wayland and X11.
I can still use NVENC in HandBrake/ffmpeg.

I tested 550 and 555. I am on X11, haven’t tested Wayland. NVENC works in Plex, handbrake, and ffmpeg. Does anyone know if there is an nVidia driver that does work I can roll back to but also supports kernel 6.8?

I am also experiencing this issue on the 555.58.02 driver (proprietary and open). Running steam from the command line shows that the HEVC encoding pipeline is started and the libnvenc library appears to load, however it returns a generic error 20 as soon as the first frame is sent and falls back to libx264.

I can also confirm that I am experiencing the same issue with the open nvidia 555.58.02 driver. After the remote play client connects, the host attempts to begin encoding with NVENC but the first frame fails. It then falls back to VA-API which also fails.

Note: I do have the NVDEC based backend for VA-API installed on my system - arch package libva-nvidia-driver - but this is designed specifically to support VA-AAPI decoding within Firefox so I wouldn't expect it to work for remote play encoding.

Finally it falls back to software encoding which succeeds.

In the past NVENC encoding was working correctly on my remote play host machine. I have not tested NVENC for other applications, but ffmpeg lists it as a valid encoder.

The issue is present on the 560.28.03 beta driver as well.

I think the issue may stem from the removal of legacy encoding profiles in newer driver versions. Can someone with access to the code base check on this?

No solution for this?

I think the issue may stem from the removal of legacy encoding profiles in newer driver versions. Can someone with access to the code base check on this?

I think this sounds very plausible and is almost definitely what happened, especially seeing as NVIDIA has announced the removal of some older presets for "Q1'24": https://discourse.gstreamer.org/t/nvcodec-nvenc-nvidia-deprecates-support-for-old-videocodec-sdk-h-264-hevc-encoder-presets-with-driver-r550-in-q124/182 (Link contains a "migration guide", also found here: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-preset-migration-guide/) (EDIT: I just saw you already posted that in #11176)

This would mean that everything is working as expected, Valve just need to update their software to make use of the new presets. DISCLAIMER: This is, of course, speculation.

I was curious if there's been any solutions people have found - for instance is there a way to re-enable the older preset manually (if that is indeed the problem)?

I haven't been able to use Remote Play for a couple of months at this point - it would be great to be able to stream from my gaming PC to my SteamDeck again!

This is still an issue after the October 2nd beta patch that (probably) touched encoding code adding AV1 encoding.

Yeah, this is frustrating - I'm facing the same issue. I upgraded the GPU in my headless docker host around the same time this issue popped up and I was wondering why my framerate was so bad.

I'm facing the same issue too :(

I solved it for me using my Intel iGPU. Had to install the 32 bit driver in my headless steam. Maybe this works also on a non headless installations. See https://github.com/Steam-Headless/docker-steam-headless/issues/161#issuecomment-2425182344

I solved it for me using my Intel iGPU. Had to install the 32 bit driver in my headless steam. Maybe this works also on a non headless installations. See https://github.com/Steam-Headless/docker-steam-headless/issues/161#issuecomment-2425182344

I am happy that u found a workaround but this is not a solution for people without igpu.

I hope they can fix it soon because using sunshine isn't ideal and has some very annoying bugs too.

I have the same issue using an RTX 2060 Super and nvidia-open 560 drivers. OBS uses NVENC just fine.
NVENC - Submitting frame for encoding failed: 20
Failed to initialise VAAPI connection: -1 (unknown libva error).
Capture method set to Desktop OpenGL NV12 + libx264 main (4 threads)

Come on Valve--this blocks all of your Linux users with NVIDIA GPUs from using Steam Link, which according to your Hardware Survey is three quarters of them. This can't be hard to fix, can it?

I agree it's super frustrating. I've switched over to using Sunshine / Moonlight for the time being and it's working quite well. I wish I didn't need to install extra software on my host machine to get things working though...

somebody test this, they just included a fix in the latest beta

Seems to be working on latest beta client on Fedora 41, RTX 4070, driver version 565.57.01. Debug overlay is showing nvenc hevc and there aren't any processes using copious amounts of CPU.

Can confirm: NVENC works here (560 driver, 2080Ti), though it doesn't seem to do HEVC right now (X11 Desktop capture + libyuv + NVENC H.264). Are you sure yours does H.265? (Mine did before.)

Anyway, this is much better than before.

Client debug stats are definitely claiming HEVC:

I am having this issue currently on cachyOS. 580.95.05 driver