protonscr

Steam client full redownloads shader pre-cache content of games on an SD card when cards are swapped

steamclosed Steam clientDistro Family: SteamOS
ValveSoftware/steam-for-linux#8823 · opened 2022-10-02 by nex86 · updated 2023-11-26 · 17 comments · github
Nnex86 2022-10-02 github

I'm getting very annoyed by having to download Shader Cache for my games every single day.
Some games even have several gigabytes of shader cache to download on a daily basis.
It's especially annoying when I'm on the go and don't always have good internet.

So isn't there a way to just disable shader cache updates and use the locally stored cache that is already downloaded?

I know there is a setting to "set updates on hold until the game is launched" but thats not the solution as it tries to update the shaders when I launch the game.
I just want to completely stop shader cache updates until I want it to.

I know there also is a setting in steam settings to disable shader pre-caching. but wouldn't that also delete the cache that is already stored on disk?

Kkisak-valve maintainer 2022-10-02 github

Hello @nex86, this issue is already being tracked at #6486 / #7519. Closing as a duplicate.

Yes, the checkbox which disables the shader pre-caching subsystem also frees the storage used by the shader cache.

Kkisak-valve maintainer 2022-10-02 github

Out of mild curiosity, which games are you seeing repeated gigabytes of downloads to Steam's shader pre-cache? When looking at Steam's download queue, under the progress bar, the left number is the download progress (interesting), and the right number is Steam's progress of delta patching the files on disk (not particularly interesting).

Nnex86 2022-10-02 github

well almost all of my games, with Devil May Cry 5 being the largest with downloading 6GB..
if it updates all of them it downloads a total of 25GB worth of shader cache (measured on statistics page of my router)

Kkisak-valve maintainer 2022-10-02 github

Can you share a log of that next time you see it? Should be found at ~/.steam/root/logs/shader_log.txt.

Nnex86 2022-10-02 github

sure can do, but now it already downloaded most of it.

Nnex86 2022-10-03 github

Can you share a log of that next time you see it? Should be found at ~/.steam/root/logs/shader_log.txt.

this is from today when I turned my Steamdeck on. https://pastebin.com/4ajc7BSi

Kkisak-valve maintainer 2022-10-03 github

Thanks, just to confirm, did you toggle the shader pre-caching setting shortly before gathering this log?

To my eyes, this looks like a full download of all shader pre-cache content, including transcoded videos as if there was nothing there to incrementally update.

In particular, there's 88 lines similar to Committed bucket # (AppID #) from 0 to big# which reads like the previous day's shader pre-cache wasn't there at all or the subsystem got reset for some reason.

Nnex86 2022-10-03 github

Thanks, just to confirm, did you toggle the shader pre-caching setting shortly before gathering this log?

No I did not.
But I played games from my other sd card yesterday before switching back to my 1TB card.
I think what also triggers a full redowmload is swapping SD cards..
I swap them a lot since I got some games I play much on the other card.

Kkisak-valve maintainer 2022-10-03 github

Let's re-open this issue report and focus on the Steam client losing track of shader pre-cache content related to swapping SD cards.

Nnex86 2022-10-04 github

what I suspect is it only does it when you swap sd cards when its powered off, and then power it on with the new card inserted.
when you hot swap them, it doesn't seem to do it.
personally, I don't really like hot swapping them, but there seems no way to safely eject them in game mode.

WWalaryne 2022-10-26 github

Hi, wanted to add my bit from the closed issue. I feel like this might be down to the Steam client's hardware detection for invalidating the shader cache. Restarting the Steam client after swapping two SD cards seems to yield the same results. It is also not limited to the shader cache, and does the same for the Steamworks Common Redistributables (when they're on internal storage).
Only other idea I have is that it may not be the hardware detection, but rather Steam detecting these games are missing from the drive, and clearing the shader cache/invalidating it so that the space is not wasted. Since I have limited internet access I'd love to help in any way to see this resolved.

EDIT:
Wanted to add that I found further evidence to my second idea as to what's happening: the Steamworks Common Redistributables size when swapping in my SD with a larger library is about 800MB.
When swapping in my SD card with fewer games (4) it's only 30MB~. This redownload happens at EVERY swap, and some games completely refuse to boot until that download is finished, even offline. And the shader redownload, only at Steam client restarts after a swap, or a reboot (or a desktop mode change).

Ddo-kevin 2023-01-11 github

I am also hotswapping microSDs as well. I like to keep the SSD reserved only for shaders and put my games on the microSDs. Everytime I swap, the games have to download the shader cache again.

Can't the Steam Deck just keep and re-use the shader cache everytime the user hotswaps?

EEverydayGamer30 2023-01-26 github

I just noticed this the other day and it's pissing me off. My steam deck keeps deleting and re-downloading the Steamworks Common Redistributables which is 400MBs for one SD card and 900MBs for the other. I have looked all over the internet for a solution and these forums seem to be the only ones that even talk about it, and they seem the be dead ends. It would seem simple enough for the Steamworks Common Redistributables to stay on the main drive and not delete themselves. The only current solution is upgrading the internal drive to the biggest nvme 2230 and biggest SD card so you don't have files being deleted between SD card swaps. I am hoping a solution can be found as it currently makes this device similar to a trash cloud gaming handheld. I made this account just to comment on this forum and will try to respond if any replies ever come.

Pperroboc 2023-07-13 github

This is partially fixed in the latest (beta) release. Steam doesn't redownload shaders when hotswapping SD cards, but it does download the Steamworks Common Redistributables.

Since it seems to be a different issue, I've since created it in #9847

Kkisak-valve maintainer 2023-08-25 github

Closing per the last comment.

GGooby135 2023-10-09 github

I'm having this same issue. I've formatted the SD card, tried ejecting from the Storage menu, swapping when off. Every time it seems to trigger a redownload of all the shader precache of games installed on the SD card . I'm on the latest beta release and the issue is not fixed.

Ggotgenes 2023-11-26 github

There seems to be a regression and reemergence of this bug in the latest stable release, 3.5.7. Swapping the micro SD card, then restarting the Steam Deck, will cause the Deck to re-download the pre-cached shaders for the games on the micro SD card that was inserted.

Nothing extracted yet.