By the way, it may be off-topic, but I am "shocked" that "A hat in time" game uses 7,5 GB according to Steam client, whereas its shadercache is bigger than the whole game (8,4 GB)

Hello @Saroumane, it may be interesting to also check du -s --apparent-size of the folder.

sure :

~/.steam/steamapps/shadercache/253230$ du -s --apparent-size
8774549	.

Edit : for the record, compiling shaders for this game adds a 5.1 GB ./mesa_shader_cache subfolder

For whole shadercache :

~/.steam/steamapps/shadercache$ du -s --apparent-size
75329744	.

Here is my "top 25" of biggest folders in shadercache in case it helps :

~/.steam/steamapps/shadercache$ du -chS --apparent-size | sort -rh
72G	total
4,2G	./253230/swarm/steam_shader_cache
4,2G	./253230/fozpipelinesv4
1,7G	./275850/swarm/steam_shader_cache
1,7G	./275850/fozpipelinesv4
1,4G	./391220/swarm/steam_shader_cache
1,4G	./391220/fozpipelinesv4
1,3G	./221040/swarm/steam_shader_cache
1,3G	./221040/fozpipelinesv4
1,1G	./570/swarm/steam_shader_cache
1,1G	./570/fozpipelinesv4
895M	./976730/swarm/steam_shader_cache
888M	./976730/fozpipelinesv4
820M	./271590/swarm/steam_shader_cache
811M	./271590/fozpipelinesv4
699M	./238010/swarm/steam_shader_cache
694M	./238010/fozpipelinesv4
688M	./251150/swarm
674M	./960910/swarm/steam_shader_cache
669M	./960910/fozpipelinesv4
630M	./1237950/fozpipelinesv4
613M	./1237950/swarm/steam_shader_cache
546M	./489830/swarm/steam_shader_cache
535M	./489830/fozpipelinesv4
521M	./403640/fozpipelinesv4
492M	./552520/swarm/steam_shader_cache
490M	./552520/fozpipelinesv4

I added a 6th step :
In Steam client, I checked option "Allow background processing of vulkan shaders"
Let PC compile shaders for ~20 hours.
Result : shadercache is now 45 GB bigger (total 117 GB). Steam client reports now 33,5 GB pre-cached

Since I posted this comment, I have restarted Steam client. And it decided (for no reason : I haven't updated anything) to recompile everything...

Which filesystem are you using?

ext4 with these (fairly classic) options in fstab : errors=remount-ro,nodev,nosuid,relatime

By the way, someone replied to my public appeal in "A hat in time" steam subforum.
Here is his shadercache for this game :

4.2G	./swarm
1.1M	./DXVK_state_cache
4.0K	./fozmediav1
182M	./nvidiav1
4.3G	./fozpipelinesv4
8.7G	.

compared to mine :

4,0K	./fozmediav1
4,0K	./nvidiav1
88K	./DXVK_state_cache
4,2G	./swarm
4,3G	./fozpipelinesv4
5,1G	./mesa_shader_cache
14G	.
14G	total

I guess he has an nvidia card, but I'm suprised that my mesa_shader_cache is 28 times (!) bigger than his nvidiav1 folder.

To share another data point, this bug also occured in my brand new A Hat in time installation:

[atemu@HEPHAISTOS 253230]$ pwd
/home/atemu/.local/share/Steam/steamapps/shadercache/253230
[atemu@HEPHAISTOS 253230]$ du -s --apparent-size && du -s
10505470        .
13913187        .

And I was wondering why it took so long to compile...

This is with an RX570 on ZFS v2.0.0 on linux_lqx 5.9.1 in NixOS unstable. I can give you the details if you want to reproduce my exact system bit-for-bit.

I recently (23rd I think) switched out a GTX 970 for my current card and removed the nvidia driver from my system closure.

I also aborted the shader compilation multiple times because it was taking so long.

For the record, the problem is now a lot worse, Steam client reports 606 MB of shaders, whereas my OS says 47 GB.
I didn't do anything special, except applying regular classic OS updates, and Steam client has currently version Sept 10 2021). No beta, no hardware changes.

Follow up on https://github.com/ValveSoftware/steam-for-linux/issues/7972#issuecomment-943104712

Steam collects shaders in a cache, these are distributed among clients and downloaded to a local cache. Then fossilize runs over these caches to replay them through your local driver and stores a pre-compiled version of it back into a cache that can later be loaded from when the game needs the shaders (instead of compiling them ad-hoc which would create stutters and frame drops). This pre-compiled cache is invalidated by driver updates because it is specific to the driver version.

This means we have two kind of caches which store probably almost the same amount of data (at least I think this is how it works):

• Raw shaders as they come from the game, then distributed via Steam cloud and downloaded to your local client. This is what Steam would download, it should not change very often (and the new bucket format allows to download only delta updates). It is stored in the fossilize cache data files. Sources of these data: you running games, Steam cloud to collect and distribute caches from other players
• Processed shaders (fossilize_replay runs all shaders from the first cache through the driver pipeline, eventually for all driver variants you installed, e.g. when having multiple GPUs) which are stored in a driver-specific format, e.g. NVIDIA creates a bin and toc file for that, these are driver-version specific so fossilize needs to re-run on updates to any graphic pipeline software component, so these are not re-distributed as far as I know (there are just too many driver variants for this to be efficient, and the file format is vendor specific and subject to change with any driver version)

This would explain why I (and others) see twice the amount of data as accounted for by Steam itself.

But it doesn't explain why I have another 100 GB of seemingly stray shader cache data. Steam should be able to clean that up - which actually may be the difficulty here: Steam cannot be sure if that data belongs to some vendor-specific cache.

Maybe Steam should show two or three numbers:

• size of actual well-known shader distribution cache (that's probably what it currently accounts for)
• size of vendor-specific caches (what fossilize replays from the distribution cache, it should know what files become created because it forces the driver to put the files into pre-defined directories, so this should be well-known, too)
• the rest is most probably stray/obsolete data from old driver versions or previous Steam versions, and Steam should optionally show that as a third storage counter, and maybe have a button to list the files and maybe delete them

@Saroumane do you use multiple libraries ? There is an issue if you move a game to another library, its shadercache and compat directories are not moved and new ones get created. This leads to extra disk use. I added a library and moved a lot of stuff around and found I had a lot of extra disk used. Check out https://github.com/ValveSoftware/steam-for-linux/issues/5717 I put a script on this issue that can clear them up (you could comment out the rm -rf commands to just get a report of any orphaned directories).

@brain-anti-freeze
I have multiples libraries but I prefer to symlink all shadercache and compatdata folders on the main drive where steam is installed.

Since steam client big update from October, I have reset shadercache. Now steam settings say that 5,4 GB of shaders are pre-cached, whereas my shadercache is 30 GB. (5,5 times bigger)
So no progress here, I think Valve should probably follow @kakra analysis/advice : https://github.com/ValveSoftware/steam-for-linux/issues/7533#issuecomment-943304012

Situation still completely out of control
Steam client now reports 287 783 MB pre-cached.
Shadercacher folder is 439 800 MB.

Since my last comment, I have disabled "allow background processing of Vulkan shaders" and start again the shadercache from scratch.
And I do not reproduce this bug anymore, maybe it has been fixed ?
I'll try re enabling "background processing", if nothing goes wrong I'll close this issue.

Back to re - disabling "background processing" : there is again shadercache space used on SSD but unseen by steam client (303 GB vs 284 GB) and moreover, it's still instable and triggers steam crashes regularly.
(In logs fossilize_repla[xxxxx]: segfault .... )

Then I also completely disabled "shader pre caching" from steam UI. It now reports "0 Mb pre-cached"
/bin/du -chS --apparent-size reports 200GB used.

Let's manually rm -rf everything in shadercache to start fresh...

6 months later, with

• "allow background processing of Vulkan shaders" disabled
• "Enable Shader Pre Caching" enabled

Shadercache folder is now stabilized around 150 GB.
Steam UI and /bin/du -chS --apparent-size give close to the same result (+/- 5%)