just trying to find a way to cache as many shaders as possible to help out linux folk

Maybe to clarify something about the DXVK state cache a little bit since I also saw you make other issues involving it: The DXVK state cache doesn't cache any shader code at all, it only contains information about how multiple shaders and sets of state are "combined" into monolithic vulkan pipelines. With a driver that support VK_EXT_graphics_pipeline_library (GPL), caching this information is only required for geometry and tessellation shaders, because GPL allows to create libraries of individual vertex and fragment shaders already as soon as the game calls the corresponding D3D Create*Shader() functions without requiring the extra info from the cache.

Now that still leaves 2 cases where DXVK might suffer from shader compilations stutter, even with GPL:

1. The game uses geometry or tessellation shaders
2. The game creates it's shaders right before using them in draw calls

A populated state cache will only help in case (1), but also consider that even for games using geometry and tessellation shaders, 99% of pipelines are usually still vertex+fragment only, so the stutter shouldn't be very frequent, even with an empty cache. Case (2) tends to happen especially with Unreal Engine 4 games, but here having a state cache would not help at all.

(I hope I'm not talking total nonsense here since I'm certainly not an expert on how DXVK works, so correct me where I'm wrong.)

Async, however, will just skip draws when pipeline compilation hasn't finished in time, which is obviously broken and thus understandable why it isn't upstream, even if it can arguably be a better user experience in some games and the brokenness might not be perceivable.

just trying to find a way to cache as many shaders as possible to help out linux folk

Maybe to clarify something about the DXVK state cache a little bit since I also saw you make other issues involving it: The DXVK state cache doesn't cache any shader code at all, it only contains information about how multiple shaders and sets of state are "combined" into monolithic vulkan pipelines. With a driver that support VK_EXT_graphics_pipeline_library (GPL), caching this information is only required for geometry and tessellation shaders, because GPL allows to create libraries of individual vertex and fragment shaders already as soon as the game calls the corresponding D3D Create*Shader() functions without requiring the extra info from the cache.

Now that still leaves 2 cases where DXVK might suffer from shader compilations stutter, even with GPL:

1. The game uses geometry or tessellation shaders

2. The game creates it's shaders right before using them in draw calls

A populated state cache will only help in case (1), but also consider that even for games using geometry and tessellation shaders, 99% of pipelines are usually still vertex+fragment only, so the stutter shouldn't be very frequent, even with an empty cache. Case (2) tends to happen especially with Unreal Engine 4 games, but here having a state cache would not help at all.

(I hope I'm not talking total nonsense here since I'm certainly not an expert on how DXVK works, so correct me where I'm wrong.)

Async, however, will just skip draws when pipeline compilation hasn't finished in time, which is obviously broken and thus understandable why it isn't upstream, even if it can arguably be a better user experience in some games and the brokenness might not be perceivable.

Thank you for the informative answer. It sounds like you know what you are talking about.

Is there a reason compute shaders were removed from 2.1? https://github.com/doitsujin/dxvk/commit/e5157a5360280fd195fe9489b2f4578e7ca6bb55#diff-df99f9ed402a6f0b0882a24fb83bb10c4c1fa53c3d6bac19370f454abdc9195a]
If compute shaders were included in the cache for say for UE4 based games, wouldn't that mean players have a better experiences as they won't have to compile at draw time? I played through all of Star Wars Fallen Order with 2.1 and still got a lot of stutter.

If I disabled Graphics Pipeline Libraries, would that result in more entries in the state cache?

Is there a reason compute shaders were removed from 2.1

The commit message already explains it: "Since API-provided compute shaders
are always compiled early, supporting them is no longer needed."
That means that no information from the state cache is needed to compile compute pipelines as soon as the game creates the compute shader.

If compute shaders were included in the cache for say for UE4 based games, wouldn't that mean players have a better experiences as they won't have to compile at draw time

No, because the cache doesn't contain any shader code, and never did. So with or without information from the cache, the compute pipeline can only be compiled after the game creates the compute shader.

If I disabled Graphics Pipeline Libraries, would that result in more entries in the state cache?

Yes, I think so (unless the code has been removed, haven't checked). However, there is no case where having those extra entries in the cache would ever help stutter when playing on a driver with GPL enabled.

For what it's worth, for some UE4 games it can reportedly help to add

[/script/engine.renderersettings]
r.CreateShadersOnLoad=1
niagara.CreateShadersOnLoad=1

to their Engine.ini.

Is there a reason compute shaders were removed from 2.1

The commit message already explains it: "Since API-provided compute shaders are always compiled early, supporting them is no longer needed." That means that no information from the state cache is needed to compile compute pipelines as soon as the game creates the compute shader.

If compute shaders were included in the cache for say for UE4 based games, wouldn't that mean players have a better experiences as they won't have to compile at draw time

No, because the cache doesn't contain any shader code, and never did. So with or without information from the cache, the compute pipeline can only be compiled after the game creates the compute shader.

If I disabled Graphics Pipeline Libraries, would that result in more entries in the state cache?

Yes, I think so (unless the code has been removed, haven't checked). However, there is no case where having those extra entries in the cache would ever help stutter when playing on a driver with GPL enabled.

Thanks for these answers, I really do appreciate them. So two questions then:

1. How do we ensure that the UE4-based shader stutter situation is mitigated as much as possible in DXVK?
2. Is there a way I can do any more to help linux bros have a pleasant experience free from stutter?

To give you an example, I have just played through Somerville a indie title with dxvk-gplasync-2.1-3 (yes, not vanilla dxvk) which created a state cache which contains about 1500 entries, 500kb in size. Presumably these contain the vertex+fragment parts you are talking about. There was very little stutter (but some) and I was using a top end Nvidia card on windows. Presumably if someone took this cache and played through the game with it, it would improve their experience no? Reducing their stutter as they didn't have to create these vertex parts right?

To give you another example, I also saw a dip from 116FPS to 45 briefly in another UE4 based while a new effect came on screen (shotgun blast, Little Nightmares 2) and it's that dip that I would like to avoid as much as possible.

To give you an example, I have just played through Somerville a indie title with dxvk-gplasync-2.1-3 (yes, not vanilla dxvk) which created a state cache which contains about 1500 entries, 500kb in size. Presumably these contain the vertex+fragment parts you are talking about.

If this was with GPL enabled, it should only contain info about tessellation and geometry shaders (unless dxvk-gplasync contains some changes i'm not aware of)

I was using a top end Nvidia card on windows

Doesn't really matter on which GPU since shader compilation happens exclusively on the CPU (although you probably didn't pair it with a super slow CPU..)

it would improve their experience no? Reducing their stutter as they didn't have to create these vertex parts right?

It could help the stutter, but probably rather for tessellation and geometry using pipelines.

To give you another example, I also saw a dip from 116FPS to 45 briefly in another UE4 based while a new effect came on screen (shotgun blast, Little Nightmares 2) and it's that dip that I would like to avoid as much as possible.

It's not clear to me if this is shader compilation related to all, I would assume it's not. Although GPL pipelines can be a bit slower on the GPU for a couple of frames before the optimized monolithic pipeline is ready, I would expect the impact on nvidia to be minimal.

Presumably if someone took this cache and played through the game with it, it would improve their experience no? Reducing their stutter as they didn't have to create these vertex parts right?

Before GPL there was already some systems to try and mitigate it in a similar way on Linux. Steam have a feature where it will try and share cache and cache info about the game. You could then precompile the known pre shared shaders before seeing them ingame.
Tho this is hardly a perfect system since it misses alot and can be quite slow depending on your CPU.

Some places also tried to collect and share dxvk state cache entries like
https://github.com/begin-theadventure/dxvk-caches