protonscr

Load modified .spv shaders from disk

dxvkclosed wontfix
doitsujin/dxvk#3402 · opened 2023-05-07 by DavidHiggis · updated 2023-05-08 · 11 comments · github
DDavidHiggis 2023-05-07 github

I am trying to load modded spirv shaders into dxvk by adding these codes:

//in dxso_module.cpp
Rc<DxvkShader> DxsoModule::compile(
    const DxsoModuleInfo&     moduleInfo,
    const std::string&        fileName,
    const DxsoAnalysisInfo&   analysis,
    const D3D9ConstantLayout& layout) {

    auto compiler = std::make_unique<DxsoCompiler>(
      fileName, moduleInfo,
      m_header.info(), analysis,
      layout);


    this->runCompiler(*compiler, m_code.iter());	// can this be skipped????
    m_isgn = compiler->isgn();

    m_meta            = compiler->meta();
    m_constants       = compiler->constants();
    m_maxDefinedConst = compiler->maxDefinedConstant();
    m_usedSamplers    = compiler->usedSamplers();
    m_usedRTs         = compiler->usedRTs();

    compiler->finalize();

    char* fna=(char*)fileName.c_str();
    *(int*)&fna[0x10]=0;
    auto kfna="zdk\\"+fileName;
    unsigned int fsz=FileExists_inspvlist((char*)kfna.c_str());
    if(fsz!=0)
    {
        return compiler->compile((char*)kfna.c_str(),fsz);
    }

    return compiler->compile();
  }

//in dxso_compiler.cpp
  Rc<DxvkShader> DxsoCompiler::compile(const char* existfile,uint32_t filesize) {
    DxvkShaderCreateInfo info;
    info.stage = m_programInfo.shaderStage();
    info.bindingCount = m_bindings.size();
    info.bindings = m_bindings.data();
    info.inputMask = m_inputMask;
    info.outputMask = m_outputMask;
    info.pushConstOffset = m_pushConstOffset;
    info.pushConstSize = m_pushConstSize;

    if (m_programInfo.type() == DxsoProgramTypes::PixelShader)
      info.flatShadingInputs = m_ps.flatShadingMask;

    auto vbf=(uint32_t*)ReadAllBytes(existfile,filesize);
    auto rff=SpirvCodeBuffer(filesize>>2,vbf);
    free(vbf);
    return new DxvkShader(info, rff);
  }

Seems it just works, my question is can the runCompiler(*compiler, m_code.iter()) be skipped if we load the shader bytecodes from disk?

BTW, in dx9, seems these:

if (bitfieldExtract((_86 != 0u) ? _104 : spec_state.dword1, 0, 1) != 0u)
    {
        r4 = vec4(textureLod(s0_2d_shadow, vec3(r5.xy, (bitfieldExtract((_86 != 0u) ? _112 : spec_state.dword4, 0, 1) != 0u) ? clamp(r5.z, 0.0, 1.0) : r5.z), r5.w));
    }
    else
    {
        r4 = mix(mix(textureLod(s0_2d, r5.xy, r5.w), textureGather(s0_2d, r5.xy + (vec2(0.99609375) / vec2(textureSize(s0_2d, 0) * ivec2(2)))).zxyw, bvec4(bitfieldExtract((_86 != 0u) ? _112 : spec_state.dword4, 16, 1) != 0u)), vec4(0.0, 0.0, 0.0, 1.0), bvec4(bitfieldExtract((_86 != 0u) ? _93 : spec_state.dword2, 0, 1) != 0u));
    }

can just be replaced by: textureLod(s0_2d, r5.xy, r5.w); //texture() or textureLod(), not causing errors in my case so far.

also these after ps_main():

ps_main();
/*
    uint _2228 = bitfieldExtract((_86 != 0u) ? _104 : spec_state.dword1, 20, 3);
    uint _2234 = bitfieldExtract((_86 != 0u) ? _93 : spec_state.dword2, 26, 4);
    if (_2228 != 7u)
    {
        float _2272;
        float _2273;
        if (_2234 <= 8u)
        {
            _2272 = float((render_state.alpha_ref << _2234) | (render_state.alpha_ref >> (8u - _2234)));
            _2273 = roundEve
....
*/

can be removed too.

There should be an official implement of loading existing .spv from disk.

Ddoitsujin maintainer 2023-05-07 github

Why do you want to replace shaders?

We had this functionality in the past but it was a bit janky and not really useful, so it eventually got removed.

Mmisyltoad 2023-05-07 github

Also why do you want to remove random bits from the shaders? Those are needed. They aren't there for no reason.

DDavidHiggis 2023-05-07 github

People already mod the game shaders with some d3d dll proxy like 3Dmigoto,
it can replace not only the shader bytecodes but also textures, by the hashes. However 3Dmigoto is long dead now.

A fork of this repo:
github.com/NVIDIAGameWorks/dxvk-remix
is going to do the same thing, but that one seems made for shilling their own RTX thingy.

In my case, by modding the game's builtin postprocessing shaders,
I can increase the bloom intensity without putting another bloom filter over it, which is dumb and heavy (like ReShade did).

More important, there are a lot of ps3-era games which are roughly ported to pc in dx9,
their shaders contain tons of nonsense things like:

	def c20, 0.00390625047, 1.52587909e-005, 5.96046519e-008
...

    mad r2.xyz, r2, 255.0, 0.5
    frc r3.xyz, r2
    add r2.xyz, r2, -r3
    dp3 r2.x, r2, c20

that one above trys to emulate the integer operation to get luminance, which is nonsense on pc.

Those roughly-ported shaders also has things like using texldl on rendertarget, and rendertargets on pc do not have lods.

Mmisyltoad 2023-05-07 github

Using texldl makes complete sense actually as it avoids implicit LOD calculation.

KK0bin maintainer 2023-05-07 github

DXVK is not a modding tool, so this is out of scope for the project.

DDavidHiggis 2023-05-07 github

It was not made for, but it can be used to. (So does 3Dmigoto, it was originally made for some VR hack.)

What I made is a provement that dxvk can become a "modding tool" by adding 50-lines ish codes.
I'm not going to PR the patch to this repo because I'm not really familiar with cpp codes so my codes of this looks hacky. (you can see it)

Support replacing assets will give more freedom to average users.

.e.g if only two or three translated shaders require d3d9.floatEmulation = True, than we can load these few translated shaders from disk.
and then turns d3d9.floatEmulation off.

KK0bin maintainer 2023-05-07 github

The problem is that lots of things in DXVK rely on the exact shaders that DXVK produces (like some of the parts you removed) and if that's not the case things will break in all kinds of subtle and less subtle ways.

BBlisto91 2023-05-07 github

The license is very permissive (tho you still need to follow the terms) so you are ofc welcome to fork and make your own variant like dxvk-remix have done if it is something you will want to pursue.
Note that i am not a dev so this isn't a official endorsement of any specific projects.

Ddoitsujin maintainer 2023-05-07 github

What I made is a provement that dxvk can become a "modding tool" by adding 50-lines ish codes.

Problem is, DXVK doesn't want to be a modding tool. There's plenty of other ways to mod D3D9/10/11 games that don't depend on implementation details.

DDavidHiggis 2023-05-08 github

So we are talking about philosophy.

Sure there are plenty of ways to make a game become what I want. Like put a bloom filter over the game's existing bloom filter, like ReShade did.

But why should I double the resources using for that? to accelerate planned obsolescence?

Also why should I endure the lazy porting made by some dirtcheap coders? to make their employer more a happy merchant? Once I know about that I can't pretend myself as a happy ignorant pig.

Simple as, Man should NOT be a slave of soulless modern mass production.

Ppchome 2023-05-08 github

Once I created simple vulkan layer to write/read .spv, because I wanted to replace a "broken" shader in an game.
But I failed to figure out which .spv file I should modify :)

Not sure if it was correct approach, but kind-of worked in simple test.

Nothing extracted yet.