protonscr

D3D9: drawing to stencil does not work if DrawIndexedPrimitive is used. DrawPrimitive works.

dxvkclosed needs apitraced3d9needs more info
doitsujin/dxvk#3175 · opened 2023-01-08 by TheIronWolfModding · updated 2023-01-08 · 3 comments · github
TTheIronWolfModding 2023-01-08 github

Please describe your issue as accurately as possible.

I am rendering VR Hidden Area Mask into the depth stencil by using code inejction. I noticed what looks like subtle difference between DX9 and DXVK. In DX9, depth stencil is written correctly via DrawIndexedPrimitive, but DXVK only DrawPrimitive works. I do not think this is culling because culling is disabled. See this:

    HRESULT hr = S_OK;
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_STENCILENABLE, TRUE));
    // LOG_IF_D3D_FAILED(device9->Clear(0, nullptr, D3DCLEAR_STENCIL, D3DCOLOR_ARGB(0, 0, 0, 0), 0.0f, 0));

    CComPtr<IDirect3DStateBlock9> savedState;
    LOG_IF_D3D_FAILED(device9->CreateStateBlock(D3DSBT_ALL, &savedState));

    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE));
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_ZENABLE, FALSE));
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_ALPHATESTENABLE, FALSE));
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_ZWRITEENABLE, FALSE));

    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_ALWAYS));
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_INCR));

    LOG_IF_D3D_FAILED(device9->SetFVF(CUSTOMFVF));
    LOG_IF_D3D_FAILED(device9->SetStreamSource(0, v_buffer, 0, sizeof(CUSTOMVERTEX)));
    
****************************************************************************
    // Works with DXVK and DX9
    LOG_IF_D3D_FAILED(device9->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 1));

    // Does not seem to work with DXVK.
    LOG_IF_D3D_FAILED(device9->SetIndices(i_buffer));
    LOG_IF_D3D_FAILED(device9->DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 0, 3, 0, 1));
****************************************************************************

    LOG_IF_D3D_FAILED(savedState->Apply());

    // From now on, fail writes outside of stencil.
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_KEEP));
    LOG_IF_D3D_FAILED(device9->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_EQUAL));
  }

If you use Windows, please check the following page: https://github.com/doitsujin/dxvk/wiki/Windows

Software information

GTR2 + injected code

System information

  • GPU: RTX 3090
  • Driver: 527.26
  • Wine version: Win10 21H2
  • DXVK version: 2.0

Apitrace file(s)

I can't easily get apitrace because code is injected and needs d3d9.dll

For instructions on how to use apitrace, see: https://github.com/doitsujin/dxvk/wiki/Using-Apitrace

Log files

  • d3d9.log:
  • d3d11.log:
  • dxgi.log:
KK0bin maintainer 2023-01-08 github

This short snippet unfortunately isn't enough to debug this.

TTheIronWolfModding 2023-01-08 github

This short snippet unfortunately isn't enough to debug this.

thanks - I will try to modify some D3D9 sample and get APITrace.

TTheIronWolfModding 2023-01-08 github

Apologies, I have some local DXVK that somehow cause this ... Sorry for the noise.

DLLs