protonscr

[D3D9] Halo Custom Edition: Flickering textures when using Open Sauce

dxvkclosed
doitsujin/dxvk#1550 · opened 2020-04-04 by TheGreatMcPain · updated 2020-05-17 · 13 comments · github
TTheGreatMcPain 2020-04-04 github

When using DXVK along side Open Sauce, which is a mod for Halo Custom Edition that is required to play mods like SPV3 and Project Takedown, there is some major texture flicker that noticeably affects Cortana.

Video Example

The cutscenes in the video are from the beginning of "The Piller of Autumn", and the beginning of "Two Betrayals".

Apitraces are from the cutscene in "Two Betrayals".

Software information

Download links:

Halo Custom Edition "Requires a Halo PC CD key."
1.0.10 Patch for Custom Edition
Halo CE Refined "Campaign I used in the video"
Universal UI "Allows me to access all campaign missions at the start"
An Open Sauce installer called AmaiSosu "Requires dotnet461 to run"

Installing:

  1. winetricks -q msxml4 mfc42 dotnet461 d3dcompiler_4{2,3,7} d3dx9
  2. Run setup for Halo Custom Edition
  3. Run 1.0.10 patch
  4. Extract the '.map' files from Halo CE Refined into the 'maps' directory inside Halo Custom Edition.
  5. Replace the 'ui.map' in the 'maps' directory with Universal UI's 'UI.map'.
  6. Install Open Sauce into the Halo Custom Edition directory using AmaiSosu.
  7. Set dinput8 to (native, builtin)
  8. Run haloce.exe with -vidmode height,width,refreshrate

System information

  • GPU: GTX 1050
  • Driver: NVIDIA Vulkan Beta 440.66.07
  • Wine version: wine-staging 5.4
  • DXVK version: 1.6 and v1.6-8-g5750a7d8+

Apitrace file(s)

Log files

Mmisyltoad 2020-04-06 github

Unfortunately, apitrace breaks when recording this trace. There's an option you can enable that fixes it, I'll try and find out what it is tomorrow for you 👍

TTheGreatMcPain 2020-04-06 github

I was wondering why halo's apitaces weren't showing the thumbnails properly.

I tried recording them on windows with and without OpenSauce, and they all came out the same way.

Mmisyltoad 2020-04-06 github

The option is in Halo CE I know that much. Something render target option in the per-adapter graphics options.

TTheGreatMcPain 2020-04-06 github

I found "DisableAlphaRenderTargets" and "DisableRenderTargets" in config.txt.

I'll try those out tomorrow.

Mmisyltoad 2020-04-06 github

Yeah it's one of those iirc -- I think the latter.

TTheGreatMcPain 2020-04-06 github

Okay I set DisableRenderTargets, and got a working apitrace via wined3d. I tried to get one with dxvk enabled, but it ended up broken.

working-wined3d-haloce.trace.tar.xz

Mmisyltoad 2020-04-06 github

That trace seems to render fine on my end, not entirely sure what I am looking for though.

TTheGreatMcPain 2020-04-07 github

The flickering shows up if i run d3dretrace.exe in a wine prefix with dxvk enabled, I wonder if it's possible to get a trace with dxvk enabled.

TTheGreatMcPain 2020-04-07 github

Okay, I tried to record a trace with dxvk installed on the prefix, and the problem I'm seeing is not showing up in the trace.

Here's the new trace anyway: new-dxvk-haloce.trace.tar.xz

EDIT: Not sure if these are of any help, but I made some validation layer logs.
With OpenSauce disabled: haloce-layers-no-os.log
With OpenSauce enabled: haloce-layers-with-os.log

EDIT 2: I have also made some renderdoc captures just in case. 4 of them have OpenSauce enabled and the other 4 have OpenSauce disabled.
haloce-dxvk-renderdoc.tar.xz
Removed since I found a solution, and I'd like to save space on my GDrive.

TTheGreatMcPain 2020-04-09 github

Well, I found the solution, and of course it had to be the most simple solution.
Based on Halo 2's similar graphical glitches I found that
adding "d3d9.invariantPosition = True" to dxvk.conf fixed it.

TTheGreatMcPain 2020-04-23 github

@Joshua-Ashton I'm going to go ahead and close this issue since I found a solution.

Mmisyltoad 2020-04-24 github

I'll add this as a default

TTheGreatMcPain 2020-05-17 github

I read the 1.7 changelog which mentioned that this issue has been fixed, so I retested with the newest commit without using d3d9.invariantPosition = True inside of dxvk.conf, and found that this issue still persists.

Nothing extracted yet.