protonscr

Skyrim: Special Edition + ENB crash under certain conditions

dxvkclosed d3d11
doitsujin/dxvk#2033 · opened 2021-04-20 by ErikSteiner · updated 2026-06-16 · 40 comments · github
6 matching comments, n / p to jump
EErikSteiner 2021-04-20 github

Unlike [#865](https://github.com/doitsujin/dxvk/issues/865) it is possible to use ENB v0.452. It is only required to set UseEffect=false in enbseries.ini. During the Bethesda Into one is able to press the key combination in order to show the ENB GUI. In the GUI you can change UseEffect to true, however this is only possible if one is not in the following places:

  • main menu
  • map
  • loading screen
  • skills menu
  • hitting the ESC key

The following places are not affected by a game crash if ENB is enabled:

  • worldspace
  • inventory
  • magic (inventory?)

Creating a trace was a bit complicated, as the ENB uses its own d3d11.dll. This resulted in an instant crash at game startup, so I used the ProxyLibrary feature of the ENB to load the trace-dll from apitrace-10.0-win64/lib/wrappers. The traces have a size of 360 MB and 2,5 GB.

Software information

System information

  • GPU: NVIDIA GTX 1080 Ti
  • Driver: 460.67
  • Wine version: Proton-6.1-GE-1
  • DXVK version: 1.8.1

Apitrace file(s)

Log files

SkyrimSE_d3d11.log
SkyrimSE_dxgi.log

Ddoitsujin maintainer 2021-04-21 github

Do we know what the issue is exactly? I really don't want to waste even more time on the horrendous shitshow that is ENB, it does some really weird shit with shaders that we just cannot support, and it's a miracle that it doesn't just crash and burn on native drivers.

Edit: Looks like it's trying to read 128 graphics pipeline UAVs when in reality there are only 64 slots available. Great quality software. Truly amazing.

EErikSteiner 2021-04-21 github

The author of ENB recently released a Linux version of 0.452. Basically he disabled an effect in the dll itself and asked a subject if the CTD in main menu is fixed. If not he disabled another thing and so on. All effects that resulted in a crash on Skyrim SE on Linux with DXVK. I guess he said something of Skylighting effects that are absent.

It is noticeable that in the Linux version only the d3d11.dll is required, whereas the Windows version also includes a d3dcompiler_46e.dll. I don't think I need to mention that it sounds completely crazy that disabling sky effects will fix the problem. I mean, why do I need them in the main menu or in other menus? However, the mentioned effect is only a guess. He does not play with open cards in this regard and is not otherwise willing to address the issue.

Is the trace file usable? I have another 2,5 GB trace, where I activated the Skill menu while UseEffect was true. It resulted in a crash as I pointed out, because it is one of these places.

If the trace file is usable, I could create another trace with the Linux version that don't cause a CTD in main menu if UseEffect is true?

Ddoitsujin maintainer 2021-04-21 github

The trace works, yes, and it shows one problem that can definitely cause crashes, gonna fix that and hope that it works.

Ddoitsujin maintainer 2021-04-21 github

Does https://github.com/doitsujin/dxvk/tree/range-checks fix the issue for you? (build here)

EErikSteiner 2021-04-21 github

I am so happy to hear from you. I tested it out,

  1. first by replacing the dll's in
    ~/.steam/root/compatibilitytools.d/Proton-6.1-GE-1/dist/lib/wine/dxvk
    ~/.steam/root/compatibilitytools.d/Proton-6.1-GE-1/dist/lib64/wine/dxvk
    with the dlls of ./dxvk-range-checks/x32 and ./dxvk-range-checks/x64 and appropriate permissions 555.
  2. I launched the game with UseEffect=false and hit the key combination for the ENB GUI during the intro and entered main menu. In the main menu I activated the effects in ENB GUI and the game crashed.

So far, nothing has changed in the fact that the game crashes in the aforementioned location. Furthermore, I also tested to open the map ingame, but again the same result. I uploaded a new trace file and logs. Thank you for your effort.

Apitrace file(s)

SkyrimSE.trace

Log files

Ddoitsujin maintainer 2021-04-22 github

Yeah, no idea what to do then. The trace you attached replays perfectly fine on my end, not really sure how to debug this further, especially since the last API call in both traces succeeds.

EErikSteiner 2021-04-22 github

This is very disappointing. I fired all logs onto that thing. Maybe there is something related to the issue? So, I load the game with UseEffect=false, activate the ENB GUI during the intro and activate the Effect when in main menu.

Apitrace file(s)

SkyrimSE.trace

Log files

steam-489830.log
user_settings.py for proton

Jjellybeanmonster 2021-04-23 github

This issue applies to me somewhat except i cant even go to exterior places in skyrim without crashing. Im only able to be in interiors when i enable the UseEffect option on enb. All the menus in game give instant crash.
More information:
GPU: RTX 3080 10gb
CPU: AMD Ryzen 9 5900x
DXVK version: 1.8.1
SkyrimSE_d3d11.log
SkyrimSE_dxgi.log

EErikSteiner 2021-04-23 github

Are you able to test this build? Moreover, could you create a trace?

  1. To create a trace, I downloaded the apitrace file
  2. In ../apitrace-10.0-win64/lib/wrappers I renamed the d3d11.dll to d3d11_trace.dll and copied d3d11_trace.dll, dxgi.dll and dxgitrace.dll to ../common/Skyrim Special Edition
  3. In enblocal.ini I set the proxy to
[PROXY]
EnableProxyLibrary=true
InitProxyFunctions=true
ProxyLibrary=d3d11_trace.dll
Jjellybeanmonster 2021-04-23 github

Ok im about to create a trace . was i supposed to try that build before i create a trace?

EErikSteiner 2021-04-23 github

It would be nice to have both situations. One trace with the current release and your configuration and one with the new build of DXVK.

Jjellybeanmonster 2021-04-23 github

what do i copy and paste from the dxvk range checks ? thats what i got from the link.

EErikSteiner 2021-04-23 github

Please feel free to either replace the DXVK dll's like I did, or by installing the package with ./setup_dxvk.sh install --with-d3d10 . For the latter, please make sure that you export the path to the prefix, before executing the sh. Otherwise .wine is the install location. So do export WINEPREFIX="/home/user/.steam/root/steamapps/compatdata/489830/pfx" or similar.

Ddoitsujin maintainer 2021-04-23 github

I mean, creating traces with different DXVK builds is probably not going to change anything since apitrace records D3D11 commands, which are usually unaffected by what DXVK is doing and that is definitely the case here.

EErikSteiner 2021-04-23 github

So that means, that we have a CTD that is not able to be tracked by a trace? Or is it possible to trace it in another mean?

Ddoitsujin maintainer 2021-04-23 github

you'd need to trace on Windows, or with wined3d if that works.

EErikSteiner 2021-04-23 github

Would you be so kind, to give an advice, how to do it. I have a second boot, but that is out of my knowledge.

Ddoitsujin maintainer 2021-04-23 github

Same as on Linux I guess? You'll have the same problems with ENB though which makes the whole thing a real pain in the arse to work with.

Jjellybeanmonster 2021-04-23 github

Coming back with bad and somewhat good news about using the trace files. Ok so the "new build" i downloaded wouldnt even let me start the game so i had to go back and get my original d3d11.ll file from my enb to even start the game. Once i was able to get into the game my framerate literally stayed on 1-2 fps. When loaded into the interior space it still was going at 1-2 fps. i was able to open the enb menu and turn on use effect without issue. The somewhat "good" news is when i went outside to the exterior i didnt crash at all but i still was going at 1-2 fps. Menus didnt crash at all either. Also is the trace files supposed to spit out some logs?

EErikSteiner 2021-04-23 github

d3d11

The trace is created in Windows by the following procedure:

  1. To create a trace, I downloaded the apitrace file
  2. In ../apitrace-10.0-win64/lib/wrappers I renamed the d3d11.dll to d3d11_trace.dll and copied d3d11_trace.dll, dxgi.dll and dxgitrace.dll to ../common/Skyrim Special Edition
  3. In enblocal.ini I set the proxy to
[PROXY]
EnableProxyLibrary=true
InitProxyFunctions=true
ProxyLibrary=d3d11_trace.dll
  1. I set UseEffect=true in enbseries.ini
  2. I launched Skyrim in Steam and waited until I reached the main menu. As there is no DXVK involved no CTD occurred. See SkyrimSE-win-d3d11.trace

Apitrace file(s)

SkyrimSE-win-d3d11.trace

DXVK

The trace is created in Windows by the following procedure:

  1. Point 1. to 4. of d3d11
  2. Create a folder called DXVK in ../common/Skyrim Special Edition and copy the content of /dxvk-range-checks/x64/ to that folder
  3. The idea is from that reddit post: Execute the command as Administrator, you very likely need to change the path to the DXVK folder: setx /M APITRACE_FORCE_MODULE_PATH "C:\SteamLibrary\SteamApps\common\Skyrim Special Edition\DXVK"
  4. Create a file dxvk.conf in ../common/Skyrim Special Edition with content:
    dxvk.hud = full,scale=.8
  5. Launch Skyrim in Steam and wait until it reaches the main menu and crash as DXVK is involved. See SkyrimSE-win-dxvk-with-range-check.trace

Apitrace file(s)

SkyrimSE-win-dxvk-with-range-check.trace

Jjellybeanmonster 2021-04-24 github

I got 3 skyrim se trace files on my desktop that i didnt notice until now. It is 8gb for some reason. im uploading it to MEGA right now incase its needed her i guess.

Jjellybeanmonster 2021-04-24 github

Those are the trace files i got.
SkyrimSE.trace

EErikSteiner 2021-04-24 github

I updated my post. If you have access to a windows machine, feel free to reproduce my steps. The trace file will be written to you Desktop. @doitsujin I hope that there are now better information in these tracing files. For the DXVK variant I used you provided build.

MmedinaKrom1 2021-05-20 github

Hi and good evening @doitsujin and @ErikSteiner,
I'm new here and on github too and really don't want brake any Issue discussion rules, so sorry for my question.
Discovered the great work result of dxvk a bit and found Erik's Issue.

So the question is: Can i use Skyrim SE with the DLL's of dxvk together with the current version of ENB files?
Or in other words: How i have to combine the ENB files with the files from dxvk V1.8

From ENB & game side i have the files from Skyrim modlist Elysium from Wabbajack:
d3d11.dll - File Version: 0.454
d3dcompiler_46e.dll
d3dcompiler_47e.dll
d3dx9_42.dll
dxgi.dll - For Reshade File VErsion: 4.9.1.1003
skse64_1_5_97.dll
(If i'm using dxgi.dll and d3d11.dll from dxvk, the ENB is not working.)

Hardware: AMD Ryzen 5800X and RX 6800XT graphics card.
Maybe I'm running against a wall and the use case is generally not possible.

Thank you in advance for any tips, where I hopefully can continue.

Cchumik6000 2021-05-26 github

Does https://github.com/doitsujin/dxvk/tree/range-checks fix the issue for you? (build here)

The problem is gone with this fix for me (latest enb).

EErikSteiner 2021-05-26 github

@chumik6000: Could you add information to your environment? Do you run it under Linux (Kernel, Proton/Wine version, winetricks etc.) or under Windows (setup).

Cchumik6000 2021-05-26 github

Windows 10, AMD RX 6900xt, latest AMD drivers.

CChaddabox 2021-07-09 github

Wondering if anyone has experienced this??:

-im in Skyrim, press shift + enter to open ENB.
-i had a cursor problem and typed “tm” into commands
-pressed shift+enter to exit enb
-now i am unable to use any ingame menus (tab, esc) i can press m for maps but it doesnt show anything on it
-i also dont have a compass/ui
i have tried typing “tm” to reverse it and the only thing that changes is i dont have a cursor on enb

BBlisto91 2022-07-23 github

@ErikSteiner Friendly ping. Are you still having this issue with latest dxvk, ENB & drivers?

Aalliumm 2023-05-13 github

I have been playing with it for a while and it seems stable to me. ENB 0.488, mesa 23.1, and Proton 8.0-2 (DXVK 2.1-4).
Winetricks d3dcompiler_47, d3dcompiler_43, and d3dx11_43.

mesa GPL was causing error messages so I disabled it with RADV_DEBUG=nogpl

BBlisto91 2023-05-13 github

What error messages

Aalliumm 2023-05-14 github 
Assertion failed!
Program Z:\mnt\ssd\Steam\steamapps\common\Skyrim Special Edition\SkyrimSE.exe
File: ../src-wine/dlls/winevulkan/loader_thunks.c
Line: 2703
Expression: "!status"

7 of these identical popups come up on game launch with GPL enabled. Sorry, I'm not sure how to get a more detailed error message. I can actually still play the game with these popups up and it doesn't crash (though i haven't tested it very much).

Ddoitsujin maintainer 2023-05-14 github

Yeah that's probably ENB producing broken DXBC shaders again which we then translate into broken SPIR-V.

Not sure how much I really care at this point to be honest, just disable GPL as a workaround if that helps, you can do this with a DXVK config file as well by setting dxvk.enableGraphicsPipelineLibrary = False.

EErikSteiner 2023-05-18 github

@ErikSteiner Friendly ping. Are you still having this issue with latest dxvk, ENB & drivers?
I was without a PC for a long time, so I'm just getting back to you. With the following parameters, Skyrim now works under Linux and ENB:

Software information
Skyrim Special Edition
ENB v0.493 (d3d11.dll, d3dcompiler_46e.dll)
Simply Cathedral 1.0

System information
GPU: NVIDIA GTX 1080 Ti
Driver: 530.41.03
Wine version: Proton-8.3-GE
DXVK version: 2.1
SteamTinkerLauncher:

  • winecfg: override d3d11 as native
  • winetricks: d3dx11_43 d3dcompiler_43 d3dcompiler_47

I did not notice any difference between enabled and disabled dxvk_async. The ENB GUI could be opened at any time. However, I noticed that with night mode enabled in Gnome, Skyrim ends up in a black screen. Here only a restart helps, because apparently Xorg or something else crashes.

EDIT: I forgot to mention that in v0.452 I am unable to open the ENB GUI.

Oonlyreportingissues 2023-09-11 github

RX 6600 user here & using Proton-GE.

Didn't have to do any winetricks, winecfg & dxvk.conf tweaks.

When starting the game, ENB alters your enbseries.ini file (if its an old one) and adds a new entry called "EnableTerrainBlending=true". Set the value to false and you should be good then.

Also, delete the enbcache folder always whenever a change is made to the ENB files.

I recommend: https://www.nexusmods.com/skyrimspecialedition/mods/78124

It works really well, just gotta turn "EnableTerrainBlending=true" to false

EErikSteiner 2023-09-15 github

Didn't have to do any winetricks, winecfg & dxvk.conf tweaks.

Are you sure, that you used the WrapperVersion?

Oonlyreportingissues 2023-09-18 github

Didn't have to do any winetricks, winecfg & dxvk.conf tweaks.

Are you sure, that you used the WrapperVersion?

Yes, but maybe I didn't need to tweak because I'm using Proton-GE? Not sure. I don't know if it's connected, but no matter which ENB preset I'm using, it looks like Silent Hill (kinda foggy appearance colour-wise). Didn't try it on vanilla Skyrim yet though, modded the game first before I tried ENB.

Edit: Actually "The Vanilla ENB Two" works well with the modded game. Switched to that one now (but had to turn "EnableTerrainBlending=true" to false again).

Ffighuass 2023-11-05 github

Yeah that's probably ENB producing broken DXBC shaders again which we then translate into broken SPIR-V.

Not sure how much I really care at this point to be honest, just disable GPL as a workaround if that helps, you can do this with a DXVK config file as well by setting dxvk.enableGraphicsPipelineLibrary = False.

This fixed the immediate crashing and I can play in indoors environments now, but the error reappears whenever I go outside. What could that mean?

Oonlyreportingissues 2023-11-09 github

This collection has some ENB mods & game fixes that might help, there is also a guide for setting up ENB for Skyrim in the description: https://next.nexusmods.com/skyrimspecialedition/collections/2bhk0y

Ddoitsujin maintainer 2026-06-16 github

No idea if this is still an issue, new shader compiler should be more robust against broken code, either way this original report won't really apply anymore given how much things have changed.

Proton versions

Launch options

Upstream links

DLLs