protonscr

System compiled dxvk causes stack smashing with wine when playing yakuza-0/kiwami.

dxvkclosed build
doitsujin/dxvk#3981 · opened 2024-05-02 by ghost · updated 2025-04-14 · 22 comments · github
2 matching comments, n / p to jump
?ghost 2024-05-02 github

Tried playing yakuza-0 / yakuza-kiwami. Both work with wine on its own; its just when I install dxvk they don't even start and show an error message.

0120:err:seh:NtRaiseException Unhandled exception code c0000409 flags 1 addr 0x7ffffd5255c2

distribution: Gentoo Linux
gpu: RTX 4080 SUPER
gpu driver: proprietary nvidia drivers version 550.78
wine version: wine-vanilla-9.7
dxvk version: 2.3.1-r1
vulkan-loader version: 1.3.280.0

I have tried enabling the environment variables below but they didn't give me anything for some reason.

VK_INSTANCE_LAYERS="VK_LAYER_KHRONOS_validation"
DXVK_LOG_LEVEL="debug"
DXVK_LOG_PATH="/home/user/"
BBlisto91 2024-05-02 github

Post a full Wine log if you can. Maybe ran with WINEDEBUG=+loaddll

Mmbriar 2024-05-02 github

Without a log, best guess would be that you probably mixed up system32 and syswow64 when installing

In a default Wine prefix that would be as follows:

export WINEPREFIX=/path/to/wineprefix
cp x64/.dll $WINEPREFIX/drive_c/windows/system32
cp x32/
.dll $WINEPREFIX/drive_c/windows/syswow64
winecfg

wine version: wine-vanilla-9.7

Also you should probably know that Yakuza 0 needs esync to reliably save: https://github.com/ValveSoftware/Proton/issues/492#issuecomment-889240630

Ddoitsujin maintainer 2024-05-02 github

If there's no log despite setting DXVK_LOG_PATH then chances are our DLLs aren't even getting loaded. Both games are known to work fine with DXVK.

?ghost 2024-05-02 github

In order to setup dxvk in my wineprefix, which I created a brand new one just for yakuza-0 I did the following:

WINEPREFIX="<game_dir>/prefix/"
WINEARCH="win64"
WINEESYNC=1
WINEFSYNC=1
WINE_FULLSCREEN_FSR=1
WINE_FULLSCREEN_FSR_STRENGTH=2
DXVK_FRAME_RATE_LIMIT=145

setup_dxvk.sh install --symlink

Then I just replaced the setup line with wine <game_dir>/<yakuza.exe>

I will try enabling wine debugging hopefully that will help.

?ghost 2024-05-02 github

I just tried WINEDEBUG=+all and WINEDEBUG=+loaddll and I am getting no extra output other than what I said.

Mmbriar 2024-05-02 github

That's impossible, there can't just be that one line with +all

?ghost 2024-05-02 github

Maybe its due to the strip useflag in gentoo I will remove it and see if it provides a log. The thing is with +all I ran it without dxvk and was getting a lot more lines of output than I usually get so I thought it was working.

Mmbriar 2024-05-02 github

I don't think it should be related to the wine build config, i don't think it's possible to disable the logging completely at build time, and it seems like it works without DXVK? But I also don't see how there could possibly be only that one line in the log and nothing else with +loaddll and DXVK loaded, it should at least print a line for loading the dlls, if nothing else.

?ghost 2024-05-02 github

I still couldn't figure out why the debug wasn't showing anything extra. However, I think I have solved part of the issue already. It seems that when I used setup_dxvk.sh (which is shipped with gentoo repos dxvk) the symlinked dxvk dlls were not working as it should. So after I did setup_dxvk.sh install --symlink and then replaced the symlinks with the dlls that you have on github it seems that dxvk is running finally (although the framerate is less than when I used to play with my amd rx 6800 could be just nvidia drivers or something) but its a lot more than stock wine.

I am confused on this as some games i.e fallout 4 is able to use the symlinked dlls however games like yakuza 0 and fallout new vegas can't. What could be the issue? I am not using crossdev-mingw useflag so its using dev-util/mingw64-toolchain rather than crossdev.

?ghost 2024-05-04 github

I was trying this again today and it seems like I was able to obtain a wine log with +loaddll.

wine.log

Thanks for the wait.

?ghost 2024-05-04 github

looks like when using system compiled dxvk on gentoo and running yakuza creates some sort of stack smashing.

Ddoitsujin maintainer 2024-05-05 github

have you tried using our release binaries instead?

I'm seeing exactly zero dxvk messages in that log though so it seems to just fail horribly before reaching any of our code for some reason.

?ghost 2024-05-05 github

have you tried using our release binaries instead?

I'm seeing exactly zero dxvk messages in that log though so it seems to just fail horribly before reaching any of our code for some reason.

I still couldn't figure out why the debug wasn't showing anything extra. However, I think I have solved part of the issue already. It seems that when I used setup_dxvk.sh (which is shipped with gentoo repos dxvk) the symlinked dxvk dlls were not working as it should. So after I did setup_dxvk.sh install --symlink and then replaced the symlinks with the dlls that you have on github it seems that dxvk is running finally (although the framerate is less than when I used to play with my amd rx 6800 could be just nvidia drivers or something) but its a lot more than stock wine.

I am confused on this as some games i.e fallout 4 is able to use the symlinked dlls however games like yakuza 0 and fallout new vegas can't. What could be the issue? I am not using crossdev-mingw useflag so its using dev-util/mingw64-toolchain rather than crossdev.

Yes I meant I used your release binaries and it was working. However, the ones that my system compiled for some reason in these games it wasn't working which is very weird. So the release binaries on github for this version of dxvk worked for yakuza 0 etc however when I try compiling these versions of dxvk on my system and used setup_dxvk.sh it doesn't work and provides that wine.log that I have posted.

?ghost 2024-05-05 github

For comparison this is the wine.log when I use the dxvk binaries provided on github.

wine-log-dxvk-bin.log

If you compare lines 217 onwards for both logs you can see that this is where it differs.

Iionenwks 2024-05-08 github

Gentoo here, while we do default to -fstack-protector-strong system-wide, the default mingw toolchain we provide does not (prefer to avoid most security features given how fragile Wine can be, Edit: the implementation of the features themselves can also be like how gcc+mingw crashed with -fstack-clash-protection for a while). I believe any users affected by this is either passing -fstack-protector-strong or similar themselves in *FLAGS or using a different toolchain that does default-ssp (e.g. with crossdev tooling in gentoo)

Does indicate there may be something wrong with either dxvk or mingw64-runtime, but to have a shot at reproducing would need to build both with it (haven't tried, Edit: and I assume the binaries here do not already).

If really needed, Gentoo could work around this by forcing explicit -fno-stack-protector in the runtime+dxvk ebuilds (ignoring user settings) though.

?ghost 2024-05-08 github

As ionenwks stated its because mingw64-toolchain was compiled with -fstack-protector-strong. However, while this does indicate that something is "wrong" disabling this and recompiling dxvk does make it work.

Ddoitsujin maintainer 2024-05-09 github

Not sure how to provide any help here, things like asan unfortunately don't work with mingw and we obviously can't use native builds with Windows games.

FWIW, building dxvk itself with -fstack-protector-strong works fine here with Kiwami at least. Don't have 0 installed right now.

Iionenwks 2024-05-10 github

Not sure how to provide any help here, things like asan unfortunately don't work with mingw and we obviously can't use native builds with Windows games.

FWIW, building dxvk itself with -fstack-protector-strong works fine here with Kiwami at least. Don't have 0 installed right now.

Yeah that gave me no issues either, I needed to build mingw's runtime itself with it then build dxvk against these to start having issues. fwiw doing that doesn't seem to break wine itself built using the same mingw.

Unfortunate but if debugging is going to be hard, guess passing -fno-stack-protector is still what we can go with downstream anyhow.

Iionenwks 2024-05-10 github

Actually, I just remembered that support for this is kind of new in mingw64 11 (it used to be straight up broken with <=10). This gives decent chances of the issue being in mingw itself rather than dxvk.

Think it can be considered experimental and skipping using it seems fair in that context.

BBlisto91 2024-11-14 github

@notmentaloutlaw Are you still having issues with this?

?ghost 2024-11-14 github

The issue was having a specific compiler flag for mingw64-toolchain before compiling dxvk. However, after disabling that flag solved the issue. I am not aware if the issue got fixed upstream as I have not yet tested it but for now I can get dxvk to work.

Vv-fox 2025-04-14 github

Damn it! What an insidious issue!
I've wasted whole bunch of time and effort while thinking that my "unsafe" Gentoo configuration (clang/libc++ with -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -O3 -march=x86-64-v2 -mtune=native -mavx -mf16c -mcrc32 -maes -mpclmul -flto=thin -Wl,--plugin-opt=O3 -fcoroutines -fpermissive -mllvm -polly -mllvm -polly-vectorizer=stripmine -mllvm -polly-parallel -mllvm -polly-omp-backend=LLVM -lgomp -fwhole-program-vtables -Wl,--gc-sections -Wl,--icf=safe -rtlib=compiler-rt -unwindlib=libunwind -Wl,-lunwind and some other stuff) was at fault.
Yet it was the exact opposite: bloated mess of a dev-util/mingw64-toolchain is the culprit - the one place Gentoo should have and failed to put filtering for -fstack-protector-strong -fstack-clash-protection.

With filtering those it works just fine. In fact, I've rebuilt wine & dxvk in the exact opposite manner: removed all filtering (LTO, AVX, etc.) and hardcoding for wine & dxvk, patched wine with NTSYNC and staging, rebuilt wine/wow64 with clang. I've also tried adding -mstackrealign -mpreferred-stack-boundary=5 -mprefer-vector-width=256 -mavx256-split-unaligned-load -mavx256-split-unaligned-store to dxvk to workaround avx issue but not sure if it actually applied to mingw.
Cyberpunk 2077 still loaded just fine.

@doitsujin
dxvk and vkd3d-proton now are the only users of bloated mess of mingw64-toolchain and also its victims. Could it be possible to drop this dependency and use clang, just like wine does?