This is a Request for Comment (RFC) and Proof of Concept (PoC) addressing the dwmapi syscall bridge and driver-level logic required to enable transparent Windows overlays in Proton.
The implementation specifically targets dwmapi stubs (e.g., DwmExtendFrameIntoClientArea) and driver behaviors that currently cause external overlays to render with opaque backgrounds, steal focus, or fail to pass mouse input (click-through) to the underlying game.
Technical Context
This work uses basic internal test overlays and Guild Wars 2 overlays (GW2TacO and Blish HUD) as development baselines, as they represent two distinct architectural approaches to transparency (Native C++ vs. Managed .NET).
Detailed research, architectural comparisons, and side-by-side visual results are available in the Project README.
Forced 32-bit Visuals: Detects .NET/WinForms signatures (WS_EX_CONTROLPARENT) during create_whole_window to force ARGB visuals immediately, bypassing X11's inability to upgrade window depth post-creation.
Input Ghosting: Implements sync_ghost_shape via XShapeCombineRegion to synchronize WS_EX_TRANSPARENT styles with the X server.
Focus Loop Prevention: Kernel-side interception in set_focus to deny focus-stealing requests from passive overlays while the game client is the active foreground window.
2. Wayland (Alpha / WIP)
DWM-to-Wayland Protocol Mapping: Implements WAYLAND_SetWindowDwmConfig to intercept DWM_CONFIG_OPAQUE_REGION, translating them into Wayland wl_regions via wayland_surface_update_input_region.
ARGB Protocol Support: Forces WL_SHM_FORMAT_ARGB8888 when WS_EX_LAYERED is present and updates the buffer queue to track pixel formats correctly.
Alpha Channel Finalization: Added a post-process pass in the surface flush loop to map pure black (0x000000) to Alpha 0 and non-black to Alpha 255 to resolve GDI XRGB transparency issues.
Current Status & Known Issues
This project is a Work in Progress (WIP) provided as a technical reference point for Valve maintainers.
Testing: Functional in standalone internal test overlays. X11 is stable enough for use with certain real-world overlays; Wayland is in active development and currently displays minor rendering artifacts during alpha transformation.
In-Game Status: Integration testing within full game environments (specifically Guild Wars 2) is ongoing but not yet fully stabilized for the Wayland driver.
Intent & Handoff
This implementation is a research-driven proof of concept developed as a side project. While I am not a professional Wine developer and cannot commit to long-term maintenance due to time constraints, I am providing this patchset as a functional baseline to addressing Wine's long-standing transparency and DWM-stub issues. I hope this serves as a useful technical reference point should Valve find interest in refining, stabilizing, or eventually integrating these patches into Proton. I am happy to mirror these branches to a GitHub repository if it would better facilitate a formal review or Pull Request.
Overview
This is a Request for Comment (RFC) and Proof of Concept (PoC) addressing the
dwmapisyscall bridge and driver-level logic required to enable transparent Windows overlays in Proton.The implementation specifically targets
dwmapistubs (e.g.,DwmExtendFrameIntoClientArea) and driver behaviors that currently cause external overlays to render with opaque backgrounds, steal focus, or fail to pass mouse input (click-through) to the underlying game.Technical Context
This work uses basic internal test overlays and Guild Wars 2 overlays (GW2TacO and Blish HUD) as development baselines, as they represent two distinct architectural approaches to transparency (Native C++ vs. Managed .NET).
Detailed research, architectural comparisons, and side-by-side visual results are available in the Project README.
experimental-10.0-20251125(Proton 10)Repository Links
Key Implementation Details
1. X11 / XWayland (Beta)
.NET/WinFormssignatures (WS_EX_CONTROLPARENT) duringcreate_whole_windowto force ARGB visuals immediately, bypassing X11's inability to upgrade window depth post-creation.sync_ghost_shapeviaXShapeCombineRegionto synchronizeWS_EX_TRANSPARENTstyles with the X server.set_focusto deny focus-stealing requests from passive overlays while the game client is the active foreground window.2. Wayland (Alpha / WIP)
WAYLAND_SetWindowDwmConfigto interceptDWM_CONFIG_OPAQUE_REGION, translating them into Waylandwl_regions viawayland_surface_update_input_region.WL_SHM_FORMAT_ARGB8888whenWS_EX_LAYEREDis present and updates the buffer queue to track pixel formats correctly.0x000000) to Alpha 0 and non-black to Alpha 255 to resolve GDI XRGB transparency issues.Current Status & Known Issues
This project is a Work in Progress (WIP) provided as a technical reference point for Valve maintainers.
Intent & Handoff
This implementation is a research-driven proof of concept developed as a side project. While I am not a professional Wine developer and cannot commit to long-term maintenance due to time constraints, I am providing this patchset as a functional baseline to addressing Wine's long-standing transparency and DWM-stub issues. I hope this serves as a useful technical reference point should Valve find interest in refining, stabilizing, or eventually integrating these patches into Proton. I am happy to mirror these branches to a GitHub repository if it would better facilitate a formal review or Pull Request.