`dxil-spirv: Invalid type ID in cast<T>` with conditional `RAY_FLAG` argument for `TraceRayInline()`

vkd3dclosed

HansKristian-Work/vkd3d-proton#2809 · opened 2026-02-06 by MarijnS95 · updated 2026-02-10 · 5 comments · github

MMarijnS95 2026-02-06 github

Hey, not sure if this is the right place (as I'm running the full vkd3d-proton package) or if this should be transferred to https://github.com/HansKristian-Work/dxil-spirv.

We're hitting dxil-spirv: Invalid type ID in cast<T>. in one of our HLSL shaders with a ray query, when testing latest master @ 298eaf00036f8f1cb4410c9e21b5208ed9a66c67, and bisected it down to the following code changes. The only other issue mentioning this here is #602.

Using "the latest" v1.8.2505.1 DXC with a simple -T cs_6_6 -E main, creating a conditional RAY_FLAG variable for RayQuery::TraceRayInline() based on two short-circuiting || or && results in dxil-spirv: Invalid type ID in cast<T>.. Replacing one of those ops with a binary variant (| or &), or making at least one of the expression sources a proper const (where the RHS is actually a compile time constant) for example solves this issue.

The full shader source and comparison is available on GodBolt, with this being the most important code snippet:

    uint bounce = g_bindingsOffset.userData0;
    bool alphaOnPrimary = (bnd.randomConstant & (1 << 1));
    bool shouldAlphaTest = (bnd.randomConstant & (1 << 0));
    // Broken
    // bool alphaTest = (alphaOnPrimary && (bounce == 0)) || shouldAlphaTest;
    // Works
    bool alphaTest = (alphaOnPrimary && (bounce == 0)) | shouldAlphaTest;
    // Also works
    // bool alphaTest = (alphaOnPrimary & (bounce == 0)) || shouldAlphaTest;

    RAY_FLAG flags = 0;
    if (!alphaTest)
        flags |= RAY_FLAG_FORCE_OPAQUE;

    q.TraceRayInline(tlas, flags, 0xff, ray);

Additionally, as mentioned, here is an example changing between uint bounce = 0; and const uint bounce = 0; to also trigger this issue.

If convenient, let me know and I can provide the compiled DXIL for these sources; though I assume being able to live tweak and recompile it is also useful?

HHansKristian-Work maintainer 2026-02-07 github

I cannot reproduce this based on that data. I tried compiling the shaders on that godbolt link, but they compile just fine. Please just provide a reproducing .dxil file so I don't have to fumble in the dark.

MMarijnS95 2026-02-09 github

@HansKristian-Work you're entirely right, I slightly reformatted the shader and by that, it no longer reproduces, my apologies.

Specifically I moved RayQuery<RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH> q; down to where it is needed, and that affects the reproducability. This apparently does not hit the error:

RAY_FLAG flags = 0;
if (!alphaTest)
    flags |= RAY_FLAG_FORCE_OPAQUE;
RayQuery<RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH> q;
RaytracingAccelerationStructure tlas = ResourceDescriptorHeap[bnd.tlasSrv];
q.TraceRayInline(tlas, flags, 0xff, ray);

But if it's anywhere above the conditional, it seems to error out:

// RayQuery<RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH> q;
RAY_FLAG flags = 0;
RayQuery<RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH> q;
if (!alphaTest)
    flags |= RAY_FLAG_FORCE_OPAQUE;
RaytracingAccelerationStructure tlas = ResourceDescriptorHeap[bnd.tlasSrv];
q.TraceRayInline(tlas, flags, 0xff, ray);

Here's new shader source.

GitHub doesn't like dxil files, even when renaming them to .txt or attaching as .tgz, give a me a bit to upload it elsewhere.

HHansKristian-Work maintainer 2026-02-10 github

I reproduced it now. Completely bogus DXIL where DXC somehow invents a phi where it makes zero sense to do so ...

MMarijnS95 2026-02-10 github

It seems I forgot to mention that we were also seeing the exact same shader fail to load on Dx12 (native Windows installation) on an Intel GPU, and that was fixed with the exact same workaround. I was almost already assuming something to be "wrong" (or very unexpected) with this DXIL produced by DXC rather than both implementations being broken.

HHansKristian-Work maintainer 2026-02-10 github

It is legal DXIL (sadly), so it's a bug that will be fixed, but patterns like this will most likely run like absolute ass in practice.

Nothing extracted yet.