Thanks for the reply
I don't know how to do this, but I recorded a video and activated all the Dxvk HUD and tested both versions. I hope it helps.

https://mega.nz/file/w5Zi3J7B#bY7N-K_MLamMJSbkkndjY80jIyOhqpLR1eLyzy7ljpc

I don't know how to do this

You can read bisect document here: https://docs.kernel.org/admin-guide/bug-bisect.html

Or in short, it can be described as dividing the changes into two groups each time and test the middle change.
For example, commit 300 is good, commit 400 is bad. So you should test commit 350. If commit 350 is good, now sth wrong is in 351-400, or if 350 is bad then sth wrong in 300-350. And you divide the new range again and again, finally you can find which commit is faulty.

I read the document and I can't do it. I have no knowledge of Linux, and I don't think you support Turnip or Android. If there is nothing you can do to help me, you can close this issue.

Thanks for developing Dxvk, I will wait for the next updates, maybe they will solve the issue.

I have no knowledge of Linux

Actually, you don't need to be familiar with Linux to do this bisect test.
You can grab build for each commit from actions here: https://github.com/doitsujin/dxvk/actions
Grep by master branch, and v2.4.1 is 4469 (page 6) and v2.5.1 is 5197 (page 1).
So you may select some commit from page 3 or 4.
Since you are facing issue on a DX9 game I suppose, you may select those commits have changes on d3d9 in preference. (Usually, those changes apply to DX11 won't have impact on DX9 games)
e.g. Why not try this commit (https://github.com/doitsujin/dxvk/actions/runs/11540148282 ) on page 3?

I found this out before already, but I'm pretty sure this is not a DXVK issue, or a Turnip one. It's probably related to termux-x11 rendering on android. Or it could be the intended behavior, because before these commits with GPU 100% the whole android ui was struggling to render, and so it's better to have a few resources not used. It's caused by 3 commits in this pull request https://github.com/doitsujin/dxvk/pull/4380

Thanks @qinlili23333 for explaining

It start with this commit
https://github.com/doitsujin/dxvk/actions/runs/11410128609

The one before it was good.

I found this out before already, but I'm pretty sure this is not a DXVK issue, or a Turnip one. It's probably related to termux-x11 rendering on android. Or it could be the intended behavior, because before these commits with GPU 100% the whole android ui was struggling to render, and so it's better to have a few resources not used. It's caused by 3 commits in this pull request #4380

https://github.com/doitsujin/dxvk/issues/4489#issuecomment-2498979802

yup you right xD deym i tested it on my self .. took so many build

On KGSL (downstream qcom's kernel driver on Android) in Turnip we have timeline semaphores emulated because it was easier to do. Though KGSL itself supports timeline semaphores.

I'm not sure how much it can affect performance.

On KGSL (downstream qcom's kernel driver on Android) in Turnip we have timeline semaphores emulated because it was easier to do. Though KGSL itself supports timeline semaphores.

I'm not sure how much it can affect performance.

It varies from game to game, in some the FPS drops to half compared to the non-Semaphore DXVK version.

Is there a possibility that Turnip will fix this in future updates?

@werman sorry for the ping, but this thread caught my attention trying to look into this same regression.

Can I confirm my understanding of how Turnip-on-kgsl "emulates" timelines?

It looks like it relies on the timestamp of the request that signals the last (most recent) kgsl_syncobj submitted in kgsl_queue_submit, in particular, it does not track the actual timeline value - am I reading this right?

So for example, let's say we submit the following command buffers (all on the same gfx_queue) in this order:

1. cmd1, sem1 (signals to 10)
2. cmd2, sem1 (signals to 20)
3. cmd3, sem1 (signals to 30)
4. cmd4, wait for sem1 at 20

The states of the kgsl_syncobj would proceed like this:

// Init
{
   .kgsl_syncobj_state state = KGSL_SYNCOBJ_STATE_UNSIGNALED,
};

// After cmd1 executes
{
   .kgsl_syncobj_state state = KGSL_SYNCOBJ_STATE_TS,
   .queue = gfx_queue,
   .timestamp = timestamp_cmd1_on_gfx_queue,
};

// After cmd2 executes
{
   .kgsl_syncobj_state state = KGSL_SYNCOBJ_STATE_TS,
   .queue = gfx_queue,
   .timestamp = timestamp_cmd2_on_gfx_queue,
};

// After cmd3 executes
{
   .kgsl_syncobj_state state = KGSL_SYNCOBJ_STATE_TS,
   .queue = gfx_queue,
   .timestamp = timestamp_cmd3_on_gfx_queue,
};

// After wait for cmd4
{
   .kgsl_syncobj_state state = KGSL_SYNCOBJ_STATE_UNSIGNALED,
};

And that wait for sem1 at 20 will also not look at the timeline target value of 20, and instead wait for the last (most recently) submitted request timestamp to be completed, AKA timestamp_cmd3_on_gfx_queue (corresponding to the timeline value of 30). It will block waiting for cmd3 to complete even if its target value (cmd2, 20) has already completed.

In other words, it seems like the semantic of the emulated timeline semaphores will always wait for the max timeline value "submitted" (AKA every request submitted on this queue at the time of that wait), even if the target value is an intermediate point on the timeline?

If this is the case, it seems like a performance penalty is applied mainly when a timeline is setup where waits may occur for an intermediate timeline value instead of the most recent timeline value, because it will instead wait for the whole batch to complete first.

I think this then characterizes what happens on Turnip-kgsl after that patch. Specifically with the DxvkQueue (I'm not sure if this mental model of the queue is sound or not @doitsujin)

Before:

submit(cmd1) -> fence1
submit(cmd2) -> fence2
submit(cmd3) -> fence3 
// cmd1 completes
finishThread(cmd1) -> wait(fence1) // after cmd1 completes
// cmd2 completes
finishThread(cmd2) -> wait(fence2) // after cmd2 completes
submit(cmd4) -> fence4

After:

submit(cmd1) -> timeline 1
submit(cmd2) -> timeline 2
submit(cmd3) -> timeline 3
finishThread(cmd1) -> wait(1) // need to wait for cmd3 to complete, much longer wait
submit(cmd4) -> timeline 4
submit(cmd5) -> timeline 5
// cmd3 completes
finishThread(cmd2) -> wait(2) // need to wait for cmd5 to complete !!
submit(cmd6)
submit(cmd7)
// cmd5 completes
finishThread(cmd3) -> wait(3) //  need to wait for cmd7 to complete !!
...

This causes two major issues:

1. Slow resource cleanup - finishThread now gets blocked frequently and for much longer, so the resources associated with the previously completed commands are no longer cleaned up on a timely manner. Especially around level loads, this could create massive spikes of memory usage from continuous texture loads that fail to be cleaned up properly
2. (More problematic) Long term queue capacity issues. The biggest problem with blocking finishThread to wait for all previously submitted command to complete before it can clean up 1 command is that the queue will keep growing. Over time, the queue (which is bounded I believe) will become filled.

This then devolves into a massive pipeline serialization, because when the queue is filled and finishThread can only clean up one cmd at a time, then you get into a world where:

1. You try to submit a command, it must wait for a free slot
2. finishThread finishes (indicating every cmd in the queue has already finished execution on the device), and frees up one slot
3. The command is submitted
4. The next command must wait for every task in the queue to finish, before it can be submitted.

We end up with a very shallow queue of commands that must be executed 1-by-1 (effectively devolving into a queue of size 1). Even more problematic is that the presentation call itself must now be serialized with the rendering, since it also goes onto the same queue. I'm going to venture that this might be what is happening and the cause of the performance regression.

Oh and the FPS pacing and counters will also be completely off even before the queue is completely filled up, because the frame counter now tracks the time of a batch of commands to complete instead of individual commands (similar for time-to-present) since the tracking occurs in the finish thread.

@doitsujin Can you look at this issue again please?