eBPF bytecode onto GPUs via dynamic PTX injection
Stopped reading, this does not apply to us.
Stopped reading, this does not apply to us.
Because PTX is Nvidia-specific?
Android uses eBPF for GPU memory profiling: https://source.android.com/docs/core/architecture/kernel/bpf#androidebpf
Because PTX is Nvidia-specific?
Yes.
Just pointing out that eBPF exists isn't really actionable though. Not particularly motivated to spend a long time reading about it and researching what it is, and how it could potentially work since the likelyhood it working out is very low.
Not particularly motivated to spend a long time reading about it and researching what it is, and how it could potentially work since the likelyhood it working out is very low.
Maybe instead of having to read the whole kernel documentation about eBPF, this blog post gives some usage examples of eBPF (and other tracing tools):
That article seems to be talking about CPU profiling?
That article seems to be talking about CPU profiling?
Yes, but eBPF is very flexible. To quote the same article:
eBPF
If you want to trace things using the Linux kernel there’s a new game in town, and it’s awesome. The eBPF subsystem allows you to attach complex programs to all sorts of different things in the kernel and efficiently shuttle data back to userspace, basically subsuming all the legacy facilities like ftrace and kprobes such that I won’t talk about them.
Things you can trace include: syscalls, low overhead tracepoints throughout the kernel, hardware performance counters, any kernel function call and arbitrary breakpoints or function calls/returns in userspace code. Combined these basically let you see anything on the system in or out of userspace.
You normally write BPF programs in C but there are perhaps even nicer toolkits for using Zig and Rust.
Using eBPF for GPUs seems to be a relatively new thing.
Most references you will find (apart from the Android example), are people wanting to use it to profile their AI/ML GPU fleets (mainly focusing on CUDA for obvious reasons, but I guess they will want to make it cross-vendor eventlually), see e.g.:
Although, I found one presentation with another use case:
"Observing GPU Runtime Behavior Inside a Self-Driving Car – Andrew Liang, Cruise"
https://www.youtube.com/watch?v=7bdy2AkRjqE
I don't know if any Linux/DRM or Mesa developers have looked into eBPF yet. Maybe it is too "CPU-centric" to quote the ACM article:
Furthermore, eBPF’s inherent CPU-centric design does not directly extend to GPU hardware without novel translation and runtime adaptation strategies.
Anyway, all of this is way too much to look into, and I'm not going to. I cannot be expected to jump on something like this unless someone makes an actual proof of concept demonstrating some novel profiling running on the GPU with Vulkan that anyone can easily build and run and easily see the benefit of. I don't expect anyone to do that of course, but that's the minimum bar to clear to get a serious look at this.
This was just a brain dumb. I thought vkd3d needs every tool available to close the performance gap.
Valve already has contractors working on the kernel/Mesa and AFAIK they fund RenderDoc development. Maybe they can pay someone to investigate how useful eBPF could be for GPU/Vulkan profiling, maybe it could help improve the kernel's VRAM management.
OK, so after reading a bit more about the topic, I came to the conclusion that eBPF will not help you much with profiling.
eBPF is basically a more convenient way to write your own kernel modules. It allows for faster development by making forking the kernel code unnecessary and still being able to maintain your own functionality (bypassing the kernel's code review process).
eBPF seems be used the write custom tools to monitor & control long running processes. It could be used to collect all kinds of system data during a gaming session. If that is useful, I don't know.
Since both UMR and the Radeon GPU Profiler directly talk to the AMDGPU kernel driver, there should be no advantage in writing your own eBPF-based tools.
Interestingly, UMR not only supports debugging, but also profiling:
https://gitlab.freedesktop.org/tomstdenis/umr/-/blob/main/doc/sphinx/source/profiler.rst
And the Radeon GPU Profiler has been integrated with RADV for some time:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3900
So feel free to close this issue, sorry for the noise.
Nothing extracted yet.
https://dl.acm.org/doi/10.1145/3723851.3726984
Maybe the devs are already aware of it, but I was wondering if eBPF would be of any use for profiling.
Edit: docs: https://docs.kernel.org/bpf/index.html