Isn't sha1 are part of openssl (or so) ? Maybe worth to use system one, or maybe it not affect performance in dxvk at all.
Isn't generic release must not contain advanced instructions?
Isn't -march=native will produce more optimized code for your local builds?
@pchome It is possible to have multiple routines, do CPU detection at initialization and pick the routine best suited to the processor.
Isn't
sha1are part ofopenssl(or so) ?
Yes.
Maybe worth to use system one, or maybe it not affect performance in dxvk at all.
This is being built as a Windows DLL. You cannot use the system OpenSSL because that does not exist on Windows.
As for affecting DXVK performance, I really ought to profile (and I will when I have a working winelib. build), but given that SHA1 is expensive, I think that it will be a non-negligible amount of time. SHA1 is being used by the state cache.
Isn't generic release must not contain advanced instructions?
Well, SSE and SSE2 are part of x86_64, so they are always safe to use there. In all other cases, it is possible to have multiple routines, do CPU detection at initialization and pick the routine best suited to the processor.
Isn't
-march=nativewill produce more optimized code for your local builds?
I am not building with -march=native in this case, but I doubt it would make a difference here because the compiler is refusing to use SSE/SSE2 intelligently on x86_64 where it is permitted to use it with or without -march=native.
I found some resources that might be useful:
https://stackoverflow.com/questions/21107350/how-can-i-access-sha-intrinsic
https://github.com/noloader/SHA-Intrinsics
https://www.nayuki.io/page/fast-sha1-hash-implementation-in-x86-assembly
The last one shows a “fast” implementation that does not look better than what GCC generates. The middle one shows performance when using the SHA instructions Intel added that AND put into Ryzen. It is about 10x what we get from raw x86 assembly if we assume that CPUs performance on the SHA assembly routines did not change much over time. The first suggests that we can reduce CPU time spent by at least a few times.
The last one shows a “fast” implementation
We can simply use -O1 for C (-fomit-frame-pointer enabled at -O1), this optimization will cost nothing.
And yes, those numbers need to be retested using recent GCC version.
The code that GCC currently generates from C code looks similiar to the “fast” implementation. We would need numbers to be certain, but my feeling is that trying to improve the SHA1 code is splitting hairs without using ISA extensions.
An AVX2 implementation:
https://software.intel.com/en-us/articles/improving-the-performance-of-the-secure-hash-algorithm-1
Going based on the public information, I suspect that the only implementations worthwhile adding would be the AVX2 version, the SHA extension version and maybe a fast generic version if it is found to be noticeably better than the C code. In particular, this remark does not leave me much belief that any other variations are worth mentioning:
The overall performance improvement of this implementation over the best known scalar implementations ranges from ~1.2X to ~1.5X, achieving as low as 5.8 cycles per byte on 1024-byte buffer being hashed on the latest generations of Intel processors.
I want a reduction that is larger than that. The C version might not be the best known scalar version (which Intel did not name), but the best known scalar version of that time is something that presumably should work everywhere, provided that we can find it. Depending on how the generic C code compares to it, the improvement from anything but the hardware SHA1 instructions might not be quite as good as I had imagined when I posted this.
Also, someone has always posted the suggestion of using ISA extensions to accelerate routines like SHA1 in every project in which I have ever participated, so I might as well have taken the initiative to preempt that with an issue for this.
Seriously, who cares? The SHA-1 code is not performance-critical and there is absolutely no reason to replace it. The implementation I'm using is in the public domain, and is by the way the same one that is used in Mesa.
d3d11.dllx1 2018-09
Running
x86_64-w64-mingw32-objdump -D ./src/d3d11/d3d11.dllafter building d3d11.dll shows that the generated assembly code for the SHA1 routines is fairly terrible. There only 2 SSE/SSE2 instructions in SHA1Init and that is it.It would be a good idea for someone to find a SSE2 routine that could be used here at the very least. It would also be useful to add support for Intel's SHA extensions when they are available:
https://en.wikipedia.org/wiki/Intel_SHA_extensions
Optimized routines for AVX and AVX2 would also be nice. It might be worthwhile to explore whether the graphite optimizations can improve things here as part of #646. @pchome is already building with graphite, so he should be able to just disassemble his binaries and look through them, provided that he has debuginfo (as I am not sure if he will be able to spot the routines without it).