As SHA-1 is basically deprecated and not recommended to be used anymore, I wouldn't spend too much into this. Besides this, for SHA-1 and SHA-256, it might be even more useful to use the SHA CPU extensions if available. While they are only introduced in Ice Lake and Zen, they will get more and more available in the future.

Hi everyone,

So this past week I've been building on Rika's work by adding an assembly version of SHA-1 for x86_64 to complement Rika's i386 version.  So far I've successfully made a version that runs twice as fast as the Pascal code.  I hoped to go even faster by making use of the SSE2 instruction set, but currently the end result is slower even though computing the common parts of 4 rounds simultaneously should be much faster.  This occurs even when I forgo writing to the stack and keep pretty much all of the state within registers.  Preliminary investigation suggests that the slowdown comes from using MOVD/Q to transfer data between the XMM registers and general-purpose registers, since they are different parts of the CPU.  I'm still amazed it causes this much latency though.

I'll keep investigating and seeing if I can squeeze out more performance, but otherwise I may just have to fall back on a non-SIMD-optimised implementation.

Kit

In 2010 Intel published SSE3 code to improve SHA1 performance. Later that year it was incorporated into OpenSSL ASM code. The OpenSSL code also includes AVX and SHA acceleration extensions.

Intel Article:
https://www.intel.com/content/www/us/en/developer/articles/technical/improving-the-performance-of-the-secure-hash-algorithm-1.html

Brief on Intel SHA extensions (also works for AMD Zen and later CPUs) https://en.wikipedia.org/wiki/Intel_SHA_extensions

OpenSSL x86 64-bit assembly code and performance chart https://github.com/openssl/openssl/blob/master/crypto/sha/asm/sha1-x86_64.pl

###################################################################### # Current performance is summarized in following table. Numbers are # CPU clock cycles spent to process single byte (less is better). #
# x86_64 SSSE3 AVX[2] # P4 9.05 -
# Opteron 6.26 -
# Core2 6.55 6.05/+8% -
# Westmere 6.73 5.30/+27% -
# Sandy Bridge 7.70 6.10/+26% 4.99/+54% # Ivy Bridge 6.06 4.67/+30% 4.60/+32% # Haswell 5.45 4.15/+31% 3.57/+53% # Skylake 5.18 4.06/+28% 3.54/+46% # Bulldozer 9.11 5.95/+53%
# Ryzen 4.75 3.80/+24% 1.93/+150%(**) # VIA Nano 9.32 7.15/+30%
# Atom 10.3 9.17/+12%
# Silvermont 13.1(*) 9.37/+40%
# Knights L 13.2(*) 9.68/+36% 8.30/+59% # Goldmont 8.13 6.42/+27% 1.70/+380%(**) #
# (*) obviously suboptimal result, nothing was done about it, # because SSSE3 code is compiled unconditionally;
# (**) SHAEXT result

Thanks for the resources - these will prove very useful!  Intel and AMD processors also have specialised SHA instructions later on.  I know the AMD Zen supports them - not sure the earliest Intel models though.

Currently I'm sticking with pure SSE2 since this is the latest instruction set that is guaranteed to be available on all x86_64 processors.  I can write versions for SSSE3 and AVX later, but currently I'm trying to identify the mysterious performance drops.

Kit

OpenSSL x86 64-bit assembly code

https://github.com/openssl/openssl/blob/master/crypto/sha/asm/sha1-x86_64.pl

To generate plain assembly from the OpenSSL perl asm script requires downloading these files to the same folder:
https://github.com/openssl/openssl/tree/master/crypto/perlasm

I missed this post - thanks Florian!

Indeed, SHA-1 is deprecated at least as far as being a cryptographic algorithm is concerned, but it still has some uses in data verification in a similar vein to MD5.  I know git uses it internally so server branches can't be corrupted.

I have probably spent too much time on SHA-1 already - its awkward size of 160 bits has always irked me... not a clean power of two!

Speaking of the Intel SHA instructions, can I introduce a merge request that adds "CPUX86_HAS_SHA" as a feature flag?  I know to add it for "cpu_zen" and later, but I'm not sure what the equivalent Intel processor is... is "cpu_core_avx2" okay or does there need to be a new one?

Kit

assembly version of SHA-1 for x86_64

Another example which I thought is beautiful in form and simplicity (I don't know how it compares for performance):
https://github.com/nayuki/Nayuki-web-published-code/blob/master/fast-sha1-hash-implementation-in-x86-assembly/sha1-fast-x8664.S

You can, but in this case it is imo more useful to check at run time by using the cpu unit as the instructions are part of a procedure not generated by the compiler.

True, that's fair.  Just having this feature for the sake of compiling the "hash" package more efficiently is... well... favouritism?!  Or just a feature that has a single purpose.

Kit