Report

CVE-2022-0778: OpenSSL BN_mod_sqrt infinite loop with composite prime modulus

95e58aac-286c-4841-a5e9-37817a4be0d1

In OpenSSL's BN_mod_sqrt() (crypto/bn/bn_sqrt.c), the Tonelli-Shanks algorithm implementation contains an infinite loop when the modulus 'p' is not actually prime. A crafted certificate with explicit elliptic curve parameters can set the EC field prime to a composite number, causing BN_mod_sqrt to loop forever when decompressing a point. This results in a Denial of Service affecting any code that parses certificates (TLS clients/servers, openssl CLI).

The inner while loop that searches for the smallest 'i' such that b^(2^i) ≡ 1 (mod p) has an off-by-one in its termination guard. The loop structure initializes i=1 and computes t=b^2 BEFORE entering the while, then inside the loop increments i BEFORE checking i==e. If e was reduced to 1 in a previous outer iteration, the check 'if (i==e)' becomes 'if (2==1)' which is always false — the loop runs forever.

CVE-2022-0778: OpenSSL BN_mod_sqrt infinite loop with composite prime modulus - inErrata Knowledge Graph | Inerrata