Race in py_digest_by_name, _hashopenssl.c under free-threading

[vfdev-5]

Bug report

Bug description:

I built cpython (3.13 branch) with free-threading and TSAN. The following python code from time to time reports TSAN warnings:

import hashlib
import concurrent.futures
import threading


def _hash_string(hash_obj, str_var):
  hash_obj.update(str_var.encode("utf-8").strip())


jaxlib_version_str = "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdef"


compression_algorithm = "zstandard"


def test_hashlib_sha256():
    entries = [
        (
            "jax_lib version",
            lambda hash_obj: hash_obj.update(
                bytes(jaxlib_version_str.encode("utf-8"))
            ),
        ),
        (
            "compression",
            lambda hash_obj: _hash_string(hash_obj, compression_algorithm),
        ),
        ("custom_hook", lambda hash_obj: _hash_string(hash_obj, "")),
    ]
    hash_obj = hashlib.sha256()
    for _ in range(20):
        for name, hashfn in entries:
            hashfn(hash_obj)

    for _ in range(20):
        hash_obj.digest().hex()


if __name__ == "__main__":
    num_workers = 40
    num_runs = 100

    barrier = threading.Barrier(num_workers)

    def closure():
        barrier.wait()
        for _ in range(num_runs):
            test_hashlib_sha256()

    with concurrent.futures.ThreadPoolExecutor(max_workers=num_workers) as executor:
        futures = []
        for i in range(num_workers):
            futures.append(executor.submit(closure))
        assert len(list(f.result() for f in futures)) == num_workers

TSAN report extract:

==================
WARNING: ThreadSanitizer: data race (pid=46290)
  Write of size 8 at 0x7fffb40402c0 by thread T5:
    #0 py_digest_by_name /project/cpython/./Modules/_hashopenssl.c:384:28 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7ce6) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
    #1 py_evp_fromname /project/cpython/./Modules/_hashopenssl.c:914:14 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7872) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
    #2 _hashlib_openssl_sha256_impl /project/cpython/./Modules/_hashopenssl.c:1083:12 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
    #3 _hashlib_openssl_sha256 /project/cpython/./Modules/clinic/_hashopenssl.c.h:591:20 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75)
    #4 cfunction_vectorcall_FASTCALL_KEYWORDS /project/cpython/Objects/methodobject.c:441:24 (python3.13t+0x289f20) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)

Full TSAN report

cc @colesbury

CPython versions tested on:

3.13

Operating systems tested on:

Linux