Cannot set AD for ChaCha20Poly1305 while processing a message

[larssilven]

Could you please provide some information on this exception. I have got it at several occasions in TLS connections but not frequently. A test could be working without failures most of the times but then the exception just appear without anything changed.
In the latest occasion I was running a test that is transferring data between client and server in both directions (just to test the robustness of my protocol). 768 GiB was transferred by 12 threads on each side. I got the exception once on the server side and after this the failing server thread and the corresponding client thread was terminated. But the other threads continued without any failures. Some info about the test:

• Exception was trowed after calling Channel::send(const uint8_t buf[], size_t buf_size)
• Botan 3.6.1 with my path for Use of none botan private keys for signing certificates. #3867
• Botan TLS used on both server and client.
• No change from the default Botan algorithms.
• Server is using a PKCS#11 private key from https://github.com/softhsm/SoftHSMv2 for authentication.
• Client is using a PKCS#11 private key from https://github.com/OpenSC/OpenSC for authentication.

Maybe you could recommend some parameters for a new build to get more info about the problem?
Do you think it could be any problem with the OpenSC private key?

BR Lars

[randombit]

I don't think OpenSC/SoftHSM being in the stack would make much change (absent a memory corruption or something like that completely making a mess, which is always possible - but nothing to indicate that so far)

I've never seen anything like this and looking at the relevant calls to set_associated_data within the TLS stack it's hard to understand how it would occur.

Is this with TLS 1.2 or 1.3?

768 GiB was transferred by 12 threads on each side.

Is more than one thread accessing a TLS::Server instance at a time? If so you would need to apply locking. TBH some kind of concurrency issue seems like the most likely cause for what you are seeing.

Things to try

At the point this exception is thrown (line 58 of src/lib/modes/aead/chacha20poly1305/chacha20poly1305.cpp) add a call to abort and try to get a backtrace. Also when it happens what are the other threads doing?

Run under helgrind (https://valgrind.org/docs/manual/hg-manual.html) and see if anything errors [it's really slow though]

[randombit]

@larssilven Any update on this? If there is a bug here we absolutely want to find and fix it. But without any reproducing (or even sporadic) testcase on our end it's hard to make much progress.

[larssilven]

Found out the cause of the problem.
A thread still was blocked with a read when another thread closed the TCP. So that explains the exception from the read. A better exception::what() could have helped to understand the problem but I have not spent too much time on the issue so it is OK.

Thank you for your help and for your excellent library.