Hi. I originally posted this here: https://sourceforge.net/p/tls/bugs/65/
I'll copy the original report. At the end I include new information.
I recently ran into a case where my application would crash when using this library (1.6.7). I determined it to be related to the addition of CRYPTO_set_locking_callback() and CRYPTO_set_id_callback(). Specifically, when I deleted the Tcl interpreter (but my application remained running), OpenSSL still called the callback functions, but they were no longer available.
Here are the general steps to reproduce this behaviour:
Have an application with an OpenSSL SSL/TLS connection
Create a Tcl interpreter
Load this package
Delete the Tcl interpreter
Try to interact with the SSL/TLS connection
I have attached a patch that resolves the problem. It works by registering another callback for when the Tcl interpreter is deleted. At that point, we set the OpenSSL callbacks to what they were prior to this library changing them.
I wrote a post about this issue and my investigation if you would like additional background:
I am not certain that the patch resolves all possible issues with using these OpenSSL callbacks however. For example, consider the case where we have an application that does the following:
Loads this library (leading to us setting the callbacks)
Application then sets its own OpenSSL callbacks (unaware they were set)
At that point, can we guarantee the new callbacks are sufficient? Also, if we then throw into the mix unloading this library, we're in the situation where there are no callbacks set at all.
Perhaps this is a far out possibility. It makes me wonder whether it is appropriate for this library to be setting these at all though. I would be very interested in your thoughts on the matter as I am by no means an expert here.
Original patch (1.6.7): https://sourceforge.net/p/tls/bugs/65/attachment/tcl-tls-openssl-callbacks.diff
I re-tested this with 1.7.11. I found the crash still occurs. I updated my patch against 1.7.11:
One thing I have not figured out yet is this: The crash happens when I test on Debian Jessie (both 1.6.7 and 1.7.11), but not when I test in Debian Stretch. I believe this may be due to the different openssl version. (Jessie is on 1.0.1 and Stretch on 1.1.0).
I noticed there is now a parameter to deinitialize the interpreter (to TclLibInit). I wonder if my strategy could be combined better with that approach.
If I can provide any more information, please let me know.
Thanks for your time.