Tcl Source Code

View Ticket
Login
☰Core Home Timeline Branches Tags Forum Tickets Download Wiki
Attach Edit New Ticket Plaintext
Ticket UUID: c35e347c6fb8df66917b1ab6974b3638925b4aa2
Title: Concurrent Tcl_Init calls race in Unix locale initialization, causing crashes on OpenBSD
Type: Bug Created on: 2026-10-07 09:19:24
Submitter: gustafn3 Assigned to: apnadkarni
Subsystem: 20. [interp] Severity: Important
Priority: 5 Medium Last modified: 2026-11-06 09:58:06
Status: Closed Closed by: apnadkarni
Resolution: Fixed Closed on: 2026-11-06 09:58:06
Version: 9.1.0
Description:

Title: Concurrent Tcl_Init calls race in Unix locale initialization, causing crashes on OpenBSD

Concurrent initialization of Tcl interpreters can cause a double free or use-after-free in setlocale(). I reproduced this with standalone Tcl 9.1.0 on OpenBSD and verified the attached patch against the reproducer.

The bug was initially observed in NaviServer where it can be mitigated by forcing serialized interpreter creation via paramter seetings (ns_param tclinitlock true). The attached script triggeres the same crash with plain Tcl 9.1.0.

Environment

  • Tcl 9.1.0, built from the release sources with --enable-symbols
  • Thread extension 3.0.7, bundled with Tcl
  • OpenBSD 7.9, arm64
  • UTM VM on an Apple M1, using hardware virtualization

The original report came from a NaviServer user on OpenBSD amd64, whose backtraces showed the same Tcl initialization path. The attached standalone reproducer requires no NaviServer.

Reproduction

The attached tcl-thread-init-race.tcl establishes eight persistent creator threads. They rendezvous before each round, then each creates and joins a fresh child thread. Child initialization invokes Tcl_Init() through the Thread extension.

Run:

   tclsh9.1 tcl-thread-init-race.tcl serialized 8 50
   tclsh9.1 tcl-thread-init-race.tcl parallel 8 50

The serialized control protects thread::create with a shared mutex. Since thread::create waits for child initialization to finish, this serializes the child initialization.

Results with unmodified Tcl 9.1.0:

   mode=serialized creators=8 rounds=50
   completed=400 failed_creators=0 (no crash observed)

   mode=parallel creators=8 rounds=50
   tclsh9.1(54092) in free(): double free 0x23c0cc14b0
   Abort trap (core dumped)

Backtrace

Relevant frames from the standalone Tcl crash:

_libc_free
freegl                              libc/locale/setlocale.c:33
_libc_setlocale                     libc/locale/setlocale.c:178
Tcl_GetEncodingNameFromEnvironment  unix/tclUnixInit.c:591
TclpSetInitialEncodings             unix/tclUnixInit.c:540
Tcl_Init                           generic/tclInterp.c:845
NewThread                          thread3.0.7/generic/threadCmd.c:1857
NewThreadProc                      generic/tclEvent.c:2070
_rthread_start

The original NaviServer report also showed a crash in strlen() called by strdup() inside setlocale(), reached through the same Tcl functions.

Cause

Tcl_Init() calls TclpSetInitialEncodings(), which calls Tcl_GetEncodingNameFromEnvironment(). Under HAVE_LANGINFO, that function performs:

   setlocale(LC_CTYPE, "")

followed by nl_langinfo(CODESET) and a copy of the returned string.

This sequence is not synchronized between concurrent interpreter initializations. setlocale() modifies process-global state and is not required by POSIX to be thread-safe. OpenBSD’s implementation can free and replace shared locale storage, explaining the observed double free and use-after-free.

The initial locale setup in Tcl_InitSubsystems() already uses TclpInitLock(), but the repeated locale detection during interpreter initialization does not.

Proposed fix

The attached patch for Tcl 9.1.0 uses the existing TclpInitLock() / TclpInitUnlock() around:

  1. setlocale(LC_CTYPE, "")
  2. nl_langinfo(CODESET)
  3. Copying the codeset into a Tcl DString

It releases the lock before resolving the copied encoding name or calling Tcl_GetEncoding().

This coordinates the repeated locale detection with Tcl’s initial locale setup without serializing the whole of Tcl_Init(). The existing locale-failure fallback and weak-import handling are preserved.

The patch is intentionally not OpenBSD-specific. Other Unix libc implementations may also require synchronization, although the crash described here is confirmed on OpenBSD. The lock coordinates Tcl’s own calls; it cannot coordinate unrelated libraries calling setlocale() independently.

Validation

After applying the patch, rebuilding, and reinstalling Tcl:

   tclsh9.1 tcl-thread-init-race.tcl parallel 8 500
   tclsh9.1 tcl-thread-init-race.tcl serialized 8 500

Both runs completed successfully:

   mode=parallel creators=8 rounds=500
   completed=4000 failed_creators=0 (no crash observed)

   mode=serialized creators=8 rounds=500
   completed=4000 failed_creators=0 (no crash observed)

These stress runs support the fix; they do not establish that every possible concurrency issue is absent.

User Comments:
serhiy.storchaka added on 2026-10-13 19:37:12:

Reproduced on OpenBSD 7.9 amd64 with the current main: the parallel mode of the attached script crashed in 5 of 5 runs (double free in free()), the serialized mode did not. With the attached patch it ran without a crash in 10 of 10 runs and in a run with 500 rounds. No caller of Tcl_GetEncodingNameFromEnvironment() holds the initialization lock, so the patch cannot deadlock.

The patch is in branch unix-locale-init-lock.

jan.nijtmans added on 2026-10-18 21:02:58:

Question: Is this already fixed now by [63941fee4f]? Maybe this patch is no longer necessary.

serhiy.storchaka added on 2026-10-18 21:44:14:

Partly. Since [63941fee4f], Tcl_Init() no longer calls setlocale() once the initial encoding is set, so tcl-thread-init-race.tcl no longer crashes on OpenBSD (0 of 10 runs, 10 of 10 just before that merge). But Tcl_GetEncodingNameFromEnvironment() still calls setlocale() without a lock: from [encoding user], from Tcl_GetEncodingNameForUser(), and from every Tcl_Init() as long as the initial encoding was not found. The attached c35e347c6f-encoding-user-race.tcl (8 threads doing [encoding user] in a loop) crashes the current trunk on OpenBSD in 10 of 10 runs, and does not crash with the patch in branch unix-locale-init-lock. So the patch is still needed.

gustafn3 added on 2026-10-19 08:24:00:

Jan. By looking at the referenced change, I expect it to prevent the crash in the original reproducer: once initial encoding setup succeeds in the main thread, subsequent interpreter initializations no longer enter the locale-detection path.

However, the change does not synchronize the setlocale() / nl_langinfo() sequence itself. The initialEncodingSet flag is unsynchronized, and direct calls to Tcl_GetEncodingNameFromEnvironment() or Tcl_GetEncodingNameForUser() still reach that sequence.

My original patch protects locale detection inside Tcl_GetEncodingNameFromEnvironment(), including copying the codeset while holding the lock. I therefore think the locking remains useful, even if the newer change removes the repeated-initialization trigger observed in NaviServer and in the rest script

One one to distinguishing whether the original reproducer passes from whether concurrent calls to the locale-detection API are safe.

jan.nijtmans added on 2026-10-19 11:19:07:

Understood.

Just one small remark: variable 'ds' is shadowed. Fixed [6e63a280be|here]

@ashok, can you have a final look?

apnadkarni added on 2026-10-20 08:05:30:

Not looked at this yet, but generally speaking, my preference is to keep interp creation (Tcl_Init being part of it) as performant as possible. Part of that is minimizing calls as well as synchronization. To that end, is there a reason for retrieving the locale from the environment on a per interp basis and not once per process (or per thread)? If not, could this be subsumed into Tcl_InitSubsystems? Thoughts?

In any case, I've put this on my queue :-)

gustafn3 added on 2026-10-20 08:38:07:

Ashok. yes, process-wide setup would be preferable. From the embedding application's perspective, there is no need to reset the process locale whenever another interpreter is initialized.

The expected performance differences depend on the initialization scope:

  • Per-interpreter setup repeats locale detection for every interpreter.
  • Per-thread setup reduces that work when a thread creates multiple interpreters, but still repeats process-global locale operations for each thread.
  • Per-process setup avoids both forms of repetition.

On OpenBSD, the current unsynchronized calls can crash, so synchronization is required if those calls remain. A per-interpreter lock would serialize the locale-detection section; a per-thread approach would reduce contention only where multiple interpreters share each thread. Note, that the submitted crash-producer creates one interpreter per fresh child thread, so per-thread initialization alone would not reduce its locale-call count.

On glibc-based Linux and macOS, the libc implementations already lock locale updates internally. Repeating those calls therefore already incurs synchronization, even without an additional Tcl lock. Avoiding redundant calls could benefit probably those platforms as well. Also, NaviServer will benefit from this.

The apparent reason for repeated calls is the startup sequencing described above TclpSetInitialEncodings(): an early attempt may lack the information needed to load the appropriate Tcl encoding. That seems to justify retrying encoding resolution, rather than repeating libc locale setup.

Could those steps be separated—capturing the locale/codeset during the existing protected initialization in Tcl_InitSubsystems(), then resolving the corresponding Tcl encoding later if necessary?

My patch was intended as a minimal correctness fix for the demonstrated race under OpenBSD; I would welcome an initialization change that removes the repeated work. The public Tcl_GetEncodingNameFromEnvironment() and Tcl_GetEncodingNameForUser() entry points would still need their behavior and concurrent use considered separately.

Thanks for looking into this!

serhiy.storchaka added on 2026-10-20 08:42:50:

Since [63941fee4f] it already works mostly that way: the locale is read once, by TclpSetInitialEncodings() called from Tcl_FindExecutable() after Tcl_InitSubsystems(), and its call in Tcl_Init() returns at once as soon as the initial encoding is set. So the lock in unix-locale-init-lock is not taken when an interp is created, except while the initial encoding could not be set yet.

It is needed for the public Tcl_GetEncodingNameFromEnvironment() and Tcl_GetEncodingNameForUser(), which call setlocale() every time, also from [encoding user]; that is what the attached reproducer crashes. Their result could be cached too, but then a later setlocale() by the application would not be seen.

gustafn3 added on 2026-10-20 08:55:06: 
Thanks, Serhiy ... this resolves the performance aspect. With the successful-initialization guard in place, the locale lock is outside the normal interpreter-creation path, so the concern about repeated synchronization is already addressed there.

The combination makes sense to me: avoid repeated initialization once it has succeeded, and protect the locale operations when explicitly reached through the public query functions or during initialization retries. Caching the public-query results would be a separate behavior change and does not seem necessary here.

For clarity, my original Tcl 9.1.0 reproducer crashed during concurrent Tcl_Init() calls, without calling [encoding user]. The public-query case demonstrates why locking remains relevant after the initialization change.
apnadkarni added on 2026-11-06 09:58:06: 
Fixed for 9.0 in [4f47985ff9] and 9.1 in [40f9131022] respectively.

Thanks for the fix.

Attachments:

This page was generated in about 0.003s by Fossil 2.29 [912711f044] 2026-10-17 13:14:07