Check-in [61ea8fa199]

Login
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:TIP #509: Initial draft complete
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 61ea8fa199ebe88172d0e8df9a3342566b2b52c977f62ae3f52d5e011b0a9c0c
User & Date: fbonnet 2018-05-25 21:51:45
Context
2018-05-26
09:49
Fixed typo in JO's name (SHAME!) check-in: 22823b2821 user: fbonnet tags: trunk
2018-05-25
21:51
TIP #509: Initial draft complete check-in: 61ea8fa199 user: fbonnet tags: trunk
09:44
Add more examples to 500 check-in: ca2e14f744 user: dkf tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to tip/509.md.

     7      7   	Post-History:   
     8      8   	Keywords:       Tcl,threads
     9      9   	Tcl-Version:	8.7
    10     10   -----
    11     11   
    12     12   # Abstract
    13     13   
    14         -TODO
           14  +This TIP proposes to improve the `Tcl_Mutex` API by enforcing a consistent
           15  +behavior on all core-supported platforms regarding reentrancy.
    15     16   
    16     17   # Context
    17     18   
    18         -TODO
           19  +This TIP is inspired by a [request from
           20  +FlightAware](https://github.com/flightaware/Tcl-bounties#make-tclxs-signal-trap-handlers-safe-to-use-with-threaded-tcl)
           21  +to fix deadlock issues with TclX signal handling. A specific issue has been
           22  +opened to discuss the proposed implementation [here](https://github.com/flightaware/Tcl-bounties/issues/32).
           23  +
    19     24   
    20     25   # Rationale
    21     26   
    22         -TODO
           27  +As of Tcl 8.6, the man page for thread support `Thread.3` states that:
           28  +
           29  +> The result of locking a mutex twice from the same thread is undefined. On some
           30  +> platforms it will result in a deadlock.
           31  +
           32  +On Windows platforms, mutexes are implemented using Win32 critical sections,
           33  +which are reentrant. `Tcl_Mutex` are just plain `CRITICAL_SECTION *`.
           34  +
           35  +On Unix platforms (this includes MacOS X), Tcl relies on the **pthread** library
           36  +for multithreading and synchronization primitives such as mutexes. `Tcl_Mutex`
           37  +are just plain `pthread_mutex_t *`. **pthread** mutexes are not reentrant by
           38  +default, though the `PTHREAD_MUTEX_RECURSIVE` attribute can be used at creation
           39  +time to make them so, but this possibility is not available on older systems.
           40  +
           41  +The Tcl philosophy has always been to erase platform-specific peculiarities in
           42  +favor of overall multi-platform consistency, sometimes to the point of
           43  +implementing or emulating commonly available features on less capable platforms.
           44  +Therefore it feels natural to pursue this goal by making `Tcl_Mutex` reentrant
           45  +on all platforms and achieving a consistent behavior on both Windows and Unix.
    23     46   
    24     47   # Specifications
    25     48   
    26         -TODO
           49  +This TIP proposes to replace the following text from the `Thread.3` man page:
           50  +
           51  +> The result of locking a mutex twice from the same thread is undefined. On some
           52  +> platforms it will result in a deadlock.
           53  + 
           54  +by the following text:
           55  +
           56  +> Mutexes are reentrant: they can be locked several times from the same thread.
           57  +> However there must be exactly one call to **Tcl_MutexUnlock** for each call to
           58  +> **Tcl_MutexLock** in order for a thread to release a mutex completely.
    27     59   
    28     60   # Portability issues
    29     61   
    30         -TODO
           62  +## Windows
           63  +
           64  +Mutexes are naturally reentrant on Windows systems, so no special work is
           65  +required.
           66  +
           67  +## Unix
           68  +
           69  +On **pthread**-based Unix systems that support the `PTHREAD_MUTEX_RECURSIVE`
           70  +attribute, all `pthread_mutex_t` made available as `Tcl_Mutex` will be created
           71  +using this attribute. This includes all but the oldest variants of Unix.
           72  +
           73  +On **pthread**-based Unix systems that do not support the
           74  +`PTHREAD_MUTEX_RECURSIVE` attribute, reentrancy will be achieved by combining a
           75  +regular, non-reentrant `pthread_mutex_t`, with a thread-specific lock counter
           76  +accessible through a `pthread_key_t` data key. This counter keeps track of the
           77  +number of calls to `Tcl_MutexLock` minus the number of calls to
           78  +`Tcl_Mutex_Unlock`. `Tcl_MutexLock` increments the counter, but only calls
           79  +`pthread_mutex_lock` when the initial value is zero. `Tcl_Mutex_Unlock` behaves
           80  +symmetrically: it decrements the counter, and only calls `pthread_mutex_lock`
           81  +when it reaches zero. This ensures that a thread never calls 
           82  +`pthread_mutex_lock` twice on the same mutex, and only calls 
           83  +`pthread_mutex_unlock` when the thread no longer holds it.
           84  +
           85  +Detection of `PTHREAD_MUTEX_RECURSIVE` availability is done at configure time
           86  +thanks to the `AC_CHECK_DECLS` autoconf macro in `tcl.m4`.
           87  +
           88  +# Potential incompatibilities
           89  +
           90  +Although this TIP introduces a major change to `Tcl_Mutex` behavior on Unix, it
           91  +is very unlikely that this will break any existing code: 
           92  +
           93  +- Windows-only code will see no change in behavior.
           94  +- Unix-only code with reentrant mutexes is fatally flawed in the current state
           95  +  of the Tcl core, since this results in a deadlock. At best, this TIP will fix
           96  +  such hard-to-reproduce situations (case in point: TclX signals), and the code
           97  +  will transition from the nonworking state to the working state (which,
           98  +  according to Dr. John Ousterhoot, is the best performance improvement). At
           99  +  worst, this change will trigger a new class of reentrancy-related bugs on
          100  +  already broken code.
          101  +- Multiplatform code in its current form behaves either inconsistently or
          102  +  consistently, depending on whether it uses reentrant mutexes or not.
          103  +  Consistent code will remain consistent, inconsistent code will become
          104  +  consistent as the Unix version aligns with the Windows version.
    31    105   
    32    106   # Related Bugs
    33    107   
    34         -TODO
          108  +[Bug #f4f44174](https://core.tcl.tk/tcl/tktview/f4f44174) demonstrates the
          109  +deadlock issue with a script based on TclX. The root cause is the asynchronous
          110  +event handler's `Tcl_Mutex` being locked twice from the same thread when a
          111  +signal handler interrupts a thread in the middle of a mutex-protected section,
          112  +which on Unix platforms results in a deadlock. The proposed implementation fixes
          113  +this issue.
    35    114   
    36    115   # Implementation
    37    116   
    38    117   The proposed implementation is available on branch
    39    118   [tip-509](https://core.tcl.tk/tcl/timeline?r=tip-509) in the Tcl Fossil
    40    119   repository.
    41    120   
    42    121   # Copyright
    43    122   
    44    123   This document has been placed in the public domain.