|
2026-10-17
| ||
| 08:10 | • Ticket [f2ff05fc84] Usage of static systemEncoding is not thread safe (vulnerabi... artifact: 4a284f5311 user: serhiy.storchaka | |
| 08:06 | Fix [f2ff05fc84]: a conversion with the system encoding could crash if another thread changed the sy... check-in: d518a8ab5d user: serhiy.storchaka tags: system-encoding-keep-retired | |
|
2025-08-14
| ||
| 11:47 | • Ticket [87b69745be] interp creation resets encoding directory search path st... artifact: 3f2c8da63a user: sebres | |
|
2025-08-13
| ||
| 17:59 | • Ticket [2570123b63] system encoding is not protected from freeing during encode/... artifact: fc3258f5aa user: sebres | |
| 16:11 | • Ticket [87b69745be] interp creation resets encoding directory search path st... artifact: 1aa7e19a58 user: sebres | |
|
2025-07-17
| ||
| 12:40 | • Ticket [61c01e0edb] likely race condition with package require in several thread... artifact: e0925741a7 user: sebres | |
|
2024-04-26
| ||
| 14:45 | • Ticket [f2ff05fc84] Usage of static systemEncoding is not thread safe (vulnerabi... artifact: 4dd9da46fd user: sebres | |
| 14:27 | • Ticket [f2ff05fc84]: 3 changes artifact: 3dbdbaa10c user: sebres | |
| 13:47 | (backported from tclSE) avoid multi-threaded vulnerability of system encoding (see tcl bug [f2ff05fc... check-in: a1c84c8069 user: sebres tags: sebres-encoding-perf-branch | |
|
2021-04-15
| ||
| 15:33 | • Ticket [f2ff05fc84] Usage of static systemEncoding is not thread safe (vulnerabi... artifact: b23cf44e38 user: sebres | |
|
2021-04-07
| ||
| 09:05 | • Ticket [f2ff05fc84]: 5 changes artifact: a0b938651a user: jan.nijtmans | |
|
2021-04-06
| ||
| 16:14 | • Ticket [f2ff05fc84]: 3 changes artifact: 788c4d160f user: sebres | |
| 16:14 | • New ticket [f2ff05fc84]. artifact: a3963ccbe8 user: sebres | |
| Ticket UUID: | f2ff05fc8411be80a93913b2b065f2c85fa9bcf0 | ||
| Title: | Usage of static systemEncoding is not thread safe (vulnerability) | ||
| Type: | Bug | Created on: | 2021-04-06 16:14:12 |
| Submitter: | sebres | Assigned to: | sebres |
| Subsystem: | 11. Conversions from String | Severity: | Severe |
| Priority: | 7 High | Last modified: | 2026-10-17 08:10:31 |
| Status: | Open | Closed by: | nobody |
| Resolution: | None | Closed on: | |
| Version: | 9.0.5, 9.1.0 | ||
| Description: | ||||
|
I know that normally the change of system encoding is either rare case, but a vulnerability remains vulnerability... Atomic incr/decr of its refCount may help here... But currently (without to have such atomic facilities) it would imply to use mutex, and it would be ugly by each conversion (would provide a multi-threaded bottleneck, even twice by start and end of convert-routines). As for atomic facilities, I made some for another issue - for example see sebres/tcl/1e522ff/tclInt.h#L131. Simplest scenario to reproduce the SF (for understanding):
| ||||
| User Comments: | ||||
jan.nijtmans added on 2021-04-07 09:05:34:
Any suggestion for a solution? Maybe store the system-encoding in thread local storage? That would mean that each thread has its own system-encoding. And that any time a new thread is created, it takes over the system encoding from its parent thread. sebres added on 2021-04-15 15:33:49:
Well, TSD solution would be possible, but in simplest case it would imply indirectly that the setting of system encoding affects only current thread (and all of its interpreters), what would be backwards incompatible. Here is a diff explaining that:
Sure, one could additionally store an epoch in TSD and compare it with a common static epoch which gets incremented by Tcl_SetSystemEncoding, so an access by Tcl_GetEncoding(interp, NULL) could check it and invalidate cached in TSD encoding if epoch changed globally. Just whole performance of that (TSD+epoch) will be almost comparable to a simple lock variant (in case of NULL by encoding), so similat to: get and incr ref in lock, convert, and decr ref in lock at end. FWIW: in my fork I used similar way, without TSD and interim lock (with atomic refCount increment/decrement and an epoch used only for Tcl_ObjType "encoding" cached in tcl-object). sebres added on 2024-04-26 14:27:32:
I backported parts of my tcl-fork responsible for encoding in sebres-encoding-perf-branch, which fixes this issue (it is 8.5 based, because it was initially implemented to that branch on my side). This also provides a performance tests showing grave improvement of that branch for all encoding operations (up to 1000x). Here is small excerpt illustrating that: ******************************************************************************** - Total 8 cases in 2.06 sec. (1.86 net-sec.): + Total 8 cases in 2.05 sec. (1.82 net-sec.): - 1734.821698 µs/# 5967895 # 3201857.947 #/sec 1863.885 net-ms + 2.314888 µs/# 8030184 # 4423539.522 #/sec 1815.330 net-ms Average: - 216.852712 µs/# 745986 # 3201849 #/sec 232.986 net-ms + 0.289361 µs/# 1003773 # 4423544 #/sec 226.916 net-ms Min: - 0.124842 µs/# 1692545 # 8010151 #/sec 211.300 net-ms + 0.125721 µs/# 1681033 # 7954126 #/sec 211.341 net-ms Max: - 478.994 µs/# 522 # 2087.7 #/sec 250.035 net-ms + 0.550516 µs/# 435911 # 1816477 #/sec 239.976 net-ms ******************************************************************************** ... ******************************************************************************** - Total 2 cases in 0.52 sec. (0.50 net-sec.): + Total 2 cases in 0.52 sec. (0.47 net-sec.): - 720.021000 µs/# 1391 # 2778.577 #/sec 500.616 net-ms + 0.774522 µs/# 1317842 # 2805745.869 #/sec 469.694 net-ms Average: - 360.010500 µs/# 695 # 2777 #/sec 250.308 net-ms + 0.387261 µs/# 658921 # 2805746 #/sec 234.847 net-ms Min: - 353.664 µs/# 708 # 2827.5 #/sec 250.394 net-ms + 0.274761 µs/# 839609 # 3639523 #/sec 230.692 net-ms Max: - 366.357 µs/# 683 # 2729.6 #/sec 250.222 net-ms + 0.499761 µs/# 478233 # 2000958 #/sec 239.002 net-ms ******************************************************************************** sebres added on 2024-04-26 14:45:45:
The encoding cache introduced there doesn't have a goal to improve performance only - the second major job, it does, is a protection of "vulnerable" call Tcl_GetEncodingName(NULL) (see lines 984-987 for details). Now the cache avoids that situation, as long as the system encoding would not be switched too often (hardly imaginable in normal regular case, that 4 threads switch it to more than 4 different encodings at once). serhiy.storchaka added on 2026-10-17 08:10:31:
Reproduced on 9.0 and main: one thread doing [file exists] on a long path while another switches [encoding system] crashes every run. Also Tcl_GetEncoding(NULL, NULL) returned systemEncoding read again after unlocking the mutex. Branch system-encoding-keep-retired is a simple alternative to sebres-encoding-perf-branch: former system encodings are kept until finalization, so a conversion can keep using the one it read, without a mutex or atomic reference counts. (The atomic reference count in sebres-encoding-perf-branch narrows the race but does not close it: the encoding can be freed between reading systemEncoding and incrementing its count.) A reference in each conversion with a mutex works too, but made [file exists] in 4 threads 1.75 times slower here. Visible change: a former system encoding is not unloaded when [encoding dirs] changes (encoding-2.2 adjusted). Test thread-9.1. We can find a better solution later, but it is better to fix the crash now. | ||||
