|
2025-08-18
| ||
| 21:53 | • Ticket [8a13caa31d] Setting of system encoding isn't applied to stdout on ce... artifact: 6192bb839e user: sebres | |
| 21:46 | circumvent for [8a13caa31dd8e3a5]: load tcltest if needed (using `load {} Tcltest`) - as a result tc... leaf check-in: 6e6d953e8c user: sebres tags: experimental-load-tcltest-implicitly | |
| 18:56 | • Ticket [8a13caa31d] Setting of system encoding isn't applied to stdout on ce... artifact: 4572100cd5 user: sebres | |
| 17:10 | • Ticket [8a13caa31d]: 5 changes artifact: 4dc4f1b7e3 user: apnadkarni | |
| 15:18 | • Ticket [8a13caa31d]: 5 changes artifact: d07bfd2fdf user: sebres | |
| 13:30 | • Closed ticket [8a13caa31d]. artifact: fe8063bdec user: apnadkarni | |
| 11:18 | • Ticket [8a13caa31d]: 3 changes artifact: dc84fb5339 user: apnadkarni | |
| 05:47 | • Ticket [8a13caa31d]: 3 changes artifact: 03240dbc71 user: apnadkarni | |
|
2025-08-17
| ||
| 17:18 | • Ticket [8a13caa31d]: 3 changes artifact: 4b3d643820 user: apnadkarni | |
| 11:54 | • Ticket [8a13caa31d]: 3 changes artifact: 92c104578d user: apnadkarni | |
| 11:23 | • Ticket [8a13caa31d]: 3 changes artifact: 4856538222 user: apnadkarni | |
| 06:33 | • Ticket [8a13caa31d]: 3 changes artifact: 71ced54c5e user: apnadkarni | |
|
2025-08-16
| ||
| 23:02 | • New ticket [8a13caa31d]. artifact: a90b13432a user: sebres | |
| Ticket UUID: | 8a13caa31dd8e3a546b8d44c235bb98cd52bd774 | ||
| Title: | Setting of system encoding isn't applied to stdout on certain circumstances | ||
| Type: | Bug | Created on: | 2025-08-16 23:02:32 |
| Submitter: | sebres | Assigned to: | nobody |
| Subsystem: | 20. [interp] | Severity: | Severe |
| Priority: | 5 Medium | Last modified: | 2025-08-18 21:53:49 |
| Status: | Closed | Closed by: | sebres |
| Resolution: | Fixed | Closed on: | 2025-08-18 21:53:49 |
| Version: | 9.0 | ||
| Description: | ||||
|
I like it when an experimental branch brings to light problems that essentially have nothing to do with the experiment itself. My lazy branch (commit [6a997a7f19eaa6fa]) I provided for [effa2e2346f8372a] shows following error in encoding.test now:
# Bug #219314 - if we don't free escape encodings correctly on channel
# closure, we go boom
set file [makeFile {
encoding system iso2022-jp
set a "乎乞也"; # 3 Japanese Kanji letters
puts $a
} iso2022.tcl]
set f [open "|[list [interpreter] $file]"]
fconfigure $f -encoding iso2022-jp
set count [gets $f line]
close $f
removeFile iso2022.tcl
list $count $line
---- Result was:
9 \x00\x00\x00\x00\x00\x00\x00\x00\x00
---- Result should have been (exact matching):
3 \u4E4E\u4E5E\u4E5F
==== encoding-24.3 FAILED
This particularly means that Changing of stdout encoding directly like here "fixes" the test:
Also it is not clear why before the commit [6a997a7f19eaa6fa] it works as expected. Since it looks like an ugly Heisenbug and it is encoding issue (again), I think it were good to document this strange behavior. | ||||
| User Comments: | ||||
apnadkarni added on 2025-08-17 06:33:28:
I am actually confused by that test. When the child process runs the And to further the confusion, why would the test fail in your apn-oo-lazy-init-sbmod but not in apn-oo-lazy-init or main? I'm speculating that the difference is the point at which the stdout channel is opened. If it was already open, the test would fail but it is not obvious looking at the diffs why the two branches would differ in that respect. May be either ObjPrintf or the Pkg* related calls in apn-oo-lazy-init-sbmod cause stdout to be opened earlier? Can't think of what else it might be without further investigation. apnadkarni added on 2025-08-17 11:23:36:
Curiously, tclsh passes the test fine, tcltest fails. So likely some difference in the initialization sequence. I'm also wondering why the read output is nuls. Definitely worth tracking down. tclsh:
tcltest on the other hand:
apnadkarni added on 2025-08-17 11:54:10:
And to further narrow down and eliminate the testing framework:
apnadkarni added on 2025-08-17 17:18:36:
@sergey, just so you don't duplicate effort, I think I understand what's going on and it's not an issue with the new code. I'll update tomorrow. apnadkarni added on 2025-08-18 05:47:52:
So, as suspected, it is not an encoding issue as much as initialization of std channels. The test failure is a result of The difference between the In the case of the In the As an illustration, if the child process script in encoding-24.3 is modified to add a First, I think this is a bug in encoding-24.3 as it presumes stdout does not exist when a script is run and Tcl does not guarantee that is the case. Fixing the test by adding a fconfigure (as Sergey originally reported) may be one option but I am not sure if the original bug specifically was intended to the system encoding being attached to the standard channel. To fix the issue regardless, one possibility is to change As an aside, I'm surprised the test output prints nuls (x00) as the line read. That is a separate issue that might need to be looked at (perhaps it's the tcltest output ascii-fication getting it wrong). apnadkarni added on 2025-08-18 11:18:31:
I was wrong about one detail. It is not the package require command (or at least not the only command) that is causing the access to tclIndex resulting in channel stdout initialization. The sequence
in tclOOScript.tcl invokes the Given it is the oo initialization itself that created stdout, it seems the only option is to fix the test, which in my mind is reasonable to do. apnadkarni added on 2025-08-18 13:30:53:
@sebres, I am closing this ticket as moving of some of the oo init script to C on the main trunk also fixes this issue and I think this lazy init should be postponed to 9.1 anyways. See the effa2e2346 ticket for more. Reopen if you think this is still pertinent. sebres added on 2025-08-18 15:18:51:
Good analyze.
Yes, but exactly that is the Heisenbug what we're looking for, isn't it? I mean why this 3 lines script shall cause that Tcl_OOInitStubs/tclOOScript.tcl/etc gets involved at all? There is nothing OO-related in that script (and a lot of others). And even if it is caused by tcltest itself, we shall probably rewrite it lazy or on demand too. Since tcltest is used in a lot of test cases as [interpreter], it'd also make tcltest faster. apnadkarni added on 2025-08-18 17:10:55:
@sebres, are you suggesting keeping this bug open or that the lazy initializing should also be in 9.0, not just 9.1? Part of the reason for closing this is to track the "lazy oo init" project in a single ticket [effa2e2346]. Indeed the impact of this lazy change on that three line script was mystifying. At least to some extent, I feel this is the result of a rather convoluted initialization sequence (perhaps necessarily so) that I just documented in the wiki. If that could be streamlined and well defined (it currently depends on which of several Tcl initialization functions is called first), many of these "heisenbugs" would go away I think. sebres added on 2025-08-18 18:56:50:
I mean that probably we shall load tcltest (or invoke Or to detach Basically, the main goal would be to make tcltest executable work more similar to tclsh, because otherwise even in test-suite we mostly test tcltest executable, which is according to new findings slightly different (at least if invoked as a sub process). sebres added on 2025-08-18 21:53:49:
Branch experimental-load-tcltest-implicitly, particularly [6e6d953e8c2eab8b] illustrate the possibility to circumvent the issues like that and makes tcltest executable to work more similar to tclsh, so would avoid certain side effects and speed-up the interpreter creation from this exe (for cases where tcltest module is unneeded). I also believe that we can improve it more, e. g. to make calling of `Tcl_OOInitStubs` from there also optional (lazy, on demand, implicitly, however - but only there where OO really needed). | ||||
