| Ticket UUID: | e533296a92d890449603ce54a5b1e2f04912e68a | ||
| Title: | SegFault on [info frame] on apply | ||
| Type: | Bug | Created on: | 2026-08-26 00:26:57 |
| Submitter: | venkat | Assigned to: | sebres |
| Subsystem: | 22. [proc] and [uplevel] | Severity: | Important |
| Priority: | 5 Medium | Last modified: | 2026-09-01 13:26:24 |
| Status: | Closed | Closed by: | sebres |
| Resolution: | Fixed | Closed on: | 2026-09-01 13:26:24 |
| Version: | 8.5+ | ||
| Description: | ||||
Running this script causes tcl to die with a segmentation violation. (This on MacOS from my own build)
proc probe {s} {
info frame -1
catch {apply {{s} {eval $s}} $s}
}
probe {catch {probe {error X}}; probe {}}
puts survived
| ||||
| User Comments: | ||||
oehhar added on 2026-08-26 12:16:36:
Good catch! Confirmed on 8.6.18 and 9.0.4 on MS-Windows. Thanks, Harald chw added on 2026-08-26 19:48:18:
Seems be present even in 8.5. This is my wildly guessed fix (diff against trunk):
---snip---
Index: generic/tclCmdIL.c
==================================================================
--- generic/tclCmdIL.c
+++ generic/tclCmdIL.c
@@ -1292,13 +1292,11 @@
case TCL_LOCATION_BC: {
/*
* Execution of bytecode. Talk to the BC engine to fill out the frame.
*/
- CmdFrame *fPtr = (CmdFrame *)TclStackAlloc(interp, sizeof(CmdFrame));
-
- *fPtr = *framePtr;
+ CmdFrame f = *framePtr, *fPtr = &f;
/*
* Note:
* Type BC => f.data.eval.path is not used.
* f.data.tebc.codePtr is used instead.
@@ -1328,11 +1326,10 @@
ADD_PAIR("cmd", TclGetSourceFromFrame(fPtr, 0, NULL));
if (fPtr->cmdObj && framePtr->cmdObj == NULL) {
needsFree = lc - 1;
}
- TclStackFree(interp, fPtr);
break;
}
case TCL_LOCATION_SOURCE:
/*
---snip---
sebres added on 2026-08-26 20:13:19:
I don't think it'd change something. I'll take a look - my fork doesn't segfault almost since scratch (8.5). sebres added on 2026-08-26 21:53:23:
I rebased it to 8.5 (initially based on that), but my original "fixed" variant was much more complex (and it is hardly possible to back-port)... Branch bug-e533296a92d89044-segfault contains the simplest fix, but the question still remains - why is a such procPtr (with cmd after apply is done) in the call-frames stack yet at that time point? apnadkarni added on 2026-08-27 03:31:55:
Sergey, I feel the fix proposed will solve this issue but there is a deeper, more serious problem as your question about procPtr implies. What I see is that the test script invokes
Note the last value in particular. Then the line fptr = TclStackAlloc returns 0x0000020cb5b110a0. Since sizeof(*fptr) is 88, this allocation overlaps the memory referenced by framePtr->framePtr->procPtr->cmdPtr. Consequently, this line overwrites It seems to me that some code path is releasing memory allocated by TclStackAlloc too early, resulting in that memory being overwritten while still in use. I don't understand the runtime frame management enough so you might be able to tell if that is the case and where it is happening much quicker than I could. apnadkarni added on 2026-08-27 03:34:54:
BTW, I think Christian's suggestion will also only hide the issue since it avoids use of chw added on 2026-08-27 05:02:58:
Ashok, indeed, my first shot did pure error hiding. Based on sebreas idea, a more gentle guess for 8.6 is here: https://androwish.org/home/info/b22c637978b91e3a sebres added on 2026-08-27 10:23:36:
I think setting As for Ashok's analyze, the mentioned overwrite looks like an after effect to me -the cmd pointer used in Tcl_ApplyObjCmd for procPtr is reserved in program stack and without the reset (leaving Tcl_ApplyObjCmd) a usage after free is possible there too. But as I said before, I also think that this would require further investigation. Unfortunately I cannot backport correct fix, because my TEBC uses completely different cmd-location approach (without scriptCLLocPtr and co) - the way how it gets "compiled" is simply different. sebres added on 2026-08-27 12:42:38:
Well, independently to the question of procPtr of finalized lambda in frames stack, the situation where procPtr->cmdPtr may be NULL is normal and expected, see for example SetLambdaFromAny. Thus the fix of me is basically correct (it simply restores the situation before the lambda invocation), so I'd merge it if no objections follow. sebres added on 2026-08-27 14:17:31:
OK, found and fixed it ultimately: this going to nested invocation of lambda, where by return from child to parent call, the info of lambda that got overwritten within child stack gets broken (or in my previous fix - get lost due to reset). Amend [a7a8993cca8e49c9] avoids this reset of lambda info in frames stack by nested invocations (reserves this info as long as it will be evaluated), so fill this info and reset the artificial command only once (at first/parent invocation); test now adjusted to cover it (lambda shall be in the last frame-info too). This makes it also a bit faster by nested invocations of lambda. However I'd admit that for performance reasons, one could rewrite this to initialize procPtr->cmdPtr only once by first invocation (as long as the lamda exists), but this would expect another clean-up and referring mechanism. jan.nijtmans added on 2026-08-27 15:03:56:
I put chw's proposal in a [152c3d8cd63c99da|branch] (based on 8.6, since I don't care about 8.5 any more ...., and without setting a field to NULL before a ckfree). This way, the two solutions (sebres' and chw's) can be compared. It appears to work for me. Can you (@sebres and @chw) come to an agreement which one is best? sebres added on 2026-08-27 18:09:57:
I backported original idea in [4ca6eb5618ce2003] and better test to this branch (as new branch) which is NRE capable now, but... as already said...
I'll try to provide better one (or backport some part from my fork). sebres added on 2026-08-27 21:19:41:
Branch bug-e533296a92-sebres-optimized is currently pure experimental (even tests would probably fail), because there is no proper clean-up/reference model for lambda-commands to nsPtr (so they are not dereferenced and apply doesn't detect yet if namespace get deleted). But the idea is very interesting in my opinion... And because that unneeded interim alloc/free of `cmdPtr` is removed in `apply` now, the branch has a bit better performance:
sebres added on 2026-08-27 22:09:32:
OK, OO is also affected by similar issue (old value of cmdPtr left in `procPtr->cmdPtr`), just it is not covered now in tests. jan.nijtmans added on 2026-08-28 08:18:50:
Thanks for doing all this! My conclusion: let's not rush this in Tcl 9.1b1. Since it's a long-existing problem without anyone noticing it, it doesn't have 'High' priority IMHO. apnadkarni added on 2026-08-28 10:07:35:
@sebres, @chw - seems reasonable to me at the level of my competency. Agree with Jan that not worth risking putting in 9.1b1. sebres added on 2026-08-31 21:44:27:
Thus fixed with mixed variant of me and Christian in 8.6 ([107a8d20dd3eac4a])... The optimized variant is also ready and can be merged to trunk as suggested by Jan and Ashok... I have no time currently and there seems to be some bug in 9.x right now, see [4937c309f4ce3657], so I merged and adjusted 8.6 only to 9.0 (blind, so wait for the tests). Thus close as fixed. jan.nijtmans added on 2026-09-01 07:20:48:
This happens with --enable-symbols builds now (seen on both Linux and MacOS): regexp.test Test file error: tcltest: /home/runner/work/tcl/tcl/generic/tclProc.c:2759: ApplyNR2: Assertion `procPtr->cmdPtr == data[0]' failed. So, I don't consider it fixed yet :-( sebres added on 2026-09-01 09:55:41:
Hmm... I cannot reproduce it on 8.6, so it looks like something with merge to 9.0 (there were some conflicts) or the fix is not suitable for 9.x (I didn't tested it in 9.x, because of build-bug I mentioned before). sebres added on 2026-09-01 10:07:46:
OK, it seems to be not suitable indeed (but I guess 8.6 may get it also some days and it is simply retarded for some reason) - test regexp-27.8 seems to destroy the lambda representation (shimmer to list due to lindex), so it is typical use after free, this time in NRE-callback... It'd be hardly solvable this way, so instead I tend to merge my optimized branch (which is not affected by that issue due to proper refcounting and clean-up in TclProcCleanupProc). sebres added on 2026-09-01 11:05:51:
Yep, 8.6 was affected too (see new test apply-9.4)... By the way, can someone take a look to the 9.x bug [4937c309f4ce3657] (and fix build with disabled zipfs)? apnadkarni added on 2026-09-01 11:45:31:
Sergey, GH actions show an assertion failure in the latest run with --enable-symbols. Looks like it might be related to your changes. Could you take a look? sebres added on 2026-09-01 13:26:24:
Yes, it was exactly the error Jan meant previously in [20f94f9068ad1e72]. This must be fixed now by merge of my optimized branch, which was not affected by the issue... Any attempt to handle it with NRE (or directly in call-stack) are doomed to fail with more "use after free" scenarios, because shimmering could remove the lambda representation and cmdPtr stored there shall not be restored, unless we'd get another one in followed nested call, so one'd rather need proper refcounting for that). | ||||
