Tcl Source Code

View Ticket
Login
Ticket UUID: e533296a92d890449603ce54a5b1e2f04912e68a
Title: SegFault on [info frame] on apply
Type: Bug Created on: 2026-08-26 00:26:57
Submitter: venkat Assigned to: sebres
Subsystem: 22. [proc] and [uplevel] Severity: Important
Priority: 5 Medium Last modified: 2026-09-01 13:26:24
Status: Closed Closed by: sebres
Resolution: Fixed Closed on: 2026-09-01 13:26:24
Version: 8.5+
Description:
Running this script causes tcl to die with a segmentation violation.  (This on MacOS from my own build)

proc probe {s} {
    info frame -1
    catch {apply {{s} {eval $s}} $s}
}
probe {catch {probe {error X}}; probe {}}
puts survived
User Comments:
oehhar added on 2026-08-26 12:16:36:

Good catch! Confirmed on 8.6.18 and 9.0.4 on MS-Windows.

Thanks, Harald


chw added on 2026-08-26 19:48:18:
Seems be present even in 8.5. This is my wildly guessed fix (diff against trunk):

---snip---
Index: generic/tclCmdIL.c
==================================================================
--- generic/tclCmdIL.c
+++ generic/tclCmdIL.c
@@ -1292,13 +1292,11 @@
     case TCL_LOCATION_BC: {
 	/*
 	 * Execution of bytecode. Talk to the BC engine to fill out the frame.
 	 */
 
-	CmdFrame *fPtr = (CmdFrame *)TclStackAlloc(interp, sizeof(CmdFrame));
-
-	*fPtr = *framePtr;
+	CmdFrame f = *framePtr, *fPtr = &f;
 
 	/*
 	 * Note:
 	 * Type BC => f.data.eval.path	  is not used.
 	 *	      f.data.tebc.codePtr is used instead.
@@ -1328,11 +1326,10 @@
 
 	ADD_PAIR("cmd", TclGetSourceFromFrame(fPtr, 0, NULL));
 	if (fPtr->cmdObj && framePtr->cmdObj == NULL) {
 	    needsFree = lc - 1;
 	}
-	TclStackFree(interp, fPtr);
 	break;
     }
 
     case TCL_LOCATION_SOURCE:
 	/*

---snip---

sebres added on 2026-08-26 20:13:19:

I don't think it'd change something. I'll take a look - my fork doesn't segfault almost since scratch (8.5).


sebres added on 2026-08-26 21:53:23:

I rebased it to 8.5 (initially based on that), but my original "fixed" variant was much more complex (and it is hardly possible to back-port)...

Branch bug-e533296a92d89044-segfault contains the simplest fix, but the question still remains - why is a such procPtr (with cmd after apply is done) in the call-frames stack yet at that time point?


apnadkarni added on 2026-08-27 03:31:55:

Sergey, I feel the fix proposed will solve this issue but there is a deeper, more serious problem as your question about procPtr implies. What I see is that the test script invokes TclInfoFrame (via InfoFrameCmd) three times. On the third invocation, debugger shows on entry to TclInfoFrame, the following values:

CmdFrame *  framePtr 0x0000020cb5b10ef8
CallFrame * framePtr->framePtr 0x0000020cb5b10db0
Proc *      framePtr->framePtr->procPtr 0x0000020cb5ce4b08
Command *   framePtr->framePtr->procPtr->cmdPtr 0x0000020cb5b110b0

Note the last value in particular. Then the line fptr = TclStackAlloc returns 0x0000020cb5b110a0. Since sizeof(*fptr) is 88, this allocation overlaps the memory referenced by framePtr->framePtr->procPtr->cmdPtr. Consequently, this line overwrites framePtr->framePtr->procPtr->cmdPtr->hPtr causing the crash. But in the meanwhile plenty of other fields are also corrupted, this is merely the one hitting a crash first.

It seems to me that some code path is releasing memory allocated by TclStackAlloc too early, resulting in that memory being overwritten while still in use. I don't understand the runtime frame management enough so you might be able to tell if that is the case and where it is happening much quicker than I could.


apnadkarni added on 2026-08-27 03:34:54:

BTW, I think Christian's suggestion will also only hide the issue since it avoids use of TclStackAlloc but does not answer why some previous TclStackAlloc seems to be premptively freed.


chw added on 2026-08-27 05:02:58:
Ashok, indeed, my first shot did pure error hiding.
Based on sebreas idea, a more gentle guess for 8.6 is here:

https://androwish.org/home/info/b22c637978b91e3a

sebres added on 2026-08-27 10:23:36:

I think setting hPtr = NULL before freeing memory is a dirty workaround, because it'd imply use after free (worse enough), as well as that the memory behind hPtr remains unchanged. It is bad idea.

As for Ashok's analyze, the mentioned overwrite looks like an after effect to me -the cmd pointer used in Tcl_ApplyObjCmd for procPtr is reserved in program stack and without the reset (leaving Tcl_ApplyObjCmd) a usage after free is possible there too. But as I said before, I also think that this would require further investigation. Unfortunately I cannot backport correct fix, because my TEBC uses completely different cmd-location approach (without scriptCLLocPtr and co) - the way how it gets "compiled" is simply different.


sebres added on 2026-08-27 12:42:38:

Well, independently to the question of procPtr of finalized lambda in frames stack, the situation where procPtr->cmdPtr may be NULL is normal and expected, see for example SetLambdaFromAny.

Thus the fix of me is basically correct (it simply restores the situation before the lambda invocation), so I'd merge it if no objections follow.


sebres added on 2026-08-27 14:17:31:

OK, found and fixed it ultimately: this going to nested invocation of lambda, where by return from child to parent call, the info of lambda that got overwritten within child stack gets broken (or in my previous fix - get lost due to reset). Amend [a7a8993cca8e49c9] avoids this reset of lambda info in frames stack by nested invocations (reserves this info as long as it will be evaluated), so fill this info and reset the artificial command only once (at first/parent invocation); test now adjusted to cover it (lambda shall be in the last frame-info too). This makes it also a bit faster by nested invocations of lambda.

However I'd admit that for performance reasons, one could rewrite this to initialize procPtr->cmdPtr only once by first invocation (as long as the lamda exists), but this would expect another clean-up and referring mechanism.
At least it'd explain why invocation of lambdas were always faster in my fork.


jan.nijtmans added on 2026-08-27 15:03:56:

I put chw's proposal in a [152c3d8cd63c99da|branch] (based on 8.6, since I don't care about 8.5 any more ...., and without setting a field to NULL before a ckfree). This way, the two solutions (sebres' and chw's) can be compared.

It appears to work for me. Can you (@sebres and @chw) come to an agreement which one is best?


sebres added on 2026-08-27 18:09:57:

I backported original idea in [4ca6eb5618ce2003] and better test to this branch (as new branch) which is NRE capable now, but... as already said...

  • I guess we can optimize the solution (no matter 8.5- or NRE-based)
  • the idea of preventive creation of artificial lambda information in order to provide ExtraFrameInfo (used by [info frame] only) is a bit strange, IMHO and such info (e. g. with lambdaPtr) can be provided on demand and retrieved inside [info frame];
  • I don't like the approach to share mem-block from TclStackAlloc across NRE-trampoline (if yieldto would change the NRE stack, e. g. I cannot exclude out of sequence errors by TclStackFree);

I'll try to provide better one (or backport some part from my fork).


sebres added on 2026-08-27 21:19:41:

Branch bug-e533296a92-sebres-optimized is currently pure experimental (even tests would probably fail), because there is no proper clean-up/reference model for lambda-commands to nsPtr (so they are not dereferenced and apply doesn't detect yet if namespace get deleted).

But the idea is very interesting in my opinion...

And because that unneeded interim alloc/free of `cmdPtr` is removed in `apply` now, the branch has a bit better performance:

    % timerate -calibrate {}
    % timerate { apply {{} {}} }
    - 0.223638 µs/# 3764760 # 4471519 #/sec 841.942 net-ms
    + 0.188577 µs/# 4337271 # 5302877 #/sec 817.909 net-ms


sebres added on 2026-08-27 22:09:32:

OK, OO is also affected by similar issue (old value of cmdPtr left in `procPtr->cmdPtr`), just it is not covered now in tests.


jan.nijtmans added on 2026-08-28 08:18:50:

Thanks for doing all this! My conclusion: let's not rush this in Tcl 9.1b1. Since it's a long-existing problem without anyone noticing it, it doesn't have 'High' priority IMHO.


apnadkarni added on 2026-08-28 10:07:35:
@sebres, @chw - seems reasonable to me at the level of my competency.

Agree with Jan that not worth risking putting in 9.1b1.

sebres added on 2026-08-31 21:44:27:

Thus fixed with mixed variant of me and Christian in 8.6 ([107a8d20dd3eac4a])...

The optimized variant is also ready and can be merged to trunk as suggested by Jan and Ashok... I have no time currently and there seems to be some bug in 9.x right now, see [4937c309f4ce3657], so I merged and adjusted 8.6 only to 9.0 (blind, so wait for the tests).

Thus close as fixed.
However as already said I still think the tclOO may be also affected by this issue (the artificial command is not restored after method invocation, so may fail by nested calls). @dkf, would you like to take a look at it?


jan.nijtmans added on 2026-09-01 07:20:48:

This happens with --enable-symbols builds now (seen on both Linux and MacOS):

regexp.test Test file error: tcltest: /home/runner/work/tcl/tcl/generic/tclProc.c:2759: ApplyNR2: Assertion `procPtr->cmdPtr == data[0]' failed.

So, I don't consider it fixed yet :-(


sebres added on 2026-09-01 09:55:41:

Hmm... I cannot reproduce it on 8.6, so it looks like something with merge to 9.0 (there were some conflicts) or the fix is not suitable for 9.x (I didn't tested it in 9.x, because of build-bug I mentioned before).


sebres added on 2026-09-01 10:07:46:

OK, it seems to be not suitable indeed (but I guess 8.6 may get it also some days and it is simply retarded for some reason) - test regexp-27.8 seems to destroy the lambda representation (shimmer to list due to lindex), so it is typical use after free, this time in NRE-callback... It'd be hardly solvable this way, so instead I tend to merge my optimized branch (which is not affected by that issue due to proper refcounting and clean-up in TclProcCleanupProc).


sebres added on 2026-09-01 11:05:51:

Yep, 8.6 was affected too (see new test apply-9.4)...
Shall be fixed now in 8.6 [e76affb84c9e89ae] and 9.0 [0449bde0a12fbad6].

By the way, can someone take a look to the 9.x bug [4937c309f4ce3657] (and fix build with disabled zipfs)?


apnadkarni added on 2026-09-01 11:45:31:

Sergey, GH actions show an assertion failure in the latest run with --enable-symbols. Looks like it might be related to your changes. Could you take a look?


sebres added on 2026-09-01 13:26:24:

Yes, it was exactly the error Jan meant previously in [20f94f9068ad1e72].

This must be fixed now by merge of my optimized branch, which was not affected by the issue... Any attempt to handle it with NRE (or directly in call-stack) are doomed to fail with more "use after free" scenarios, because shimmering could remove the lambda representation and cmdPtr stored there shall not be restored, unless we'd get another one in followed nested call, so one'd rather need proper refcounting for that).