| Ticket UUID: | 1501749 | |||
| Title: | Wish core dump with window-delete + index | |||
| Type: | Bug | Version: | None | |
| Submitter: | pcmacdon | Created on: | 2006-06-06 16:47:46 | |
| Subsystem: | 18. [text] | Assigned To: | fvogel | |
| Priority: | 7 High | Severity: | Minor | |
| Status: | Closed | Last Modified: | 2024-12-07 11:30:29 | |
| Resolution: | Fixed | Closed By: | fvogel | |
| Closed on: | 2024-12-07 11:30:29 | |||
| Description: |
Wish core dumps with the script below. To reproduce
run it and press <PgDn> then <Down> until the window
gets mapped, deleted and replaced with blue text.
Does not occur every time, but has been reproduced
under versions Tk 8.3, 8.4 and 8.5 and CVS HEAD.
Following is the gdb traceback of CVS HEAD.
**************************************
#0 0x400d7b8e in TkTextUpdateLineMetrics
(textPtr=0x80e8ca8, lineNum=103,
endLine=202, doThisMuch=256)
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../generic/tkTextDisp.c:3035
3035 if (TkBTreeLinePixelEpoch(textPtr,
linePtr)
(gdb) p linePtr->pixels
$3 = (int *) 0x1
(gdb) bt
#0 0x400d7b8e in TkTextUpdateLineMetrics
(textPtr=0x80e8ca8, lineNum=103,
endLine=202, doThisMuch=256)
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../generic/tkTextDisp.c:3035
#1 0x400d78fd in AsyncUpdateLineMetrics
(clientData=0x80e8ca8)
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../generic/tkTextDisp.c:2901
#2 0x401c3ac5 in TimerHandlerEventProc ()
from /home/pcmacdon/tmp/TclCvs/lib/libtcl8.5.so
#3 0x401a9a3a in Tcl_ServiceEvent ()
from /home/pcmacdon/tmp/TclCvs/lib/libtcl8.5.so
#4 0x401a9d3c in Tcl_DoOneEvent ()
from /home/pcmacdon/tmp/TclCvs/lib/libtcl8.5.so
#5 0x40042146 in Tk_MainLoop ()
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../generic/tkEvent.c:2150
#6 0x40053579 in Tk_MainEx (argc=-1, argv=0xbffff63c,
appInitProc=0x804887c <Tcl_AppInit>, interp=0x804b360)
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../generic/tkMain.c:308
#7 0x08048864 in main (argc=2, argv=0xbffff634)
at
/home/pcmacdon/tmp/TclCvs/tk/unix/../unix/tkAppInit.c:68
#8 0x42017499 in __libc_start_main () from
/lib/i686/libc.so.6
(gdb) l
3030
3031 /*
3032 * Now update the line's metrics if
necessary.
3033 */
3034
3035 if (TkBTreeLinePixelEpoch(textPtr,
linePtr)
3036 !=
textPtr->dInfoPtr->lineMetricUpdateEpoch) {
3037 if (doThisMuch == -1) {
3038 count += 8 *
TkTextUpdateOneLine(textPtr, linePtr,
0,
3039 NULL, 0);
***********************************************************
# Demonstrate TEXT using a window frame as placeholder for
# a section rendering. When window gets mapped, it's
# deleted and the section gets rendered.
set str [string repeat "The quick red fox jumped over
the lazy brown dog\n" 100]
pack [text .t] -fill both -expand y -side left
pack [scrollbar .sb -command ".t yview"] -fill y -side left
.t conf -yscrollc ".sb set"
.t insert end $str\n
.t window create end -window [frame .f1 -width 800
-height 800]
.t insert end \n$str
.t tag conf foo -background lightblue
proc MAPPED w {
puts "MAPPED"
set i [.t index $w]
set oi [.t index @0,0]
.t delete $w
destroy $w
.t insert $i [string toupper $::str] foo
.t mark set insert $oi
.t see $oi
}
bind .f1 <Map> "MAPPED %W"
focus .t
| |||
| User Comments: |
fvogel added on 2024-12-07 11:30:29:
The fix for this ticket had to be revisited for the revised text widget again, see [a67252b384]. fvogel added on 2022-02-14 08:10:26: For the records, a different fix to prevent a crash in textWind-18.2 was needed on macOS with the revised_text branch. See [14752d4b87]. fvogel added on 2022-01-31 20:46:01: Github actions shows there is no crash anymore, and textWind-18.[12] pass on all platforms with all compilers tested a CI. Branch bug-1501749fff is now merged into core-8-6-branch and trunk. fvogel added on 2022-01-30 16:40:55: Using valgrind on Linux, even if I do not reproduce the crash I could find what the problem is with both textWind-18.1 and textWind-18.2. I have fixed both issues, by committing respectively [19d36803] and [9e98a6d7]. Let's see the outcome at GitHub actions. fvogel added on 2022-01-27 21:58:37: Test textWind-18.1 crashes at GitHub actions:
I'm not reproducing the latter, and I can't test the former. Both textWind-18.1 and textWind-18.2 pass in all other configurations tested at GitHub (except gcc(no) on Win and clang(no, --disable-aqua) on macOS, which were untested/cancelled). In particular, the tests pass on macOS with clang(no, --enable-aqua). fvogel added on 2022-01-23 15:19:46: I have now committed [157d5a2a] to fix the crash happening with my first script below. The associated regression test, textWind-18.1, now passes. Also the original script from the OP does no longer crash. fvogel added on 2022-01-23 09:45:22: I have committed [511cf9cc] to fix the crash happening with my second script below. Moreover I have added a new test, textWind-18.2, to check this fix. This new test crashes before the fix and passes with the fix as expected. Now, let's look at the first script below (which is aka textWind-18.1 in the test suite). fvogel added on 2022-01-08 23:30:30: While the proposed fix [83d63659] indeed solves the crash when doing:
pack req Tk
pack [text .t]
for {set i 1} {$i < 100} {incr i} {.t insert end "Line $i\n"}
.t window create end -window [frame .f -background red -width 80 -height 80]
bind .f <Map> {.t delete .f}
after 100 {.t yview end}
It does not solve other similar crashes such as:
pack req Tk
pack [text .t]
for {set i 1} {$i < 100} {incr i} {.t insert end "Line $i\n"}
.t window create end -window [frame .f -background red -width 80 -height 80]
bind .f <Map> {destroy .t}
after 100 {.t yview end}
My fix [83d63659] was not correct. It does not properly handle possible removal (undisplay or deletion or destruction) of the embedded window nor of the whole text widget under our feet upon a <Map> event on an embedded window. I'm backing out this erroneous fix in the bugfix branch in [871384dd], and I'm merging this backout in core-8-6-branch and trunk. The non-regression test case textWind-18.1 associated to the present ticket is kept but marked with the 'knownBug' constraint. fvogel added on 2015-09-29 19:32:38: Fix merged to core-7-5-branch and trunk. Thanks to all who helped. Closing now. oehhar added on 2015-09-24 07:01:51: This bug hunting looks like a masterpiece. Cudos to Alexandre et Francois ! Harald fvogel added on 2015-09-23 19:49:07: Fix proposal (with new test) pushed in branch bug-1501749fff fvogel added on 2015-09-22 21:52:06: Or just this patch fixes it:
Index: generic/tkTextWind.c
==================================================================
--- generic/tkTextWind.c
+++ generic/tkTextWind.c
@@ -1121,10 +1121,16 @@
EmbWinBboxProc(textPtr, chunkPtr, 0, screenY, lineHeight, baseline,
&lineX, &windowY, &width, &height);
windowX = lineX - chunkPtr->x + x;
+ /*
+ * Mark the window as displayed so that it won't get unmapped.
+ */
+
+ client->displayed = 1;
+
if (textPtr->tkwin == Tk_Parent(tkwin)) {
if ((windowX != Tk_X(tkwin)) || (windowY != Tk_Y(tkwin))
|| (Tk_ReqWidth(tkwin) != Tk_Width(tkwin))
|| (height != Tk_Height(tkwin))) {
Tk_MoveResizeWindow(tkwin, windowX, windowY, width, height);
@@ -1132,16 +1138,10 @@
Tk_MapWindow(tkwin);
} else {
Tk_MaintainGeometry(tkwin, textPtr->tkwin, windowX, windowY,
width, height);
}
-
- /*
- * Mark the window as displayed so that it won't get unmapped.
- */
-
- client->displayed = 1;
}
♀
/*
*--------------------------------------------------------------
*
fvogel added on 2015-09-22 21:31:24: For some reason I can now reproduce every time, with the watchpoint being hit each time (even without disabling randomization in gdb). I must have watched the wrong address previously, or done something otherwise incorrect.
The problem with the approach using a watchpoint is that it will only trigger when the contents at the given address gets _changed_. When this happens, the mechanism which freed the TimerHandler structure already vanished in the mist.
To understand what is happening, I need is to know when the TimerHandler structure is freed, and what code performs this free.
A valgrind output is interesting:
% .t yview end
% ==1937== Invalid write of size 4
==1937== at 0x40F727A: TkTextEmbWinDisplayProc (tkTextWind.c:1142)
==1937== by 0x40E608B: DisplayDLine (tkTextDisp.c:2460)
==1937== by 0x40E8BBA: DisplayText (tkTextDisp.c:4270)
==1937== by 0x424E6F6: TclServiceIdle (tclTimer.c:736)
==1937== by 0x422B7E8: Tcl_DoOneEvent (tclNotify.c:987)
==1937== by 0x40557CA: Tk_MainLoop (tkEvent.c:2139)
==1937== by 0x40665F1: Tk_MainEx (tkMain.c:351)
==1937== by 0x804880D: main (tkAppInit.c:66)
==1937== Address 0x5230444 is 12 bytes inside a block of size 24 free'd
==1937== at 0x4023B6A: free (vg_replace_malloc.c:366)
==1937== by 0x418B63B: TclpFree (tclAlloc.c:723)
==1937== by 0x41973FC: Tcl_Free (tclCkalloc.c:1221)
==1937== by 0x40F68DC: TkTextWinFreeClient (tkTextWind.c:710)
==1937== by 0x40F6955: EmbWinDeleteProc (tkTextWind.c:752)
==1937== by 0x40E0156: TkBTreeDeleteIndexRange (tkTextBTree.c:1453)
==1937== by 0x40DA210: DeleteIndexRange (tkText.c:3163)
==1937== by 0x40D6935: TextWidgetObjCmd (tkText.c:1088)
==1937== by 0x418FA95: TclEvalObjvInternal (tclBasic.c:3727)
==1937== by 0x4190C3C: TclEvalEx (tclBasic.c:4426)
==1937== by 0x4190210: Tcl_EvalEx (tclBasic.c:4083)
==1937== by 0x4043909: Tk_BindEvent (tkBind.c:1769)
==1937==
The first part ("Invalid write...") we already knew: TkTextEmbWinDisplayProc (tkTextWind.c:1142) writes in an area that was freed previously. The offending line reads client->displayed = 1;
But the second part is new and interestingly tells where the block was freed.
During execution of the script bound to <Map> (i.e. '.t delete .f'), all clients of the embedded window are freed by EmbWinDeleteProc calling TkTextWinFreeClient for each of them, and the embedded window itself gets freed.
So when trying to display this embedded window (TkTextEmbWinDisplayProc), it just does no longer exist.
The fix should then be, in DisplayDLine, to refrain from calling TkTextEmbWinDisplayProc on a non-existing embedded window, or, equivalently, to bail out early in TkTextEmbWinDisplayProc in that situation.
However, I note that just commenting out the line triggering invalid write i.e.
// client->displayed = 1;
does not seem to have any adverse effect:
- there is no crash anymore
- valgrind no longer complies about any issue
- the test suite does not report new failures
ferrieux added on 2015-09-22 08:36:02: OK, so maybe evidence so far can be explained as follows: - the script you crafted triggers a "root corruption" deterministically - this "root corruption" non-deterministically percolates in the RAM due to the details of allocation addresses; - the propagation sometimes ends up dumping core just after setting "our" clientData to 1, sometimes elsewhere. - that's why the watchpoint sometimes fires, sometimes doesn't If this interpretation is consistent with all your observations, you can then simply focus on the "better known path": redo the test until you hit the watch point. At that time, climb back (up the stack) to the "root corruption", and turn it into an assert as suggested before. fvogel added on 2015-09-21 21:32:57: Hmm, another try shows a watchpoint that is being hit:
284 timerHandlerPtr = (TimerHandler *) ckalloc(sizeof(TimerHandler));
(gdb)
290 memcpy((void *)&timerHandlerPtr->time, (void *)timePtr, sizeof(Tcl_Time));
(gdb)
291 timerHandlerPtr->proc = proc;
(gdb)
292 timerHandlerPtr->clientData = clientData;
(gdb)
293 tsdPtr->lastTimerId++;
(gdb) p timerHandlerPtr->clientData
$3 = (void *) 0xa006de0
(gdb) p &timerHandlerPtr->clientData
$4 = (void **) 0xa05f0c4
(gdb) watch *(int *)0xa05f0c4
Hardware watchpoint 3: *(int *)0xa05f0c4
(gdb) c
Continuing.
Hardware watchpoint 3: *(int *)0xa05f0c4
Old value = 167800288
New value = 1
0xb76c1281 in TkTextEmbWinDisplayProc (textPtr=0xa006de0, chunkPtr=0xa060650, x=3, y=0, lineHeight=80, baseline=43, display=0x9f41328, dst=67108885,
screenY=82) at /home/francois/Documents/tcltk/fossil/tk/unix/../generic/tkTextWind.c:1142
1142 client->displayed = 1;
(gdb)
As far as I can say, I see the same thing as on Windows with Visual.
fvogel added on 2015-09-21 21:13:04: It crashes, clientData corrupted, watchpoint missed. ferrieux added on 2015-09-21 20:43:04: Thanks for the switch. But do you mean that it still crashes, clientData corrupted but watchpoint missed, OR that it no longer crashes ? fvogel added on 2015-09-21 20:08:55: I have followed your advice and did as you recommended below on 2015-09-17 21:25:37 I have used a Linux Debian 6 I have at hand (in a virtual machine), that I had installed previously with debug tools, fossil, and very recent Tcl/tk source code. The watchpoint does not trigger for me. If you have any better luck trying on your side, I would be very interested in where I messed this up... ferrieux added on 2015-09-21 18:52:47: So, you're saying that (1) Tcl_CreateTimerHandler is passed a clean clientData, (2) you set a watchpoint on the clientData field of the timer structure, and (3) the timer callback is later called with a corrupted clientData without the watchpoint ever kicking in before ? Time to switch to unix or mingw and use a real debugger I guess :) fvogel added on 2015-09-21 17:09:04: Well, no. Actually my first statements were wrong. TkTextEmbWinDisplayProc does not get a corrupted chunkPtr. Instead, chunkPtr is correct until Tk_MaintainGeometry returns (inside TkTextEmbWinDisplayProc). That call maps the embedded window, which triggers the binding on <Map>, which deletes the embedded window and frees the corresponding structures (chunks, dislpay line, and embedded window itself). On return from the call to MaintainGeometry, everything related to this embedded window and the dispay line on which it was (supposed to be) displayed is freed. Well, why not. It's deleted after all. So far I do not see why the TimerHandler structure gets corrupted. I still did not find how and when this happens. For sure, when AsyncUpdateLineMetrics gets these data it is corrupted and causes the crash. And, yes, reading the man page for Tcl_Preserve/Release, I think these are called in DisplayText precisely to handle the case we have here, and I don't think this mechanism is sufficient regarding asynchronous updates (basically: Tcl_Release is called before AsyncUpdateLineMetrics is reached). ferrieux added on 2015-09-21 16:06:07: I have nothing to say about the big picture, but I can answer your last question: "how the clientData saved in the TimerHandler structure becomes wrong", since you answered it yourself earlier:
- TkTextEmbWinDisplayProc gets a corrupted chunkPtr
- off this chunkPtr is made (by indirection) a corrupted ewPtr = chunkPtr->clientData
- off this ewPtr is made a corrupted client = EmbWinGetClient(textPtr, ewPtr);
-off this corrupted client
client->displayed = 1;
writes a random (long)1 in a random word of memory, which happens to land on the timer structure's clientData field.
So we're only left to explain the first corruption of the chain; from your description I understand it might be linked with an early free of an internal structure , and that you're surprised that Tcl_Preserve didn't preserve it enough. Is this summary accurate ?
fvogel added on 2015-09-20 19:07:31: I tried asserting as you suggested, and this made me realize my findings were not fully correct.
Look at this call stack:
FreeDLines
TextChanged
TkTextChanged
DeleteIndexRange
TextWidgetObjCmd
TclEvalObjvInternal
TclEvalEx
Tcl_EvalEx
Tk_BindEvent
TkBindEventProc
Tk_HandleEvent
Tk_MapWindow
Tk_MaintainGeometry
TkTextEmbWinDisplayProc
DisplayDLine
DisplayText
When the text widget is scrolled (on "yview end"), DisplayText is launched, as an idle callback, which leads to DisplayDLine and TkTextEmbWinDisplayProc.
DisplayText and DisplayDLine seem correct to me.
DisplayDLine calls TkTextEmbWinDisplayProc when it encounters an embedded window.
In TkTextEmbWinDisplayProc, chunkPtr is correct until Tk_MaintainGeometry is called.
Tk_MaintainGeometry calls Tk_MapWindow, which runs the script bound to the <Map> event (".t delete .f"), which in turn launches deletion of the embedded window by calling DeleteIndexRange.
In FreeDLines, the display line pointer is unlinked from the other display lines, and both this display line pointer and the chunks associated to it are _ckfreed_.
Then, back up in DeleteIndexRange, TkBTreeDeleteIndexRange is called, which calls TkTextInvalidateLineMetrics, which schedules
Tcl_CreateTimerHandler(1, AsyncUpdateLineMetrics, (ClientData) textPtr);
Then, going on climbing the call stack back up, when returning to TkTextEmbWinDisplayProc, the chunks corresponding to the embedded window just deleted do no longer exist and the display line pointer neither. At this specific stage the fact that they were freed can be seen (by checking chunkPtr after the call to Tk_MaintainGeometry, or by monitoring the memory through watch points) but this is not yet a problem I think.
Problem arises when trying later to access the corrupted textPtr obtained from the saved clientData in AsyncUpdateLineMetrics.
I still don't get the full picture of the problem. I don't see how the clientData saved in the TimerHandler structure becomes wrong. I have also noticed the calls to Tcl_Preserve/Tcl_Release in DisplayText, which should be here specifically to prevent problems like the one we're facing in this ticket (but is this mechanism still working with asynchronous updates...?).
ferrieux added on 2015-09-19 08:03:36: So it lokks like you nailed it: "in DisplayText the loop calls DisplayDLine(textPtr, dlPtr, prevPtr, pixmap); on each display line but prevPtr is NOT the line preceding dlPtr. Instead, prevPtr is the last line of the widget." You might insert an assert at this very spot (before the call to DisplayDLine), checking exactly for that (assuming you can turn this criterion into actual code -- you're the expert here, I'm not at all familiar with the text widget's data structures). Then you might discover a wider range of offending contexts where the assertion does not hold either (I hope, not too wide...). This should lead to the root cause. fvogel added on 2015-09-18 21:48:41: Thanks to both for your valuable help. I appreciate this very much.
Progress report now from my side:
1. I have managed to make the debugger break when the concerned address changes contents. I had to manually force
OPTIMIZATIONS = -Od
directly in rules.vc to obtain a build with no optimization. This is a bit strange since I always build with symbols, and in makefile.vc it is stated that:
# symbols = Debug build. Links to the debug C runtime, disables
# optimizations and creates pdb symbols files.
This is not really true, I still get optimizations -Ot and -Oi when building with OPTS=symbols. Maybe worth to be noted.
2. Actually, instructing Visual Studio to set a data breakpoint (a watchpoint) on &timerHandlerPtr->clientData (when within TclCreateAbsoluteTimerHandler)works nicely even with the default build with symbols (i.e. with -Ot and -Oi optimizations).
I must have made some mistake the first times I tried, sorry for the false alarm.
3. I have trimmed down the test case as follows:
package require Tk
catch {destroy .t .f}
pack [text .t]
for {set i 1} {$i < 50} {incr i} {.t insert end "Line $i\n"}
.t window create end -window [frame .f -background red -width 80 -height 80]
bind .f <Map> {.t delete .f}
after 1000 {.t yview end}
Paste this in tclsh85g.exe: for me this produces a fully repeatable crash with the current core-8-5-branch.
Note that the original report states that the crash is not systematic, but for me it is (or it has such a high occurrence that it is as if).
4. Trying on Linux I did not yet, since I can trigger the issue on Windows and I'm more familiar with Windows than with Linux. Once fixed, I will make the above test case into a proper test of the test suite, and run this on both Windows and Linux.
5. When the watchpoint triggers, it shows corruption in TkTextEmbWinDisplayProc.
The 0x00000001 I _always_ get for textPtr when entering AsyncUpdateLineMetrics comes from
client->displayed = 1;
at the very end of TkTextEmbWinDisplayProc
The call stack at this point is:
TkTextEmbWinDisplayProc
DisplayDLine
DisplayText
TclServiceIdle
Tcl_DoOneEvent
It turns out that:
- chunkPtr received by TkTextEmbWinDisplayProc is corrupted
- in DisplayDLine dlPtr->chunkPtr (the first chunk of the display line) is also corrupted
- the display line pointer dlPtr itself received by DisplayDLine is corrupted
- in DisplayText the loop calls DisplayDLine(textPtr, dlPtr, prevPtr, pixmap); on each display line but prevPtr is NOT the line preceding dlPtr. Instead, prevPtr is the last line of the widget.
ferrieux added on 2015-09-18 07:31:59: If Harald's suggestion doesn't do it, you might want to switch to unix (or to mingw if you cannot leave Windows). Since the bug was reported on unix, I guess it is "portable" ;-) Note also that optimizations do lower the frequency of true memory accesses, resorting to registers here and there, but here we're facing a situation where the contents of memory get scrambled, so at least you should at least catch the point where the (supposedly scrambled) register is written back to memory. If not, the debugger is to blame. oehhar added on 2015-09-18 06:54:53: Francois, it is perhaps inadequate, but the "stop on modification" never worked for me with Visual C. I knew I changed somethin and it never stopped. I always supposed, this are optimisations I can not stop. Take care to compile without any optimisation, so if the debugger puts a trap on a memory address, it really fires. Anyway, thanks for caring, Harald fvogel added on 2015-09-18 06:31:57: Thanks for the tip, I understand the point. I put the recipe into practice, except that I did it on Windows (same kind of things exist on Visual C++ Express Edition). Creation of the TimerHandler structure happens in TclCreateAbsoluteTimerHandler, and there timerHandlerPtr->clientData gets the correct value (for instance: 0x02ec9710). Then, I set a watchpoint (a "data breakoint" in Visual) on *(int *)0x02ec9710, instructing the debugger to break when the memory pointed by the contents of 0x02ec9710 changes. I also add a watchpoint on 0x02ec9710 directly. When resuming execution, none of these two watchpoints kicks in. ferrieux added on 2015-09-17 21:25:37: To answer this question, you may step into Tcl_CreateTimerHandler up to the point where it copies the clientData into a field of a freshly ckalloc()ed TimerHandler structure. Then use gdb's "watch" command to set a permanent watchpoint on that field's address: (1) use "print" to get the fieldd's address in (hex); (2) watch *(int *)0x12345678 (replace with actual value). Then use "continue". When the watchpoint kicks in, you'll be in the seat of the offending code. fvogel added on 2015-09-17 20:49:59: Upon entering AsyncUpdateLineMetrics, right at the beginning:
register TkText *textPtr = (TkText *) clientData;
clientData is 0x00000001, which means that textPtr receives 0x00000001. That is wrong.
This AsyncUpdateLineMetrics was scheduled in TextInvalidateLineMetrics by:
Tcl_CreateTimerHandler(1,AsyncUpdateLineMetrics, (ClientData) textPtr);
At this point, textPtr still has a valid value pointing to a text widget record.
What on earth can possibly happen in between...?
fvogel added on 2015-09-15 20:42:18: Reproduced on current core-8-5-branch. It crashes in the asynchronous callback for recalculation of line metrics (AsyncUpdateLineMetrics) that is launched after the deletion of frame .f1 I will have a look. pcmacdon added on 2006-06-06 23:56:33: File Added - 180758: tw2.tcl Logged In: YES user_id=190660 Script attached below. | |||
Home
Timeline
Branches
Tags
Forum
Tickets
Download
Wiki