Tk Source Code

View Ticket
Login
Ticket UUID: 8bd39bede9e017bf515264c6f6cbd2205a40ca8c
Title: Word wrap regressions since [e3ef73ae0b] (mojibake line breaks)
Type: Bug Version: trunk
Submitter: nab Created on: 2026-10-01 20:30:10
Subsystem: 18. [text] Assigned To: nobody
Priority: 5 Medium Severity: Important
Status: Closed Last Modified: 2026-10-03 13:25:51
Resolution: Fixed Closed By: nab
    Closed on: 2026-10-03 13:25:51
Description:

Since the merge of tk_text_grapheme in e3ef73ae0b, -wrap word of the text widget:

  1. cuts a word at a tag boundary inside it;
  2. loses the break after white space followed by a tab (UAX #14 has no break before a tab), so the text after the tabs runs past the right margin;
  3. computes the breaks over the whole rest of the segment for each chunk, and mojibake_grapheme_prev() rescans from the start of the buffer at each step, which is quadratic on long segments.

Cases 1 and 2 come from the backward search in TkTextCharLayoutProc: the analysed buffer ends at the end of the segment (a tag boundary), and mojibake_line_breaks() always marks a break at both ends of its buffer. So the end of a fragment is taken as a break, and for a chunk holding a single tab, the artificial break at position 0 is found before the real one at position 1, then rejected.

The tests textDisp-2.10, 2.23 and 26.14.2 caught this; their expected results were rewritten in e3ef73ae0b (textDisp-2.23 now expects a negative spacing, 1 5 -2 1).

Script (case 1):

pack [text .t -font {Courier -12} -width 20 -wrap word]
.t insert end "Short line\nLine 2 is long enough to wrap around a couple of times"
.t tag add y 2.19 end
update
set res {}
for {set i 2.0} {[.t compare $i < 2.end]} {set i [.t index "$i + 1 displaylines"]} {
    lappend res $i
}
set res
  • 9.1.0: 2.0 2.15 2.30 2.49
  • trunk 2d659391fc: 2.0 2.19 2.39 ("Line 2 is long enou" | "gh to wrap around a" ...)

Case 2: the body of textDisp-26.14.2 as in 9.1.0 gives 112 168 112 168 on trunk, instead of 112 56 112 56.

Proposed fix: branch nab-wrap-locale, check-in 3be9cd51f5:

  • WordBreakIndex computes the breaks with the text which follows the chunk, in its segment and in the next character segments, in a bounded window (128 bytes after the fitting part, more than the longest dictionary word);
  • the end of the segment is a break only if no text follows (end of line, image, window), as in 9.1.0;
  • breaks after ASCII white space, as in 9.1.0, besides the UAX #14 breaks which are also word boundaries;
  • searches from right to left without rescanning.

The breaks wanted by e3ef73ae0b are kept: Thai, Lao and Khmer still wrap at their words, and the other UAX #14 breaks still apply.

textDisp-2.10, 2.23, 19.11.23, 26.14.2 and 27.1 are restored as in 9.1.0. The full textDisp.test of 9.1.0 passes with this check-in (400 tests, 0 failure; 3 failures on trunk before it), and the text test family of trunk passes (1786, 0).

With the breaks after tabs restored, textDisp-26.14.2 wraps between two tabs again, the case of 7353d930de: its fix in branch text-tab-chunk-free b5684e005a is independent and still needed. Both together pass the textDisp.test of 9.1.0 and the text test family of trunk.

Checked on macOS (aqua) only. textDisp-19.11.23 and 27.1 already passed on macOS with their 9.1.0 results: if they were changed because of Windows or X11, please check them there.

User Comments: nab added on 2026-10-03 13:25:51:

Merged into trunk in [9bc623c8ed]. No backport is needed: the regression only exists in trunk.


serhiy.storchaka added on 2026-10-03 12:48:20:

Merged [text-tab-chunk-free]. Please merge nab-wrap-locale.


jan.nijtmans added on 2026-10-03 09:45:51:

Great! +1


nab added on 2026-10-03 09:30:43:

Yes, I think nab-wrap-locale is ready to be merged. Please merge [text-tab-chunk-free] with it: the branch restores the break between two tabs, which triggers [7353d930de]. Tested together with current trunk on macOS: the text tests and the textDisp.test of 9.1.0 pass.


kevin_walzer added on 2026-10-02 19:21:03:
Nicolas, are you at a point where you believe your branches that address this ticket can be merged?

serhiy.storchaka added on 2026-10-02 06:56:19:

Tested on X11 (with and without bidi, and without Xft) and on Windows 11: the example gives the same result as 9.1.0, and the textDisp.test of both the branch and 9.1.0 passes. The other text tests pass too, except textImage-4.3 without Xft and textImage-4.2 on Windows, which also fail on trunk.

Builds without bidi warn about the unused variable bTotal.

[text-tab-chunk-free] is still needed: LayoutDLine still frees the chunk of the last tab during textDisp.test.


kevin_walzer added on 2026-10-01 20:42:09:
This branch addresses the wrap issue documented in the sample script, and script breaks look unchanged on macOS and X11. I do not have access to my Windows system at this time.