|
2026-11-05
| ||
| 20:25 | • New ticket [eb6a8f11] X11: the insertion cursor between lam and alef changes the drawing of the Arabic text (textDisp-42.7 fails). artifact: f7184f0d user: serhiy.storchaka | |
|
2026-11-02
| ||
| 16:31 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Closed with 5 other changes artifact: 6995bb65 user: Arnault01 | |
| 15:30 | • Ticket [59d6d5f6]: 3 changes artifact: 8e110ebd user: nab | |
| 13:06 | • Ticket [59d6d5f6]: 3 changes artifact: bdbe19ac user: serhiy.storchaka | |
| 13:05 | Text widget: the highlight of a part of a joined right-to-left word (e.g. one Arabic letter selected) did not match its bbox (see [59d6d5f6c0]): the chunks of a stretch were measured while the line was laid out, with the string cut at the end of each chunk, so the last letter of a chunk took its final form. Place the chunks of a right-to-left string by the advances in the whole string, as CharBboxProc() measures them. Test textDisp-37.4: moving the insertion cursor does not change the bboxes (it also covers a space between Hebrew words on Windows). leaf check-in: 4cb0789f user: serhiy.storchaka tags: text-rtl-highlight | |
|
2026-11-01
| ||
| 16:04 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Closed with 5 other changes artifact: f2e26889 user: kevin_walzer | |
| 12:04 | Fix text shimmering while a selection is dragged on macOS (see [59d6d5f6c0]): since [3403347da9] every mark, the insertion cursor included, ended the base-char stretch, so the text on each side of the cursor was shaped and placed apart (glyphs moved by up to a pixel on macOS, Arabic lost its joining everywhere). Only embedded windows and images end the stretch now, FinalizeBaseChunk skips zero-width chunks again, as in 9.0, and the block cursor draws the text under it in context. Tests textDisp-42.3 to 42.8. check-in: 8a2c92b3 user: nab tags: nab-text-mark-stretch | |
| 04:51 | • Add attachment bididemo-click.tcl to ticket [59d6d5f6] artifact: 85adcd0b user: Arnault01 | |
| 04:51 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Closed with 4 other changes artifact: 9356cdb6 user: Arnault01 | |
| 04:38 | • Ticket [59d6d5f6]: 5 changes artifact: 58600f55 user: Arnault01 | |
|
2026-10-31
| ||
| 18:03 | • Closed ticket [59d6d5f6]. artifact: cb9ae77a user: kevin_walzer | |
| 17:51 | Fix cursor insertion bug in RTL text and selection bug in mixed-bidi text [59d6d5f6c0]; restore broken RTL text display on Windows; implement ::tk::startOfCluster/endOfCluster over Uniscribe API on Windows so it is always available check-in: 3403347d user: kevin_walzer tags: trunk, main | |
| 15:05 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: fbc41f4a user: kevin_walzer | |
| 15:05 | • Add attachment Screenshot 2026-10-31 110309.png to ticket [59d6d5f6] artifact: 746c2512 user: kevin_walzer | |
| 14:59 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: dc5e0fce user: kevin_walzer | |
| 13:51 | • Ticket [59d6d5f6]: 3 changes artifact: 9cce42d3 user: serhiy.storchaka | |
| 13:50 | Fix the insertion cursor at the start of a line of right-to-left text (see [59d6d5f6c0]): it was drawn at the left end of the line. ReorderBidiChunks() now also moves the zero-width chunks before a stretch. closed check-in: a4124a72 user: serhiy.storchaka tags: text-bidi-chunk-order | |
| 12:30 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 3fda89c6 user: kevin_walzer | |
| 04:51 | • Ticket [59d6d5f6]: 3 changes artifact: c293a695 user: anonymous | |
| 04:49 | • Add attachment orphan-nikkud-chunkbidi.patch to ticket [59d6d5f6] artifact: 5052032b user: anonymous | |
| 04:40 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 7e74dd01 user: anonymous | |
| 04:35 | • Add attachment ltr-chunk-isrtl.patch to ticket [59d6d5f6] artifact: 25322fda user: anonymous | |
| 04:19 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 88a43e32 user: anonymous | |
|
2026-10-30
| ||
| 21:51 | • Ticket [59d6d5f6]: 3 changes artifact: 2e5feb66 user: kevin_walzer | |
| 15:14 | • Ticket [59d6d5f6]: 3 changes artifact: edbd2554 user: serhiy.storchaka | |
| 15:12 | Fix [f5b47c533f] (see [59d6d5f6c0]): a tag (e.g. the selection) splitting right-to-left text in a text widget made the line scrambled: the font back-end draws a stretch of text in visual order, but its chunks were placed from left to right in logical order, so the bboxes, the selection highlight and the mouse did not match the text and moved whenever the tags changed. Place the chunks of each stretch in visual order (rule L2 of UAX #9), keep the drawing origin of the stretch and measure from the logical positions, and walk the chunks in visual order for the background and @x,y. tk::TextClosestGap: the gap after a right-to-left character is on its left. Tests textDisp-37.4 to 37.6. check-in: 0e34e33c user: serhiy.storchaka tags: text-bidi-chunk-order | |
| 11:48 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 5b7888c9 user: kevin_walzer | |
| 11:34 | • Ticket [59d6d5f6]: 3 changes artifact: 2aeaa41d user: kevin_walzer | |
| 08:51 | • Ticket [59d6d5f6]: 3 changes artifact: 04347d8f user: serhiy.storchaka | |
| 04:25 | • Ticket [59d6d5f6]: 3 changes artifact: 6aacd944 user: Arnault01 | |
| 04:17 | • Add attachment dancing-mixed-fix.patch to ticket [59d6d5f6] artifact: 92bf76ad user: Arnault01 | |
| 04:16 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: f805ee31 user: Arnault01 | |
| 03:49 | • Ticket [59d6d5f6]: 3 changes artifact: 4559430e user: Arnault01 | |
| 03:20 | • Open ticket [59d6d5f6]. artifact: febe54a3 user: kevin_walzer | |
| 01:45 | • Ticket [59d6d5f6]: 4 changes artifact: 47f55870 user: Arnault01 | |
| 01:44 | • Closed ticket [59d6d5f6]. artifact: ee158333 user: Arnault01 | |
| 00:05 | • Ticket [59d6d5f6]: 3 changes artifact: ba8f24d9 user: kevin_walzer | |
|
2026-10-29
| ||
| 20:20 | • Ticket [59d6d5f6]: 3 changes artifact: 96ba6ca2 user: kevin_walzer | |
|
2026-10-28
| ||
| 19:21 | • Ticket [59d6d5f6]: 3 changes artifact: 87d4b036 user: kevin_walzer | |
| 19:20 | • Ticket [59d6d5f6]: 3 changes artifact: f6f0b621 user: kevin_walzer | |
| 19:02 | Patches for shimmering bidi text in [59d6d5f6c0] check-in: d681110d user: kevin_walzer tags: dancing_bidi | |
| 04:00 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 2260b173 user: kevin_walzer | |
| 03:07 | • Add attachment text-cluster-snap.patch to ticket [59d6d5f6] artifact: 7d85c4e7 user: Arnault01 | |
| 03:07 | • Add attachment tk-windows-uniscribe-measure.patch to ticket [59d6d5f6] artifact: a881bb1c user: Arnault01 | |
| 03:07 | • Add attachment tkTextDisp-bidi-reorder.patch to ticket [59d6d5f6] artifact: 4d969f56 user: Arnault01 | |
| 03:06 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: fd55b2ab user: Arnault01 | |
|
2026-10-17
| ||
| 01:21 | • Ticket [59d6d5f6]: 3 changes artifact: f0798241 user: Arnault01 | |
| 01:21 | • Ticket [59d6d5f6]: 3 changes artifact: 2b15dd94 user: Arnault01 | |
| 01:20 | • Add attachment patch2.txt to ticket [59d6d5f6] artifact: 63fdeaab user: Arnault01 | |
| 01:00 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: b96eb5eb user: Arnault01 | |
|
2026-10-16
| ||
| 08:17 | • Ticket [59d6d5f6]: 3 changes artifact: 3d22f29b user: serhiy.storchaka | |
| 08:16 | • Ticket [59d6d5f6]: 3 changes artifact: e2556d7d user: anonymous | |
| 07:09 | • Add attachment patch.txt to ticket [59d6d5f6] artifact: 52052b34 user: anonymous | |
|
2026-10-15
| ||
| 18:41 | • Ticket [59d6d5f6] Proposed fix for dancing selection in mixed RTL display status still Open with 3 other changes artifact: 7656b65e user: kevin_walzer | |
|
2026-10-14
| ||
| 06:03 | • Ticket [59d6d5f6]: 3 changes artifact: f0ea75fc user: Arnault01 | |
| 05:59 | • Ticket [59d6d5f6]: 2 changes artifact: 4f33a1d4 user: Arnault01 | |
| 03:14 | • Ticket [59d6d5f6]: 3 changes artifact: d2c53589 user: Arnault01 | |
| 03:13 | • New ticket [59d6d5f6]. artifact: c38795e7 user: Arnault01 | |
| Ticket UUID: | 59d6d5f6c049c611f1528af0c4c989e6c48e5d15 | |||
| Title: | Proposed fix for dancing selection in mixed RTL display | |||
| Type: | Bug | Version: | ||
| Submitter: | Arnault01 | Created on: | 2026-10-14 03:13:22 | |
| Subsystem: | 18. [text] | Assigned To: | kevin_walzer | |
| Priority: | 5 Medium | Severity: | Severe | |
| Status: | Closed | Last Modified: | 2026-11-02 16:31:59 | |
| Resolution: | Fixed | Closed By: | Arnault01 | |
| Closed on: | 2026-11-02 16:31:59 | |||
| Description: |
Summary ------- When selecting text with the mouse in a Text widget displaying right-to-left (RTL) text (e.g., Hebrew or Arabic), the highlighted selection behaves erratically: while dragging, the highlight "dances"/flickers, extending and shrinking non-deterministically instead of growing steadily from the anchor point. On mouse release, the final selection does not always match the visual range that was highlighted during the drag. Environment ----------- - Tk version: 9.1 (also reproducible on 8.6) - Platform: Linux / X11 but also Windows. - Observed via Python's tkinter binding, but the issue lives in the Tk Text widget itself and reproduces with pure Tcl/Tk. Steps to reproduce ------------------ 1. Create a Text widget and insert a paragraph of RTL text (e.g., Hebrew: "שלום עליכם, זהו טקסט מימין לשמאל עם מספר מילים.") 2. Click with the left mouse button at one end of the paragraph and drag slowly across the text. Analysis -------- This appears to be related to how the Text widget computes selection boundaries from pointer coordinates in bidirectional text: the mapping between display (visual) columns and byte/character indices is ambiguous near bidi runs, and the anchor/current-index arithmetic seems to be done in logical indices while the visual feedback is computed in display order, causing the selection to oscillate. Suggested fix ------------- Compute the selection anchor and drag endpoint consistently in visual (display) space, then map back to logical indices once per pointer event, so the visual highlight grows monotonically during the drag. I have a fix generated by Mistral AI on one of my project and I suggest to use it to fix the issue in Tk. Can you provide a way to submit code in python for the fix ? I failed to add the code to the fix description and I failed to attached a file. | |||
| User Comments: |
Arnault01 added on 2026-11-02 16:31:59:
Thank you for the validation :) 2 questions: 1. Can I have a summary, detailing what is left to be fixed if any ? 2. Can I have a release date integrating this branch ? I'm eager to use it :) nab added on 2026-11-02 15:30:57: Tested on macOS (aqua, TkDefaultFont):
serhiy.storchaka added on 2026-11-02 13:06:01: On trunk, the highlight of a part of a joined Arabic word does not match its bbox (X11 and Windows). In السعر (TkDefaultFont), selecting ع (bbox width 7) highlights 2 pixels, and ل (bbox width 4) highlights 10 pixels, shifted left. The chunks are measured during the layout with the string cut at the end of each chunk, so the last letter of a chunk takes its final form. Fixed on text-rtl-highlight in [4cb0789f36]: the chunks of a right-to-left string are placed by the advances in the whole string, as for the bbox. Tested on X11 and Windows, not on macOS. textDisp-42.7 fails on X11 on trunk with and without this fix: the insertion cursor at 1.49, inside the lam-alef ligature, changes how the ligature is drawn. kevin_walzer added on 2026-11-01 16:04:06: This is all fixed. Thank you! Arnault01 added on 2026-11-01 04:51:12: With my AI assistant, here is what I can say about bug 1: With recent step list, caret can be placed at the right margin of the hebrew paragraph and clicks on the right position correctly the insertion at the beginning of the line (tested with X11 and Windows). So it's still "can't reproduce actually on the tip of the dancing_bidi branch". I'll attach an headless test that leads me to that conclusion. Arnault01 added on 2026-11-01 04:38:39: Hello Kevin, For bug 1. To reproduce, run the bidi text widget demo and click the mouse at the start of the Hebrew clock on the right hand side. Can you elaborate as to: - What is the observed behavior - What should be the correct behavior Meanwhile I start to investigate. kevin_walzer added on 2026-10-31 18:03:52: Fix for these issues committed in [3403347d]; behavior is correct on X11, Windows and macOS. No new failures in textDisp.test. Thanks to all who reported these issues and assisted with patch submission. kevin_walzer added on 2026-10-31 15:05:43: Just attached a screenshot that shows RTL on trunk on Windows. kevin_walzer added on 2026-10-31 14:59:53: Serhiy, can you please review the dancing_bidi branch? RTL display is currently broken on trunk on Windows, and the dancing_bidi branch addresses that successfully. The function you are patching has been completely removed in my branch, and reinstating it restores the broken RTL display on Windows. We need to address the insertion cursor issue across platforms, the shimmering 'sel' issue on Windows, and retain correct layout of RTL text on Windows. In this case testing on X11 is NOT enough. serhiy.storchaka added on 2026-10-31 13:51:27: Reproduced on X11, also on trunk. On text-bidi-chunk-order the click gives the correct index (2.0, trunk gives 2.1), but the cursor was drawn at the left end of the line: the insertion cursor before the first chunk of a line was not moved with the reordered chunks. Fixed on text-bidi-chunk-order in [a4124a7246]. kevin_walzer added on 2026-10-31 12:30:49: To reproduce, run the bidi text widget demo and click the mouse at the start of the Hebrew clock on the right hand side. anonymous added on 2026-10-31 04:51:00: I just added a patch for the orphan nikkud issue. anonymous added on 2026-10-31 04:40:38: ltr-chunk-isrtl.patch is the proposed patch for bug 2. anonymous added on 2026-10-31 04:19:00: Hello Kevin, I'm working on trying to resolve both bugs. So far I cannot reproduce bug 1. "insert the caret at the right margin". Can you provide a more detailed step list or, better, a test script ? It is possible that since the comment it has been fixed in the tip of the dancing_bidi branch (core.tcl-lang.org). kevin_walzer added on 2026-10-30 21:51:19: I have applied all of Arnault's patches in the dancing_bidi, and there are two remaining bugs: 1. On all platforms, the caret/cursor is inset one glyph at the start of a paragraph of RTL text - you cannot insert it at the right margin. 2. On Windows, mix bidi text shimmers when selected - the 'sel' tag appears to be rendered in mirror relation to the cursor and text, so RTL text will draw a LTR selection bar. I have taken this branch as far as I can - hopefully someone else can look at it with fresh eyes and complete the fixes. I will be glad to test additional patches but cannot invest additional time in this. serhiy.storchaka added on 2026-10-30 15:14:29: Kevin, branch text-bidi-chunk-order is based on main: it places the chunks of each stretch in visual order and measures them from their logical positions, without the maxPrefixWidth clamp. On X11 it fixes the dancing with pure and mixed RTL text, and all text tests pass. On Windows there are no regressions, but a tag still moves RTL text, because of the measuring in [f3d964112f]. I could not test it on macOS. Unlike dancing_bidi it does not reorder stretches with different fonts, and it does not change tkWinFont.c. The new tests textDisp-37.4 to 37.6 may be useful for dancing_bidi too. kevin_walzer added on 2026-10-30 11:48:39: I went ahead and applied the three patches on macOS to a clean update of trunk and they do NOT solve the visual shimmering issue with pure RTL text. So I am going to continue my work on the dancing_bidi branch. The outstanding issue on Windows is visual artifacts with tags - the 'sel' tag is applied in a mirror image, so when a highlight is drawn RTL, text color changes LTR within that tag boundary. kevin_walzer added on 2026-10-30 11:34:48: Serhiy or Arnault, can you set up a new branch from main and these patches apply? I was doung my own work in dancing_bidi and things may have gotten mixes up. The current status of dancing_bidi is that all visual artifacts are corrected on macOS/X11 but not yet on Windows. serhiy.storchaka added on 2026-10-30 08:51:15: I tried branch dancing_bidi ([dec0422697]) on X11 (Xvfb, --enable-bidi), alone and with dancing-mixed-fix.patch. A tag splitting RTL text no longer moves it, but plain left-to-right text regresses; trunk ([d005489a6b], the base of the branch) passes all these tests. With the patch, textTag.test passes, but:
In the patch, inLayout is not restored when LayoutDLine() returns early for a line without characters (noCharsYet). On Windows 11 the branch (without the patch) has no regressions in these files. Arnault01 added on 2026-10-30 04:25:51: Here is my review: Overall The branch replaces my original chunk-reordering approach with a cleaner architecture: glyphs are never moved — the font backend keeps full responsibility for drawing — while the generic layer only records where each chunk lies (for backgrounds, hit-testing, bboxes). This is the right division of labor and fixes the regression seen with my first patch. Coding guidelines Mostly good Tcl/Tk style — Tcl_Alloc/Tcl_Free paired, const where appropriate, NULL guards on clientData, the classic *---* comment blocks, and a zero-cost fast path for lines without RTL chunks. A few deviations worth tidying up before merge: 1. Mixed indentation in LayoutDLine — the re-indented base-chunk finalization block uses spaces instead of tabs, breaking the file's tab style. 2. C99 declaration in loop header — for (int lo = i, hi = j; ...) in the rule-L2 code; Tcl/Tk style declares at block start. 3. Duplicated clamp block — the maxPrefixWidth clamp is copy-pasted in both the _WIN32 and X11/macOS branches of CharChunkMeasureChars. Two identical ~30-line blocks with subtle non-monotonicity logic in each is a maintenance hazard; a shared helper would be safer (the Windows-only original was one block). 4. Comment headers for ChunkBidiDir deviate slightly from the surrounding *---* formatting. Possible regressions 1. Confirmed bug — maxPrefixWidth outlives the layout. The clamp stores the whole-line width during layout and then rewrites post-layout measurements (CharBboxProc widths become "right edge of the line − x" on mixed-direction lines). This is the "still issues with mixed-bidi" symptom; it does not affect pure-RTL lines (clamp skipped via isRtl && placed). Fix: gate the clamp on an inLayout flag set in LayoutDLine (patch attached to the ticket). Arnault01 added on 2026-10-30 04:16:51: Following up on the remaining mixed-bidi issues on dancing_bidi: I found the root cause, and a small patch fixes it. Cause. The maxPrefixWidth clamp in CharChunkMeasureChars() keeps the value from the layout (the width of the whole line). After layout, when CharBboxProc() measures a single character, the clamp still runs and rewrites the correct result into lineWidth + charWidth. So on mixed-direction lines [bbox] returns the right x but width = right edge of the line − x for every character, which misplaces the caret and TextClosestGap. Pure-RTL lines are unaffected because the clamp is skipped there. Fix (attached, against dancing_bidi): gate the clamp on a new inLayout flag set in LayoutDLine(). The clamp keeps its purpose during layout (the Uniscribe non-monotonic case), and no longer applies to post-layout measurements (bboxes, hit-testing), which are anchored by startX and self-consistent. Result (X11 CI, --enable-bidi): before the patch, my mixed-line bbox/index round-trip test fails; with it, all my tests pass: pure-RTL stability under sel, cluster snapping, mixed-line stability, round-trip. Logs: https://github.com/Surcouf01/tk/tree/ci-results/dancing-bidi-fossil Happy to adjust the patch if you see a better way to scope the clamp. Arnault01 added on 2026-10-30 03:49:31: Hello Kevin, Can you drive me about how to perform the code review ? kevin_walzer added on 2026-10-30 03:20:09: I am working from this ticket. Arnault01 added on 2026-10-30 01:45:20: Closed because three separate and dependant tickets have been created kevin_walzer added on 2026-10-30 00:05:11: I re-opened the [dancing_bidi] branch and have what I believe are satisfactory results with selection and cursor tracking in RTL text. User-tested on the widget demo on X11; still need to test on macOS and Windows. Please review. kevin_walzer added on 2026-10-29 20:20:57: I closed the "dancing_bidi" branch because I was not getting anywhere with it. Starting over with trunk. kevin_walzer added on 2026-10-28 19:20:20: Thank you for the patches. I applied the three patches and tried the simple demo outlined in the ticket description - create a text widget on X11 and insert Hebrew: "שלום עליכם, זהו טקסט מימין לשמאל עם מספר מילים." I then tried to select the text, and what I saw was a serious regression: instead of a jumpy/"dancing" selection highlight, the glyphs themselves vanished and were redrawn in LTR order, and were dragged along the cursor path. This is the old legacy behavior of the text widget when I first started introducing bidi text. It took several weeks of work to correct this behavior, with changes in both the generic text layer (tkTextDisp.c) and the platform font implementations (tk*Font.c). These patches cannot be accepted in their present form. What will be acceptable is a smooth selection process that avoids the minor visual artifacts of the highlight drawing. The current setup in trunk leaves the text layout unaffected, which is critical to preserve. You are welcome to keep trying with this. kevin_walzer added on 2026-10-28 04:00:31: Please provide a succinct, non-AI summary of each patch - what it does and what platform it applies to. No new Tcl commands will be considered as part of a bugfix - let’s address the “dancing” behavior. I will try to review these soon after reviewing your briefer summary. Arnault01 added on 2026-10-28 03:06:35: I have forked the github tcltk repo to work on the 2 fixes (plus one more I encountered during tests).
Here is a description of the problems:
# Tk text widget — "dancing selection" with RTL text: submission drafts
Three independent tickets for the Tk bug tracker (core.tcl-lang.org, Tk — Text widget).
Each patch is self-contained; Tickets 1 and 2 were validated in headless CI
(Xvfb, Tcl 9 + Tk trunk, `--enable-bidi`, Xft + SheenBidi + HarfBuzz + ICU 78)
on the public fork https://github.com/Surcouf01/tk — branches
`bugfix/dancing-selection-bidi` and `bugfix/selection-cluster-snap`, workflow
logs published on the `ci-results/*` branches. Ticket 1 was additionally
validated on Windows (MSVC `nmake -f makefile.vc`, Tcl 9 trunk + Tk trunk,
Arial 14, X11-less) where the same reproducible test now passes; Ticket 3 is
the Windows-only bug that surfaced during that validation. The bugs are
separable: each patch fixes its own test and leaves the others failing on its
branch, which is expected (they are different bugs).
---
## Ticket 1 — RTL layout scrambled when a tag splits an RTL line
**Title:** text widget: RTL line layout scrambled when a tag (e.g. the selection) splits it — "dancing selection"
**Type:** Bug
**Description:**
When a tag — including the implicit `sel` tag — splits a display line containing
RTL text into several chunks, `LayoutDLine` places the chunks left-to-right in
logical order while the font backend shapes each individual chunk right-to-left.
The result is a scrambled line: reading order jumps around, and while dragging
the mouse the moving selection boundary keeps re-splitting the line, so the
layout changes under the pointer at every index step. Visually, the selection
"dances".
Reproducer (X11, Tk trunk built with `--enable-bidi`):
```tcl
package require Tk
text .t -font {DejaVuSans 16} -wrap none
.t insert 1.0 "שלום עליכם, זהו טקסט מימין לשמאל"
pack .t
update
# record the x of each character bbox, then apply the sel tag:
.t tag add sel 1.4 1.8
update
# record the x of each character bbox again: every index has moved
```
Expected: the visual position of every character is unchanged by the tag.
Observed (trunk): all 32 character positions move when the tag is applied; the
first four characters even end up on the wrong side of the line. Sample data
(x per char, before → after): `949 938 933 920 …` → `684 673 668 655 725 713 …`
**Root cause:**
The bidi resolution happens inside the font backend for a single contiguous
run, but nothing reorders the *chunks* of the display line when a tag splits
that run. This is the chunk-granularity counterpart of what the revised text
widget already solved ([e7c85cd941]).
**Proposed fix** (attached: `tkTextDisp-bidi-reorder.patch` against trunk
`generic/tkTextDisp.c`):
1. `BidiReorderChunks()`, called at the end of `LayoutDLine()`: resolves the
strong bidi direction of each char chunk (`ChunkBidiDir()`, a tri-state
variant of `ChunkIsRtl` using the same Unicode ranges — Hebrew, Arabic,
Syriac, NKo, Thaana, Samaritan, presentation forms, historical RTL scripts),
resolves neutrals (simplified N1/N2), then applies rule L2 by reversing each
maximal run of RTL chunks, reassigning their x positions in visual order.
The run keeps its left edge and its total width, so the line length and all
downstream geometry are unaffected. Non-character chunks never move.
2. `CharBboxProc()`: with `TK_LAYOUT_WITH_BASE_CHUNKS`, `CharChunkMeasureChars`
always anchors the advance at the *base* chunk, so the mirrored bbox of an
RTL chunk must be computed from the base chunk, not from the chunk itself:
mirror around the right edge of the run (the right edge of the base chunk,
which the reorder puts first-logical → rightmost). Without this, every char
of the non-base chunks collapses to a single x. The mirrored x is *not*
clamped to the chunk box: in base-chunk space it is expressed in the line's
coordinate system and legitimately falls outside the chunk that merely
carries the character's style (e.g. a `sel` chunk placed at the right edge
of an RTL line while its characters render further left) — clamping it
there re-created the dancing selection under Windows.
3. `TkTextDLineInfo()`: after the reorder the first chunk of the list is no
longer the leftmost one, so the left edge of the line is the minimum chunk x.
4. Windows only (`#ifdef _WIN32`, base-chunk block of
`CharChunkMeasureChars`): clamp the value returned by `MeasureChars` to be
no less than the span start. The Uniscribe backend returns non-monotonic
positions (notably when the measured range ends with `\n`, see Ticket 3);
without the clamp the newline chunk gets a negative width that corrupts the
DLine length and, with `-justify right`, shifts RTL lines out of the
widget.
Deliberate simplifications (documented in the code): the paragraph base level
is always LTR; European numbers are treated as neutrals; reordering is at chunk
granularity, not character granularity.
**Evidence:**
- Branch: https://github.com/Surcouf01/tk/tree/bugfix/dancing-selection-bidi
- CI (headless Xvfb, repro above scripted): `PASS: RTL layout is stable under
the sel tag` — all 32 character x positions identical before/after the tag,
`dlineinfo` unchanged (655 3 308 24 19). Logs:
https://github.com/Surcouf01/tk/tree/ci-results/bugfix/dancing-selection-bidi
(`.ci-logs/test1.log`); baseline on trunk-aligned `main` FAILs the same test.
- Windows (manual build, MSVC `nmake -f makefile.vc`, Arial 14, widget
884x532, `-wrap none`, same scripted test with `-justify right`): all 32
character x positions identical before/after the tag,
`indices whose x moved:` empty — `PASS`. Diagnostic branch with
instrumentation: https://github.com/Surcouf01/tk/tree/win-debug-bidi
(probes are compiled out in the attached patch; nothing is gated on
environment variables in the submitted code).
- Manual drag-testing of an interactive selection on a pointed-Hebrew line is
still recommended as final confirmation (CI is headless).
---
## Ticket 2 — Selection boundary lands inside a grapheme cluster
**Title:** text widget: mouse selection boundary can land inside a grapheme cluster (nikkud/teamim), selection oscillates while dragging
**Type:** Bug
**Description:**
With pointed Hebrew (or any script with combining marks), a base character and
its marks form one grapheme cluster occupying a single visual position. When the
mouse clicks near the right half of such a cluster, `::tk::TextClosestGap`
returns `"$pos + 1 char"` from the historical code path — an index that lands
*inside* the cluster, on a combining mark. During a drag, the boundary then
oscillates between the cluster's indices: the selection "dances" and characters
with nikkud flicker in and out of the selection.
Reproducer:
```tcl
package require Tk
text .t -font {DejaVuSans 16}
.t insert 1.0 "בְּרֵאשִׁית בָּרָא" ;# pointed Hebrew, clusters = base + marks
pack .t
update
# click at an x inside the right half of a cluster (e.g. the בְּ at the line start):
.t index @23,8
```
Observed (trunk): the returned index's character is U+05B4 — i.e. strictly inside
a cluster, not a valid gap between rendered positions.
**Proposed fix** (attached: `text-cluster-snap.patch` against trunk
`library/text.tcl`):
`::tk::TextClosestGap` now snaps the returned boundary to the edges of the
grapheme cluster containing `$pos`, using the ICU-based `tk::startOfCluster` /
`tk::endOfCluster` commands (UAX #29):
- `$pos` strictly inside a cluster → apply the half-width test to the base
character and return the nearest cluster edge (`$cstart` or `$cend`);
- `$pos` is the start of a multi-character cluster and the click is on the
right half → return `$cend` (the gap *after* the cluster) instead of
`"$pos + 1 char"`, which would land on a combining mark;
- otherwise the historical behavior is unchanged.
The commands only exist when libicu was found at start-up; without libicu the
procedure is byte-for-byte the historical behavior (guarded by
`[info commands ::tk::startOfCluster]`). The fix is script-agnostic: it applies
to Arabic diacritics, Indic conjuncts, emoji ZWJ sequences, etc., not only to
nikkud.
**Evidence:**
- Branch: https://github.com/Surcouf01/tk/tree/bugfix/selection-cluster-snap
- CI: `PASS: selection boundaries never split a grapheme cluster` (a scripted
scan of all gaps on a pointed-Hebrew line checks no returned boundary sits on
a combining mark). Logs:
https://github.com/Surcouf01/tk/tree/ci-results/bugfix/selection-cluster-snap
(`.ci-logs/test2.log`); baseline on trunk-aligned `main` FAILs the same test.
---
All three patches are independent; they can be applied and reviewed separately.
(Patch 1 includes the small Windows clamp of Ticket 3 because the Windows
validation of the bidi fix requires it; if the maintainers prefer, that hunk
can be reviewed as part of Ticket 3 instead.)
---
## Ticket 3 — Windows: Uniscribe returns non-monotonic widths; newline chunk gets negative width, right-justified lines overflow
**Title:** text widget, Windows: newline chunk gets a negative width (Uniscribe non-monotonic measurement) — right/center-justified lines are misplaced and overflow the widget
**Type:** Bug
**Description:**
On Windows, with `TK_LAYOUT_WITH_BASE_CHUNKS` (defined in `win/tkWinInt.h`),
`CharChunkMeasureChars` measures a span by calling `MeasureChars` over the
prefix `[0, bend)` of the base-chunk string. For some ranges the Uniscribe
backend returns a *right edge to the left of the span start* — i.e.
`*nextXPtr < startX` — which is geometrically impossible. Two observed
manifestations (Tcl 9 + Tk trunk, Arial 14, a one-line widget):
1. When the measured range includes the trailing `\n` of the line
(`bstart=58 bend=59 startX=262`), `MeasureChars` returns `nextX=167`: the
newline chunk is created with width **-95**. `LayoutDLine` then computes
`dlPtr->length = 167` instead of 262.
2. With `-justify right`, `jIndent = maxX - dlPtr->length` overshoots by the
negative width: the display line is placed at x=713 instead of 618, its
right edge (975) is beyond the widget (884), and the `bbox` of the last
characters returns -1. This is visible *before* any tag or selection is
involved — plain trunk behaviour on Windows.
Prefix measurements of bidi text are also non-monotonic in intermediate
ranges (`bend=19 → 975` vs `bend=20 → 885`), which corrupts per-character
bbox positions for the affected characters; the clamp does not mask that
larger issue but prevents the geometrically-impossible negative widths.
**Proposed fix** (attached: `tk-windows-uniscribe-measure.patch`, one hunk in
`generic/tkTextDisp.c`): after the `MeasureChars` call in the `#ifdef _WIN32`
base-chunk block of `CharChunkMeasureChars`, clamp:
```c
if (*nextXPtr < startX) {
*nextXPtr = startX;
}
```
A span cannot end to the left of where it begins; control characters such as
`\n` then get zero width (matching the X11 backend, where the same call
returns a monotonic cumulative width and the newline chunk is zero-width),
the DLine length is correct and justification places the line inside the
widget.
**Evidence:**
- Instrumented diagnostic run on Windows (branch
https://github.com/Surcouf01/tk/tree/win-debug-bidi, probes gated on the
`TK_DEBUG_BIDI` environment variable): newline chunk `w=-95`,
`dlPtr->length=167`, `jIndent=713`, line overflowing the widget; with the
clamp: `CCM-CLAMP: nextX 167 -> 262`, newline chunk `w=0`, `length=262`,
`jIndent=618`, line placed on [618, 880], inside the widget.
- The same clamp is part of `bugfix/dancing-selection-bidi`
(https://github.com/Surcouf01/tk/tree/bugfix/dancing-selection-bidi),
where the attached Windows reproducible passes with it and fails without it.
- X11 is unaffected by construction: the X11/macOS block of
`CharChunkMeasureChars` already receives monotonic cumulative widths from
its backend, and the CI on `bugfix/dancing-selection-bidi` passes unchanged
with and without this hunk.
Arnault01 added on 2026-10-17 01:21:25: Proposed patch for point 2 below. Arnault01 added on 2026-10-17 01:21:25: Proposed patch for point 2 below. Arnault01 added on 2026-10-17 01:00:34: Thank you for the thorough testing and analysis — very helpful. You are right that my patch addresses a different (and much smaller) issue: it only keeps selection boundaries out of clusters of combining marks, which stabilizes dragging in pointed Hebrew (nikkud/teamim) but does nothing for plain RTL text. The reproducer shows the actual root cause is in the display engine: when a tag — including the "sel" tag — splits an RTL line, the chunks are laid out left-to-right in logical order while each chunk renders right-to-left, so every change of the selection moves the text under the pointer. That is clearly the dominant bug, and the fix is the one already applied to the revised text widget in [e7c85cd941]: resolve the bidi levels and reorder the chunks of each display line. I agree that approach needs to be ported to the classic text widget at C level. On the patch itself, I take the point: * It should reuse tk::startOfCluster / tk::endOfCluster instead of hand-rolled cluster arithmetic in ::tk::TextClosestGap. * The hand-written list of combining marks is incomplete (Thai, Bengali, Hangul jamo, emoji modifiers, ZWJ sequences, ...) — a category-based test (e.g. via the Unicode general category Mn/Me/Mc or the Tcl_UniCharIsGraphemeCluster logic used elsewhere) would be preferable. * Bidi controls (U+200E, U+200F, U+202A–U+202E) are not combining marks and should not be classified as such. Given this, I see three options for how to proceed: 1. Keep this ticket focused on the bidi chunk-ordering bug (the real "dancing selection"), to be fixed at C level by porting the approach of [e7c85cd941] to the classic text widget. 2. If there is interest, I can rework my Tcl patch as a small follow-up: snap ::tk::TextClosestGap boundaries to cluster edges using tk::startOfCluster / tk::endOfCluster, as a separate enhancement for pointed text (vocalized Hebrew, Arabic with harakat) — filed as its own ticket. 3. Otherwise I withdraw the patch in its current form. Please let me know which you prefer; I'm happy to do the rework in option 2 regardless. serhiy.storchaka added on 2026-10-16 08:17:46: Reproduced on X11 (main, --enable-bidi). Without tags the right-to-left text is placed correctly, but a tag which splits it, the sel tag included, scrambles the layout: the chunks are placed from left to right in logical order, while each of them is drawn from right to left.
pack [text .t -font {{DejaVu Sans} 14}]
.t insert end "שלום עליכם, זהו טקסט מימין לשמאל"
update
foreach i {0 4 8} {lappend a [lindex [.t bbox 1.$i] 0]}
.t tag add sel 1.4 1.8
update
foreach i {0 4 8} {lappend b [lindex [.t bbox 1.$i] 0]}
puts "$a / $b" ;# 949 914 877 / 684 725 953
So each change of the selection moves the text under the pointer, and the selection jumps while dragging. The same problem was fixed in the revised text widget in [e7c85cd941], by resolving the bidi levels and reordering the chunks of each display line. The attached patch does not fix this: it only keeps the selection boundaries out of clusters of combining marks, and the example text has none. As for the patch itself: Tk already has tk::startOfCluster and tk::endOfCluster, which text.tcl uses for the cursor movement and deletion; a hand-written list of combining marks misses many scripts (Thai, Bengali, Hangul jamo, emoji modifiers, ZWJ sequences, ...), and bidi controls (U+200E, U+200F, U+202A–U+202E) are not combining marks. anonymous added on 2026-10-16 08:16:00: Tcl patch attached to the issue. kevin_walzer added on 2026-10-15 18:41:07: We are not able to accept patches written in Python. The code should be written in C against Tk's C API (if that is the correct place to target) or in Tcl. | |||
Attachments:
- bididemo-click.tcl [download] added by Arnault01 on 2026-11-01 04:51:50. [details]
- Screenshot 2026-10-31 110309.png [download] added by kevin_walzer on 2026-10-31 15:05:15. [details]
- orphan-nikkud-chunkbidi.patch [download] added by anonymous on 2026-10-31 04:49:43. [details]
- ltr-chunk-isrtl.patch [download] added by anonymous on 2026-10-31 04:35:29. [details]
- dancing-mixed-fix.patch [download] added by Arnault01 on 2026-10-30 04:17:16. [details]
- text-cluster-snap.patch [download] added by Arnault01 on 2026-10-28 03:07:42. [details]
- tk-windows-uniscribe-measure.patch [download] added by Arnault01 on 2026-10-28 03:07:42. [details]
- tkTextDisp-bidi-reorder.patch [download] added by Arnault01 on 2026-10-28 03:07:42. [details]
- patch2.txt [download] added by Arnault01 on 2026-10-17 01:20:32. [details]
- patch.txt [download] added by anonymous on 2026-10-16 07:09:34. [details]
