Tk Source Code

View Ticket
Login
2025-06-01
20:41 Ticket [5d0bc3cf] text widget: block cursor has wrong horizontal size status still Closed with 4 other changes artifact: 9fa51f65 user: fvogel
20:38 Ticket [55660624] Block cursor in text widget may hide the character underneath status still Open with 4 other changes artifact: 4f99faec user: fvogel
20:30 Closed ticket [5d0bc3cf]: text widget: block cursor has wrong horizontal size for connected unicode codepoints: Tk9 only plus 6 other changes artifact: 693352dc user: fvogel
20:11
Merge trunk (the fixes for Issue1. and Issue 2 discussed in [5d0bc3cfec] (see my comment dated 2025-05-31 14:25:43) are not needed for the revised text widget, which already behaves as expected to this regard. check-in: f1b573a0 user: fvogel tags: revised_text, tip-466
20:04
Fix Issue 1. and Issue 2. discussed in [5d0bc3cfec] (see my comment dated 2025-05-31 14:25:43). Also add caveat about the block cursor rendering. All patches provided by Christian Werner. check-in: 5d286a06 user: fvogel tags: trunk, main
19:59
Fix Issue 1. and Issue 2. discussed in [5d0bc3cfec] (see my comment dated 2025-05-31 14:25:43). Also add caveat about the block cursor rendering. All patches provided by Christian Werner. check-in: a3d271ba user: fvogel tags: core-9-0-branch
19:50
Fix Issue 1. and Issue 2. discussed in [5d0bc3cfec] (see my comment dated 2025-05-31 14:25:43). Also add caveat about the block cursor rendering. All patches provided by Christian Werner. check-in: 5c235c24 user: fvogel tags: core-8-6-branch
19:48
Add caveat about the block cursor rendering (see [5d0bc3cfec]). Patch provided by Christian Werner. Closed-Leaf check-in: 1d39bb14 user: fvogel tags: bug-5d0bc3cfec
19:41
When the text widget block cursor is in a tab, reduce its width to a whitespace width. This is part of in [5d0bc3cfec] (see Issue 2. in my comment dated 2025-05-31 14:25:43). Patch provided by Christian Werner. check-in: 9211ca27 user: fvogel tags: bug-5d0bc3cfec
19:21
Fix error in TextBlinkProc() for the text widget block cursor, as reported in [5d0bc3cfec] (see Issue 1. in my comment dated 2025-05-31 14:25:43). Patch provided by Christian Werner. check-in: 76ef49f3 user: fvogel tags: bug-5d0bc3cfec
08:12 Ticket [5d0bc3cf] text widget: block cursor has wrong horizontal size for connected unicode codepoints: Tk9 only status still Open with 3 other changes artifact: 90951f3e user: chw
06:47 Ticket [5d0bc3cf]: 3 changes artifact: fd0a1007 user: fvogel
04:42 Ticket [5d0bc3cf]: 3 changes artifact: 97774232 user: chw
2025-05-31
21:02 Ticket [5d0bc3cf]: 3 changes artifact: daed5aab user: fvogel
18:41 Ticket [5d0bc3cf]: 3 changes artifact: 3fd38463 user: chw
17:02 Ticket [5d0bc3cf]: 3 changes artifact: 5bb3fa4d user: fvogel
16:45 Ticket [5d0bc3cf]: 3 changes artifact: a5125cc1 user: chw
14:25 Ticket [5d0bc3cf]: 4 changes artifact: 720620b6 user: fvogel
2025-05-27
12:35 Ticket [5d0bc3cf]: 4 changes artifact: 2fb0696d user: oehhar
06:03 Ticket [5d0bc3cf]: 3 changes artifact: e355e36c user: fvogel
2025-05-20
16:26 Ticket [5d0bc3cf]: 4 changes artifact: 6ce166be user: oehhar
2025-05-18
12:56 Ticket [5d0bc3cf]: 3 changes artifact: e1f62941 user: fvogel
2025-05-02
06:53 Ticket [5d0bc3cf]: 4 changes artifact: 612b36ec user: oehhar
06:51
[5d0bc3cf] Second half of Androwish commit https://androwish.org/home/info/ae43177ce04ec244 Closed-Leaf check-in: 82feefa9 user: oehhar tags: 5d0bc3cf-text-blockcursor
06:44
[5d0bc3cf] add androwish commit https://androwish.org/home/info/85ea46a8c74b8f6f check-in: c3c9b7cd user: oehhar tags: 5d0bc3cf-text-blockcursor
2025-04-23
22:07 Ticket [5d0bc3cf] text widget: block cursor has wrong horizontal size for connected unicode codepoints: Tk9 only status still Open with 3 other changes artifact: 8fc16e2b user: fvogel
08:06 Ticket [5d0bc3cf]: 4 changes artifact: 0413033d user: oehhar
08:01
[5d0bc3cf] block cursor on tab has space width: https://androwish.org/home/info/ae43177ce04ec244 check-in: 3a6853ea user: oehhar tags: 5d0bc3cf-text-blockcursor
2025-04-22
18:01 Ticket [5d0bc3cf] text widget: block cursor has wrong horizontal size for connected unicode codepoints: Tk9 only status still Open with 3 other changes artifact: 220f04d1 user: chw
2025-04-08
19:54 Ticket [5d0bc3cf]: 4 changes artifact: 4d28c159 user: oehhar
18:41 Ticket [5d0bc3cf]: 3 changes artifact: 85d2eedb user: chw
17:19 Ticket [5d0bc3cf]: 4 changes artifact: c747c076 user: oehhar
13:37 Ticket [5d0bc3cf]: 3 changes artifact: 9860c4e8 user: jan.nijtmans
09:25 Ticket [5d0bc3cf]: 4 changes artifact: 94393ced user: oehhar
2025-04-07
08:34 Ticket [5d0bc3cf]: 4 changes artifact: b8caa7a3 user: oehhar
08:27
Ticket [5d0bc3cf] Text blockcursor: Androwish checkin https://www.androwish.org/home/info/b8894bb57964415f (thanks, Christian) check-in: 9ae9ae31 user: oehhar tags: 5d0bc3cf-text-blockcursor
2025-04-06
20:06 Ticket [5d0bc3cf] text widget: block cursor has wrong horizontal size for connected unicode codepoints: Tk9 only status still Open with 3 other changes artifact: 01ed83bd user: chw
19:25 Ticket [5d0bc3cf]: 3 changes artifact: 9f41039d user: fvogel
2025-04-04
14:00 Ticket [5d0bc3cf]: 4 changes artifact: 2bf64b1c user: oehhar
13:35 Ticket [5d0bc3cf]: 3 changes artifact: 28a50426 user: chw
10:23 Ticket [5d0bc3cf]: 4 changes artifact: f1c7c4f3 user: oehhar
10:14 Ticket [5d0bc3cf]: 3 changes artifact: 13717cb4 user: chw
10:06 Ticket [5d0bc3cf]: 4 changes artifact: debaafdd user: oehhar
09:56 Ticket [5d0bc3cf]: 3 changes artifact: 6cf9d1b2 user: chw
09:52 Ticket [5d0bc3cf]: 3 changes artifact: 4d14db1c user: chw
09:11 Ticket [5d0bc3cf]: 3 changes artifact: 6d0448ed user: oehhar
09:07 Ticket [5d0bc3cf]: 4 changes artifact: ebe0b6de user: oehhar
08:56 Ticket [5d0bc3cf]: 5 changes artifact: d8cf8613 user: oehhar
08:49
Ticket [5d0bc3cf] Text blockcursor: Androwish checkin https://androwish.org/home/info/85ea46a8c74b8f6f (thanks, Christian) check-in: 1d2248a3 user: oehhar tags: 5d0bc3cf-text-blockcursor
2025-04-03
21:26 Ticket [5d0bc3cf] Bug in TextBlinkProc() ??? status still Open with 3 other changes artifact: 051f9c1f user: chw
16:17 Ticket [5d0bc3cf]: 3 changes artifact: 43f4b178 user: chw
15:28 Ticket [5d0bc3cf]: 4 changes artifact: 9625eb16 user: oehhar
15:18 Ticket [5d0bc3cf]: 4 changes artifact: fce88506 user: oehhar
15:16
Ticket [5d0bc3cf]: adoption to 9.1: textPtr->insertWidth is now an Obj check-in: 60151aa1 user: oehhar tags: 5d0bc3cf-text-blockcursor
15:11
Ticket [5d0bc3cf] Fix block cursor of text widget. Original contribution for 8.6 by Christian Werner (thanks) check-in: 5263ef5e user: oehhar tags: 5d0bc3cf-text-blockcursor
14:58 Ticket [5d0bc3cf] Bug in TextBlinkProc() ??? status still Open with 4 other changes artifact: 3f13b71d user: oehhar
2025-04-01
06:18 New ticket [5d0bc3cf]. artifact: cd9702dc user: chw

Ticket UUID: 5d0bc3cfec7c1adb87053bc49fce770309d125de
Title: text widget: block cursor has wrong horizontal size
Type: Bug Version: most likely all
Submitter: chw Created on: 2025-04-01 06:18:52
Subsystem: 18. [text] Assigned To: fvogel
Priority: 5 Medium Severity: Minor
Status: Closed Last Modified: 2025-06-01 20:41:32
Resolution: Fixed Closed By: fvogel
    Closed on: 2025-06-01 20:41:32
Description:
See https://core.tcl-lang.org/tk/artifact?name=738743581c2dda0e&ln=3727

This line should be (at least seems more logical)

  TkTextRedrawRegion(textPtr, x - insertWidth / 2, y,

instead of (seems plain wrong due to textPtr->width not measuring pixels)

  TkTextRedrawRegion(textPtr, x - textPtr->width / 2, y,
User Comments: fvogel added on 2025-06-01 20:30:23:

Thanks for the text.n update it looks OK to me.

I have now merged the patch for Issues 1. and 2., plus the text.n update, in core-8-6-branch, core-9-0-branch and trunk. The revised_text branch only needed the update to text.n, which is now committed there as well.

Regarding Issue 3., I have opened a new ticket [55660624f7].

I'm closing the present ticket now.


chw added on 2025-06-01 08:12:15:
> If you can propose better documentation text I will commit it happily.

What about:

---8><---8><---
Index: doc/text.n
==================================================================
--- doc/text.n
+++ doc/text.n
@@ -33,11 +33,12 @@
 Specifies a boolean that says whether separators are automatically inserted in
 the undo stack. Only meaningful when the \fB\-undo\fR option is true.
 .OP \-blockcursor blockCursor BlockCursor
 Specifies a boolean that says whether the blinking insertion cursor should be
 drawn as a character-sized rectangular block. If false (the default) a thin
-vertical line is used for the insertion cursor.
+vertical line is used for the insertion cursor. For further discussion
+refer to section \fBTHE INSERTION CURSOR\fB below.
 .OP \-endline endLine EndLine
 Specifies an integer line index representing the line of the underlying
 textual data store that should be just after the last line contained in
 the widget. This allows a text widget to reflect only a portion of a
 larger piece of text. Instead of an integer, the empty string can be
@@ -915,10 +916,17 @@
 with the
 .QW "\fIpathName \fBmark unset\fR"
 widget command. The \fBinsert\fR mark represents the position of the insertion
 cursor, and the insertion cursor will automatically be drawn at this point
 whenever the text widget has the input focus.
+.PP
+The \fB\-blockcursor\fR widget option controls the drawing of the cursor.
+However, drawing the cursor as a solid blinking block is not exactly
+performed as in real or emulated terminals. The character at the cursor
+position is always drawn in it's foreground color, i.e. not in
+"reverse video", which can lead to unwanted visual effects and even
+hide the character entirely, when the cursor is in its on-state.
 .SH "THE MODIFIED FLAG"
 .PP
 The text widget can keep track of changes to the content of the widget by
 means of the modified flag. Inserting or deleting text will set this flag. The
 flag can be queried, set and cleared programmatically as well. Whenever the
---8><---8><---

> I'm ready to admit that this is a case where -insertbackground does
> not work. But I simply can't bring myself to commit the code you
> proposed to solve this issue 3.. I'm not taking the responsibility
> of landing this in trunk. I'm about to jump off this subject now,
> but before that I will merge your code for issue 1. and issue 2.

Thanks, two points resolved. Maybe we can agree at least, that at a
later point in time I will bring up another RFE to fix this IMO big
deficiency.

fvogel added on 2025-06-01 06:47:39:
> * With "not documented" I've meant the missing description of the
> proportionality of ergonomics to the -insertofftime option.

If you can propose better documentation text I will commit it happily.

> * Compare the text widget's logic to how xterm or gnome-terminal
> draw the block cursor. They draw the character under the cursor
> in a different color resembling reverse video of vt100 terminals.
> 
> * Imagine a text widget on a monochrome EInk display: the color
> of the block cursor is no solution.

I'm ready to admit that this is a case where -insertbackground does not work. But I simply can't bring myself to commit the code you proposed to solve this issue 3.. I'm not taking the responsibility of landing this in trunk. I'm about to jump off this subject now, but before that I will merge your code for issue 1. and issue 2.

chw added on 2025-06-01 04:42:03:
Francois, more disappointed than sarcastic.

* With "not documented" I've meant the missing description of the
proportionality of ergonomics to the -insertofftime option.

* Compare the text widget's logic to how xterm or gnome-terminal
draw the block cursor. They draw the character under the cursor
in a different color resembling reverse video of vt100 terminals.

* Imagine a text widget on a monochrome EInk display: the color
of the block cursor is no solution.

fvogel added on 2025-05-31 21:02:13:

The -insertbackground option of the text widget is documented... from the text widget man page in the options man page (how surprising).

I'm unsure whether you're sarcastic in your answer or not, so I take the shortest path and name it an agreement or at least an acceptation from you.

If I'm mistaking and if you can't live with simply using -insertbackground and really really think some hack should absolutely be made then I would prefer it to go in TkTextInsertDisplayProc() here similarly as the case of the selection background and insertion cursor background being the same. For instance (obviously here "grey" should be refined, this is just a POC):

Index: generic/tkTextMark.c
==================================================================
--- generic/tkTextMark.c
+++ generic/tkTextMark.c
@@ -624,10 +624,11 @@
     /* TkText *textPtr = chunkPtr->clientData; */
     TkTextIndex index;
     int halfWidth, insertWidth, insertBorderWidth;
     int rightSideWidth;
     int ix = 0, iy = 0, iw = 0, ih = 0, charWidth = 0;
+    XColor bgColor;
 
     Tk_GetPixelsFromObj(NULL, textPtr->tkwin, textPtr->insertWidthObj, &insertWidth);
     Tk_GetPixelsFromObj(NULL, textPtr->tkwin, textPtr->insertBorderWidthObj, &insertBorderWidth);
     halfWidth = insertWidth/2;
     if (textPtr->insertCursorType) {
@@ -658,13 +659,22 @@
      * the cursor.
      */
 
     if (textPtr->flags & GOT_FOCUS) {
 	if (textPtr->flags & INSERT_ON) {
-	    Tk_Fill3DRectangle(textPtr->tkwin, dst, textPtr->insertBorder,
-		    x - halfWidth, y, charWidth + insertWidth,
-		    height, insertBorderWidth, TK_RELIEF_RAISED);
+	    Tk_Get3DBorderColors(textPtr->insertBorder, &bgColor, NULL, NULL);
+	    if (bgColor.pixel != textPtr->fgColor->pixel) {
+		Tk_Fill3DRectangle(textPtr->tkwin, dst, textPtr->insertBorder,
+			x - halfWidth, y, charWidth + insertWidth,
+			height, insertBorderWidth, TK_RELIEF_RAISED);
+	    } else {
+		Tk_3DBorder borderPtr = Tk_Get3DBorder(textPtr->interp, textPtr->tkwin, "grey");
+
+		Tk_Fill3DRectangle(textPtr->tkwin, dst, borderPtr,
+			x - halfWidth, y, charWidth + insertWidth,
+			height, insertBorderWidth, TK_RELIEF_RAISED);
+	    }
 	} else if (textPtr->selBorder == textPtr->insertBorder) {
 	    Tk_Fill3DRectangle(textPtr->tkwin, dst, textPtr->border,
 		    x - halfWidth, y, charWidth + insertWidth,
 		    height, 0, TK_RELIEF_FLAT);
 	}


chw added on 2025-05-31 18:41:03:
Almost nothing except that it is nowhere mentioned (i.e.
documented), it is unusual regarding the standard behavior
of tons of commonly used terminal emulation frameworks,
and finally there's an admittedly infinitesimal chance
for a monochrome X11 display (which should be extinct
today anyway) somewhere in the lesser known part of the
universe.

fvogel added on 2025-05-31 17:02:46:
What's wrong with configuring the text widget with -insertbackground grey (for instance)?

chw added on 2025-05-31 16:45:33:
Francois, while I agree that the code for point 3 is ugly
and should be written in a saner and cleaner way, point 3
is the most annoying for me: the defaults of the text widget
yield an almost unusable block cursor when enabled. And
this should definitely be addressed. Either by ditching the
block cursor entirely or by fixing it once and for all.
Why have a defect feature by default anyway?

fvogel added on 2025-05-31 14:25:43:

Analyzing the code in the bugfix branch, I believe the patch does exactly three things:

  1. Fix the error initially reported in this ticket (the textPtr->width --> insertWidth change in the call to TkTextRedrawRegion() in TextBlinkProc()).

This change is OK for me and should indeed be done. The current statement must be an error indeed (textPtr->width is a measure in characters, not pixels) and was here since the implementation of TIP #155 was merged in [e58248ce5f].

  1. Limit the width of the block cursor to 1 character width when the block cursor is on a tab (in trunk the block cursor width on a tab is the tab width).

Sounds like a good idea indeed. Implemented by adding an output parameter to TkTextIndexBbox() returning the cursor width. Nice and clean. Imprimatur from me on this.

  1. Avoid the masking of the character under the block cursor when it's in the ON state (which is particularly annoying when the block cursor has -insertofftime 0, i.e. when it does not blink).

This is obtained by a (sorry!) awful wart creating, at display time, a temporary chunk with a style having a graphics context foreground color pulled from the text widget background color (picked from its border attribute) instead of from the foreground color of the text. As a result the character "under" the block cursor is drawn in the background color of the text widget instead of its foreground color, which reveals this character since the cursor is drawn with the foreground color of the widget. Well, the code is rather ugly. I think it's a wart, and it may miss its target anyway as long as the widget has (tagged) colored characters. In any case, the overall argument against this wart is that the problem this hack tries to address can be entirely avoided just by setting the -insertbackground option of the widget. So I don't think this is a good idea at all and I'm opposed to merging this part of the branch.

Apart from the three items above, this ticket is hosting discussions about things that IMHO are unrelated to the behavior of the block cursor. Those things (behavior of the text widget with some specific unicode content, schizophrenia between how the cursor moves and how glyph widths are computed, and other side considerations) should be the object of specific tickets. Please avoid mixing things in a single ticket, this brings lots of confusion rather than help (at least for me). One ticket per single issue is the guideline, as a principle.

Finally, the current situation for the revised text widget is:

  1. Already fixed (albeit differently), see TkTextGetCursorBbox().
  1. Already done, ditto see TkTextGetCursorBbox().
  1. Not done, and I don't believe it should be done, as explained above.

And last, here is the test code I'm using (if you have something better please post it):

package require Tk
pack [text .t -blockcursor true]
set str [string repeat "123\t56\t890" 20]
set str [string repeat $str\n 10]
.t insert end $str
.t tag add large 3.0 5.0
.t tag configure large -font {Arial 20}

Note: I have tried all this on Windows only so far. This should be good enough because 1. and 2. are platform-independent code. Only 3., in case it would be kept, would need to be tested on the three platforms (and on Linux both with and without Xft). But again, I definitely think we shouldn't do 3.


oehhar added on 2025-05-27 12:35:02:

I will try to get some information from Christian. I have not seen any effect by the patch except the tab block cursor.

The main open point is IMHO glyph line break using ICU glyphs.

Thanks for all, Harald


fvogel added on 2025-05-27 06:03:53:
The patch does much much more than what is in the present ticket report.

Moreover I seeem to get lost in the discussion below. Would it be possible to have a simple, short, statement of what the branch is supposed to fix and what are the remaining issues (since the exchanges below seem to state there are such issues)?

oehhar added on 2025-05-20 16:26:44:

Francois, thanks for looking into this, I really appreciate.

One question to your post:

  • do you think, the functionality in the main branch is ok?
  • or do you think that the functionality in the bug branch is ok?

IMHO, ICU should be used for glyph clustering anyware.

The current functionality is already great, that composed unicode codepoints are rendered. Only the flag stuff is not rendered properly on Windows.

As seen with other stuff, Christian has patched Linux to support rotated color icons. It may happen, that he will also get active with the flags.

The one step cursor move over the two glyphs "DE" is already great. Only the character break at the end of the line is not working.

For me, practically, 9.0 is so much better than 8.6 in all those respects. Thanks for all the work, Harald


fvogel added on 2025-05-18 12:56:51:

I have spent some time diving into this ticket again, and would like to save this effort for others doing the same. Therefore I'm pasting Christian Werner's latest post on tcl-core about this (April 5th, 2025), which sums up the situation quite nicely and should not be lost:

From my point of view, the block cursor now is working, although
the code of the patch is still debatable and possibly could be
improved and/or simplified.

However, some concerns regarding Tk came up, which can be read in
the comments to ticket https://core.tcl-lang.org/tk/info/5d0bc3cf (i.e. the present ticket).

My conclusions so far:

* Most likely, the block cursor is fine on MacOS, so please test.
  No actions needed on this platform, hopefully.

* On X11 and Win32 TIP#621 is in the way: it implements run-time
  linking of ICU for determining bounds of glyph clusters among
  other features. And Tk 9.x uses these functions for navigating
  e.g. by cursor keys through the chars in a text widget. However,
  since both, X11 and Win32 font rendering isn't glyph cluster
  aware at all, this leads to rather ugly behavior which Harald
  documented in the ticket.

  Action required: it must be decided, if tk::startOfCluster and
  tk::endOfCluster with ICU backing is to be used in Tk's text
  widget or not. If not, the block cursor will be fine. If it
  shall be used, the whole complexity of char measurement in the
  guts of Tk must be improved to take glyph clustering into
  account when measuring the pixel widths of char sequences.


oehhar added on 2025-05-02 06:53:19:

Thanks Francois ! At least, someone else who cares ;-).

Included the two androwish commits now.

Compiles on Windows with MS-VC2022 and shows the small block cursor on a Tab.

Thanks for all, Harald


fvogel added on 2025-04-23 22:07:59:

Hmmm... see https://androwish.org/home/info/85ea46a8c74b8f6f ?


oehhar added on 2025-04-23 08:06:53:

Hi Christian, thanks for the patch. Parts of it are now in the branch with commit [3a6853ea].

The part in file "tkTextDisp.c " around line 2575 is currently not in the main branch. There is no call to "TkTextIndexBbox" here in the main branch. The code looks totally different.

I added a comment to tell what happens. Maybe, this may be back-ported ;-).

I use the test script:

pack [text .t -blockcursor 1]
.t insert end "1\t2"

The test is effective for me. I tested on Windows with MS-VS 2022 64 bit. The block cursor on the tab is now one whitespace wide.

That is big happyness, as the first time, I see something with the changes in the branch ;-).

Thanks, Harald


chw added on 2025-04-22 18:01:36:
Another annoying property of the block cursor is its varying width
when the insert mark is on a tab character. See this check-in

 https://androwish.org/home/info/ae43177ce04ec244

for a proof of concept solution. It forces the block cursor to the
width of a space when on a tab which seems more logical.

oehhar added on 2025-04-08 19:54:50:

> So my theory of multi brain schizophrenia is still valid. > Time for therapy?

Yes, like facts and alternate facts. Earth also need a thrapy.

I am just impressed what already works. I suppose, you, Jan and many others have spent many days to make this happen!

To make the flag-stuff work is a long way on Windows. We first need color fonts rendering.

I am also with you that a single source of truth would be great. And that icu is IMHO the way to go.

The last test with "a\u0300e\u0301i\u0302o\u0303u\u0304" could also first be normalized by icu, as all those two codepoint sequences have a one codepoint representation:

Look at the codepoints:

  • "a": very simple
  • "\u0300": Combining Grave Accent.

The combination of those two is "à", which has the unicode codepoint "\u00E0". So, I suppose a preliminary normalization step (by icu), we will have transformed "a"+"\u0300" to "\u00e0".

That is another alternate fact...

Thanks for all, Harald


chw added on 2025-04-08 18:41:20:
Howdy Harald,

> Tk 9.0.1: all works well.

For things which the current Windows API respects as
grapheme cluster. BTW, the same snippet renders different
in Wine. And then there are the regional indicator symbols
which Windows won't make into flags but ICU sees as a
combined item.

So my theory of multi brain schizophrenia is still valid.

Time for therapy?

oehhar added on 2025-04-08 17:19:07:

I have tried the example from the referenced ticket:

set txt a\u0300e\u0301i\u0302o\u0303u\u0304
pack [text .t -width 20 -height 3 -blockcursor 1]
.t    insert end $txt

Windows status:

  • Tk 8.6: displays correct, but cursor steps through the components e.g. "a", "`". Block cursor size changes sometimes to nearly zero width.
  • Tk 9.0.1: all works well.

Harald


jan.nijtmans added on 2025-04-08 13:37:55:

Looks related to [this] ticket


oehhar added on 2025-04-08 09:25:14:

For me, a block cursor was present in times, when fix font width was the normal thing. My C64 had a block cursor.

I would not mind, if the block cursor would be of fix width of an average character. It may be an advantage, that its position is easier to locate.

Nevertheless, all the other mega-Wizard points of this ticket:

  • combining collated Unicode codepoints in one glyph
  • wrap-around at glyph boundaries

are IMHO more important and super-challenging ;-).

Take care, Harald


oehhar added on 2025-04-07 08:34:54:
Here is a copy of the core messages by Christian:

---

Date: Tue, 1 Apr 2025 22:25:36 +0200

Dear fellow readers of the core mailing list,

after some playing with the text widget's -blockcursor option my conclusion
is unfortunately, that it is almost unusable when turned on, at least when
the default options of the text widget are in place, which is a very common
use case.

Rationale: the block cursor in its active state (INSERT_ON) draws a rectangle
in the text widget's foreground color. Following that drawing the text is drawn
in the text widget's foreground color resulting in a rectangle in the text
widget's foreground color, i.e.  without any information which was in place
when the widget/cursor was in inactive (INSERT_OFF) state. This makes the
block cursor indefinitely hiding the character under the cursor when the
-insertofftime option is set to zero. Bluntly, the "-blockcursor 1" option
is not usable at all since it's invention, BTW this was in 2003.

Here is a POC how it can be remedied:

---8><---
--- old/tkTextDisp.c
+++ new/tkTextDisp.c
@@ -15,6 +15,7 @@
  #include "tkInt.h"
 #include "tkText.h"
+#include "tk3d.h"
  #if defined(_WIN32) && !defined(PLATFORM_SDL)
 #include "tkWinInt.h"
@@ -2444,10 +2445,13 @@
 				 * must make sure it's large enough to hold
 				 * line. */
 {
-    TkTextDispChunk *chunkPtr;
+    TkTextDispChunk *chunkPtr, tmpChunk, *otherChunkPtr = NULL;;
     TextDInfo *dInfoPtr = textPtr->dInfoPtr;
     Display *display;
     int height, y_off;
+    struct TextStyle tmpStyle;
+    TkBorder *borderPtr;
+    CharInfo ci;
 #ifndef TK_NO_DOUBLE_BUFFERING
     const int y = 0;
 #else
@@ -2539,6 +2543,38 @@
 	     * here.
 	     */
 +	    if (textPtr->insertCursorType &&
+		    ((textPtr->flags & (GOT_FOCUS | INSERT_ON)) ==
+			(GOT_FOCUS | INSERT_ON)) &&
+		    (chunkPtr->nextPtr != NULL) &&
+		    (chunkPtr->nextPtr->displayProc == CharDisplayProc) &&
+		    (chunkPtr->nextPtr->numBytes > 0)) {
+		/*
+		 * Make a temporary chunk for displaying the text
+		 * within the block cursor later on.
+		 */
+
+		otherChunkPtr = &tmpChunk;
+		*otherChunkPtr = *(chunkPtr->nextPtr);
+		otherChunkPtr->undisplayProc = NULL;
+		otherChunkPtr->numBytes = 1;
+		tmpStyle = *otherChunkPtr->stylePtr;
+		otherChunkPtr->stylePtr = &tmpStyle;
+		tmpStyle.bgGC = None;
+		borderPtr = (TkBorder *) textPtr->border;
+		tmpStyle.fgGC = borderPtr->bgGC;
+		ci = *((CharInfo *) (otherChunkPtr->clientData));
+		otherChunkPtr->clientData = (ClientData) &ci;
+		ci.numBytes = 1;
+		if ((ci.chars[0] == '\n') ||
+			(ci.chars[0] == ' ') || (ci.chars[0] == '\t')) {
+		    /*
+		     * Ignore newline and other whitespace.
+		     */
+
+		    otherChunkPtr = NULL;
+		}
+	    }
 	    continue;
 	}
 @@ -2569,6 +2605,28 @@
 		    display, pixmap, dlPtr->y + dlPtr->spaceAbove);
 	}
 +	if ((otherChunkPtr != NULL) && (textPtr->tkwin != NULL) &&
+		!(textPtr->flags & DESTROYED)) {
+	    /*
+	     * Draw text within the block cursor.
+	     */
+
+	    int x = otherChunkPtr->x + dInfoPtr->x - dInfoPtr->curXPixelOffset;
+
+	    if ((x + otherChunkPtr->width <= 0) || (x >= dInfoPtr->maxX)) {
+		/*
+		 * See note above.
+		 */
+
+		x = -otherChunkPtr->width;
+	    }
+	    otherChunkPtr->displayProc(textPtr, otherChunkPtr, x,
+		    y + dlPtr->spaceAbove, dlPtr->height - dlPtr->spaceAbove -
+		    dlPtr->spaceBelow, dlPtr->baseline - dlPtr->spaceAbove,
+		    display, pixmap, dlPtr->y + dlPtr->spaceAbove);
+	    otherChunkPtr = NULL;
+	}
+
 	if ((textPtr->tkwin == NULL) || (textPtr->flags & DESTROYED)) {
 	    /*
 	     * A displayProc called in the loop above invoked a binding
---8><---

I've tested it in an X11 environment. No idea if this works in Win32 or MacOS.

Cheers,
Christian

---

Date: Tue, 1 Apr 2025 22:57:05 +0200

On 04/01/2025 10:25 PM, Christian Werner wrote:
> after some playing with the text widget's -blockcursor option my conclusion…

I should have admitted, that the POC patch is not ideal, since it leaves
artefacts of the double drawing of the character under the cursor due to
antialiasing, when the xft font rendering is in effect.

Nevertheless, it gives a more natural feedback resembling a good old vi
in a good old terminal. And wasn't that the original intention and means
of existence of the block cursor option?

BR,
Christian

---

Date: Wed, 2 Apr 2025 20:56:08 +0200

On 04/01/2025 10:57 PM, Christian Werner wrote:

> …
> Nevertheless, it gives a more natural feedback resembling a good old vi
> in a good old terminal. And wasn't that the original intention and means
> of existence of the block cursor option?

Meanwhile the patch evolved to work on Windows and MacOS, too, see

  https://www.androwish.org/home/info/b8894bb57964415f

Cheers,
Christian

---

The last mentioned patch is now transfered to the bug branch with checkin [9ae9ae31].

I see no visual effect on Windows, but it compiles without issues. Still would love feedback from other platforms.

I have to admit that this is total Wizard stuff, like the color font patch by Christian to Xflt what Androwish & friends feature.

My appreciation,
Harald

chw added on 2025-04-06 20:06:04:
Francois,

in Harald's words TRIPLE-BUMMER. And the top is, that the code
in revised_text is completely different in this regard and not
straight forward. With tons of worms of logic for drawing as far
as I've seen in five minutes.

fvogel added on 2025-04-06 19:25:37:

On top of all this: do we have a problem with the revised text widget (revised_text branch)? If so, the fix should probably be adapted before merging main into revised_text.


oehhar added on 2025-04-04 14:00:59:

Christian,

thanks a lot to look into this.

This is highly tricky:

What this browser (firefox on Windows) does:

  • "\U1F1E9" renders as a blue inverted D "🇩"
  • "\U1F1EA" renders as a blue inverted W "🇪"
  • "\U1F1E9\U1F1EA" renders as a German national flag "🇩🇪". It is larger, than its components. This is perhaps the acheved size of the block cursor.

Ideal behaviour for the unicode sequence "\U1F1E9\U1F1EA":

  • It should be rendered as the German flag using colors defined by the characters.
  • It should be handled as one glyph in any case. This implies: end-line break, cursor positioning, block cursor size.

The current Windows implementation for 9.0 differs in the following points:

  • No color
  • Displayed as two characters "DE"
  • Handled as one glyph on cursor traversal (due to icu)
  • text widget block cursor is smaller than both characters together and larger than one character. It might be the size of the imaginary flag.
  • "D" and "E" is separated on wrap-around at the end of the widget
  • the bugfix branch makes no visible difference

Double-BUMMER, create multiple tickets.

I would nevertheless love to have a platform where the patch makes a difference and describe this. Bhaviour before/behaviour after.

Does someone voluntear to test on MacOS or Linux with magic color character patch by Christian?

Incredibble Wizard stuff ! Harald


chw added on 2025-04-04 13:35:55:
Harald,

from what I quickly observed using Ashok's magicsplat 9.0.1 installation,
it is indeed the text bindings using some ICU grapheme cluster info for
cursor positioning, however the text rendering not at all due to using
good ole GDI interfaces under the hood. And this causes IMO confusion all
over the place. In order to draw the block cursor proper the char
measurement machinery would have to know about the grapheme cluster.

What else can I say: only use the ICU grapheme cluster facility if you
can ensure, that the font measurement and rendering infrastructure also
supports it (and at the same Unicode version level hopefully).

This could be the case on MacOS currently. Definitely not on X11 and Win32.

Bummer! This should become a ticket of its own, eventually.

oehhar added on 2025-04-04 10:23:56:

Christian,

I like the discussion.

I would love to enhance the original ticket text with a use-case, what is addressed and on which platform and Tk version.

I understand that Androwish has a feature set min-bound by Tk8.6.10 and max-unbound. And the Androwish solution may not fit to the Tk main branch reality.

Yes, the introduction of icu has potentially many side-effects. Nevertheless, it is IMHO the way to go. But I am not sure, if it is used for to find the next glyph. I am sure it is used to find the next word.

Thanks for all, Harald


chw added on 2025-04-04 10:14:59:
Harald,

> Isn't icu used in Tk 9.0 for cursor traversal? May icu be responsible for the one glyph magic?

that's a very important point. Expect another can of worms.
My understanding is that ICU can know of the boundaries of
grapheme clusters in an UTF-8 (or Unicode) sequence. But
this has nothing to do with the rendering which isn't ICU's
business but some platform infrastructure. What if those
two brains have different opinions? Schizophrenia even?

oehhar added on 2025-04-04 10:06:05:

Christian, thanks for the golden insights.

In Tk 9.0, the "DE" is rendered at least in a special font and it is as one character. It is black on white, but works. And it is seen as one glyph, in the sense, that the cursor moves over "DE" in one step.

Isn't icu used in Tk 9.0 for cursor traversal? May icu be responsible for the one glyph magic?

And yes, the block cursor is not two glyphs, but 1,5...

Thanks for all, Harald


chw added on 2025-04-04 09:56:55:
And BTW when the symbol is not inverted for the default
foreground/background of the text widget, this indicates
that a color (Emoji) font is used for rendering. That
in this case there can't be a color change is expected,
since all color comes from the glyph of the font.

chw added on 2025-04-04 09:52:30:
Harald,

not quite true. It depends on the underlying font renderer and
related infrastructure. On MacOS the symbol TK_DRAW_IN_CONTEXT
is defined which uses the logic of the MacOS font renderer to
deal with grapheme clusters. In this case, for \U1F1E9\U1F1EA
the German flag is rendered which counts as one item. On Win32
(and X11, too) those are two items, one with a D enclosed in a
rectangle, and one with an E. If only a rectangle is drawn,
then either your Tk does not support Unicode beyond BMP or
the font doesn't have that symbol. So we're in muddy waters
now and have to navigate in a swamp of platform idiosyncrasies.
For now, i.e. 2025 AD, I tend to see all of this untestable.
So let's wait another decade ;-) All in all, the flubbed repair
of the block cursor took more than 20 years already.

oehhar added on 2025-04-04 09:07:09:

I tried with "-blockcursor 0". Here are the results:

  • 8.6.14, win32: two question marks are shown. Two characters. Correct cursor and navigation.
  • 9.0, win64: "DE" is shown but not inverted (as advised by character definition). "DE" is seen as one character.
  • bug branch, checkin [1d2248a3], win64: same as 9.0

So, apparaently, the aim is, that "DE", "\U1F1E9\U1F1EA" is one character. If this is true, the block cursor size is to short.

Learning... Harald


oehhar added on 2025-04-04 08:56:47:

Thanks for the code change. it is now in checkin [1d2248a3].

Thanks for the recipe. I tried:

pack [text .t -blockcursor 1]
.t insert end "\U1F1E9\U1F1EA"
The characters are:
  • \U1F1E9: country sign letter "D": 🇩
  • \U1F1EA: country sign letter "E": 🇪

Here are my test results with this recipe:

  • 8.6.14, win32: two question marks are shown. When moving on the, the block cursor is slightly larger
  • 9.0, win64: "DE" is shown but not inverted (as advised by character definition). The block cursor sees only one character. The block cursor is horizontally larger.
  • bug branch, checkin [60151aa1], win64: "DE" is shown, but the "E" has a smaller font or index shape. The block cursor sees only one character. The block cursor is horizontally larger.
  • bug branch, checkin [1d2248a3], win64: same as 9.0

It is probably me, Windows or main branch instead 8.6...

Thanks for all your work, Harald


chw added on 2025-04-03 21:26:45:
The grapheme cluster blues should be solved in

  https://androwish.org/home/info/85ea46a8c74b8f6f

This approach seems to do the Right Thing™ at
least on the exact start of a grapheme cluster.
When the cursor (block or I-beam doesn't matter)
is in the middle, still strange things happen.

But this is an entire other can of worms and
independent of the block cursor (i.e. not
related to this patch).

chw added on 2025-04-03 16:17:17:
Howdy Harald,

I'm still improving my block cursor POC since we can have grapheme
clusters at least on MacOS, i.e. insert one of those flag sequences,
e.g. "\U1F1E9\U1F1EA", then move the block cursor over it and observe
how broken it is rendered when the blink is on. I guess I better shall
use CharChunkMeasureChars() in order to find the proper length of the
sequence. Stay tuned for another update.

oehhar added on 2025-04-03 15:28:35:

I have tried:

pack [text .t -blockcursor 1]
in 9.0 and the branch and see no visual difference...

Both block cursors look correct.

Harald


oehhar added on 2025-04-03 15:18:53:

Starting with commit [5263ef5e], it is now in branch [5d0bc3cf-text-blockcursor].

After 9.1 adoption, all tests pass (what was already the case before).

Thanks, Harald


oehhar added on 2025-04-03 14:58:04:

From the core list by Christian (thanks !):

Meanwhile the patch evolved to work on Windows and MacOS, too, see

https://www.androwish.org/home/info/b8894bb57964415f