| Ticket UUID: | 1b81ff43a2592e6e8de2d18e2186101cc88c2294 | |||
| Title: | tk doesn't support font shaping on unix, required for some scripts to be read properly | |||
| Type: | Bug | Version: | 9.1 | |
| Submitter: | anonymous | Created on: | 2020-11-29 13:52:21 | |
| Subsystem: | 46. Unix Fonts | Assigned To: | kevin_walzer | |
| Priority: | 5 Medium | Severity: | Important | |
| Status: | Closed | Last Modified: | 2026-05-20 13:10:42 | |
| Resolution: | Fixed | Closed By: | kevin_walzer | |
| Closed on: | 2026-05-20 13:10:42 | |||
| Description: |
Font handling in xlib library doesn't support shaping, as it is required to read many scripts (such as arabic). also, BiDi required to read Arabic (and herbew) in the correct way, both scripts aren't readable because of not implemeting BiDi in Unix. please note that they are done in many other OSes natively (in mac os and windows). but as Xlib is such low level layer to implement such features. most other widget toolkits (such as gtk+ and qt) use their own code to achieve such effects. also recently harfbuzz appeared as easy and brief library that do shaping for many scripts. Again, arabic can't be read (or written) without shaping (and BiDi). thanks | |||
| User Comments: |
kevin_walzer added on 2026-05-20 13:10:42:
After additional questions and discussion on Tcl-Core raised no objections, and the test suite passes on all platforms, I am merging the rtl_branch in https://core.tcl-lang.org/tk/info/8ce823d20c81448c. Thanks to all for their input. kevin_walzer added on 2026-05-15 13:32:27: Testing can easily be done with the new bidi text demo in the widget demo. kevin_walzer added on 2026-05-15 02:31:03: Re-opening this ticket because of significant additional work on bidi text into the rtl_text branch that would benefit from user review.
Please add any additional notes to this ticket. kevin_walzer added on 2026-03-27 23:40:42: tk_bidi branch merged into trunk with commit 058e738f. kevin_walzer added on 2026-03-23 20:56:06: Please try the latest commit. It has some additional optimization. kjnash added on 2026-03-23 17:42:54: I've just tried this out on x86_64 Linux (Alma 8), building Tk 4eec40aa40 with Tcl 9.1a1. $ /usr/local/Tcl-9.1/bin/wish9.1 biditest.tcl X Error of failed request: BadLength (poly request too large or internal Xlib length error) Major opcode of failed request: 139 (RENDER) Minor opcode of failed request: 20 (RenderAddGlyphs) Serial number of failed request: 636 Current serial number in output stream: 667 $ An interactive wish shell could be opened; sourcing biditest.tcl gave a similar X failure. I tried with just the text-widget part of biditest.tcl, reducing the text widget path to ".t". The command ".t mark set insert 1.7" caused the first line to split into two as if a newline had been inserted (though not at 1.7, the split was in the middle of the next word). The display corrected itself if the insert mark was moved to a different line. Clicking in the text widget moved the insert mark but also caused Tk to hang and consume 8 GB of memory. The host became so unresponsive that the only remedy was to switch off the power. kevin_walzer added on 2026-03-23 14:14:06: I have implemented changes that now make X11 and Windows first-class citizens on Tk for bidirectional text and complex script shaping, bringing them into alignment with macOS. This means that RTL languages are now presented in Right-to-Left format, and their glyphs are now "shaped" into smooth script according to those langauges' conventions rather than being drawn one glyph at a time, which is not legible. macOS has the most modern, advanced text engine of Tk's platforms: Core Text. Core Text provides full support for complex RTL languages such as Arabic and Hebrew out of the box, so Tk has long had this support "for free" on macOS. See the bidi_macos.png attachment to see what this looks like. Windows and X11 have older text engines: GDI on Windows and Xft on X11. Neither supports RTL languages out of the box, so additional work has been needed to update these platforms. Windows does have a library that adds RTL support to GDI text: Uniscribe. I added this library to the win/tkWinFont.c file to implement bidirectional rendering and text shaping. See the bidi_windows.png attachment to see what this looks like. Linux has various libraries to support RTL languages. The most common stack is Harfbuzz for text shaping and FriBidi for bidirectional processing. These libraries are very capable and well-tested, but adding them would add large external dependencies to Tk. It was preferable to look for smaller libraries that would accomplish the same objective and which could be directly integrated with Tk. I found two: kb_text_shape.h for text shaping (zlib license) and SheenBidi for bidirectional rendering (Apache license). These libraries are easy to integrate with Tk's source tree and build system, have permissive licenses, and most importantly, do a good job. I hope this stack becomes a widely-used combination on X11. I added the support from these libraries to unix/tkUnixRFont.c. See the bidi_x11.png attachment to see what this looks like. My work on RTL languages is included in the tk_bidi branch. Please review and provide any comments as soon as possibsle. Since these updates are all internal and do not present any user-visible API changes, I want to merge this work as a bug fix in time for 9.1. kevin_walzer added on 2026-03-11 04:23:17: Reviewing options to address. | |||
Attachments:
- biditest.tcl [download] added by kevin_walzer on 2026-03-23 14:17:48. [details]
- bidi_x11.png [download] added by kevin_walzer on 2026-03-23 13:42:32. [details]
- bidi_windows.png [download] added by kevin_walzer on 2026-03-23 13:42:06. [details]
- bidi_macos.png [download] added by kevin_walzer on 2026-03-23 13:41:37. [details]
