Check-in [b092de73ae]

Login
Home Timeline Branches Tags Help
Bounty program for improvements to Tcl and certain Tcl packages.

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:New TIP #548: Deprecate Tcl_WinUtfToTChar() and Tcl_WinTCharToUtf() and provide more flexible replacement functions.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: b092de73ae97a11610b975fd54e139035c3e022ae5e169789b50daf7c0d7160f
User & Date: jan.nijtmans 2019-06-03 14:41:54
Context
2019-06-03
15:14
More TIP #548 explanation. check-in: 0220e14300 user: jan.nijtmans tags: trunk
14:41
New TIP #548: Deprecate Tcl_WinUtfToTChar() and Tcl_WinTCharToUtf() and provide more flexible replacement functions. check-in: b092de73ae user: jan.nijtmans tags: trunk
2019-05-31
13:53
TIP's 537 and 544 are final now. check-in: 558b3ef217 user: jan.nijtmans tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to index.json. 

1

2
3
4
5
6
7
8
...
544
545
546
547
548
549
550
551
552

{"tip": {

	"547":{"url":"./tip/547.md","keywords":"Tcl","created":"31-May-2019","post-history":"","state":"Draft","tcl-branch":"tip-547","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 547: New encodings: UTF-16, UCS-2.","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","Jan Nijtmans <[email protected]>"],"is-jest":false},
	"546":{"url":"./tip/546.md","keywords":"Tcl, traces","created":"20-May-2019","post-history":"","state":"Draft","vote":"Pending","tcl-version":"8.7","type":"Project","title":"# TIP 546: Typedefs to Support Source-Compatible Migration","author":["Don Porter <[email protected]>"],"is-jest":false},
	"545":{"url":"./tip/545.md","keywords":"Tk, image","created":"19-May-2019","state":"Draft","tk-branch":"tip-545-svg-options","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 545: image photo SVG options: remove -unit and add target height/width option","author":["Harald Oehlmann <[email protected]>"],"is-jest":false},
	"544":{"url":"./tip/544.md","keywords":"Tcl","created":"15-May-2019","post-history":"","state":"Final","tcl-branch":"tip-544","vote":"Done","tcl-version":"8.7","type":"Project","title":"# TIP 544: Export TclGetIntForIndex()","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","François Vogel <[email protected]>"],"is-jest":false},
	"543":{"url":"./tip/543.md","keywords":"Tcl, traces","created":"10-May-2019","post-history":"","state":"Draft","vote":"Pending","tcl-version":"9.0","type":"Project","title":"# TIP 543: Eliminate TCL\\_INTERP\\_DESTROYED flag value","author":["Don Porter <[email protected]>"],"is-jest":false},
	"542":{"url":"./tip/542.md","keywords":"Tcl","created":"10-May-2019","post-history":"","state":"Draft","tcl-branch":"utf-max","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 542:Support for switchable (Androwish-compatible) Full Unicode support.","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","Jan Nijtmans <[email protected]>"],"is-jest":false},
	"541":{"url":"./tip/541.md","keywords":"Tcl","created":"8-May-2019","post-history":"","state":"Final","tcl-branch":"bug-2858503fff","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 541: Add support for 'end' index in ttk::combobox current","author":["Francois Vogel <[email protected]>"],"is-jest":false},
................................................................................
	"5":{"url":"./tip/5.md","created":"17-Oct-2000","post-history":"","state":"Final","vote":"Done","tcl-version":"8.4","type":"Project","title":"# TIP 5: Make TkClassProcs and TkSetClassProcs Public and Extensible","author":["Eric Melski <[email protected]>"],"is-jest":false},
	"4":{"url":"./tip/4.md","created":"26-Oct-2000","post-history":"","state":"Draft","vote":"Pending","type":"Informative","title":"# TIP 4: Tcl Release and Distribution Philosophy","discussions-to":"news:comp.lang.tcl","author":["Brent Welch <[email protected]>","Donal K. Fellows <[email protected]>","Larry W. Virden <[email protected]>","Larry W. Virden <[email protected]>"],"is-jest":false},
	"3":{"url":"./tip/3.md","created":"14-Sep-2000","obsoleted-by":"467","post-history":"","state":"Accepted","vote":"Done","type":"Process","title":"# TIP 3: TIP Format","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>"],"is-jest":false},
	"2":{"url":"./tip/2.md","created":"12-Sep-2000","post-history":"","state":"Draft","vote":"Pending","type":"Process","title":"# TIP 2: TIP Guidelines","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>","Don Porter <[email protected]>","Mo DeJong <[email protected]>","Larry W. Virden <[email protected]>","Kevin Kenny <[email protected]>"],"is-jest":false},
	"1":{"url":"./tip/1.md","created":"14-Sep-2000","obsoleted-by":"386","post-history":"","state":"Withdrawn","vote":"No voting","type":"Informational","title":"# TIP 1: TIP Index","author":["TIP Editor <[email protected]>"],"is-jest":false},
	"0":{"url":"./tip/0.md","created":"11-Dec-2000","post-history":"","state":"Final","vote":"Done","type":"Process","title":"# TIP 0: Tcl Core Team Basic Rules","author":["John Ousterhout <[email protected]>"],"is-jest":false},
	"@min": 0,
	"@max": 547
}, "@timestamp": 1559310652}

>







 







|
|

1
2
3
4
5
6
7
8
9
...
545
546
547
548
549
550
551
552
553

{"tip": {
	"548":{"url":"./tip/548.md","keywords":"Tcl","created":"3-June-2019","post-history":"","state":"Draft","tcl-branch":"tip-548","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 548:Deprecate Tcl\\_WinUtfToTChar() and Tcl\\_WinTCharToUtf() and provide more flexible replacement functions.","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","Jan Nijtmans <[email protected]>"],"is-jest":false},
	"547":{"url":"./tip/547.md","keywords":"Tcl","created":"31-May-2019","post-history":"","state":"Draft","tcl-branch":"tip-547","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 547: New encodings: UTF-16, UCS-2.","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","Jan Nijtmans <[email protected]>"],"is-jest":false},
	"546":{"url":"./tip/546.md","keywords":"Tcl, traces","created":"20-May-2019","post-history":"","state":"Draft","vote":"Pending","tcl-version":"8.7","type":"Project","title":"# TIP 546: Typedefs to Support Source-Compatible Migration","author":["Don Porter <[email protected]>"],"is-jest":false},
	"545":{"url":"./tip/545.md","keywords":"Tk, image","created":"19-May-2019","state":"Draft","tk-branch":"tip-545-svg-options","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 545: image photo SVG options: remove -unit and add target height/width option","author":["Harald Oehlmann <[email protected]>"],"is-jest":false},
	"544":{"url":"./tip/544.md","keywords":"Tcl","created":"15-May-2019","post-history":"","state":"Final","tcl-branch":"tip-544","vote":"Done","tcl-version":"8.7","type":"Project","title":"# TIP 544: Export TclGetIntForIndex()","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","François Vogel <[email protected]>"],"is-jest":false},
	"543":{"url":"./tip/543.md","keywords":"Tcl, traces","created":"10-May-2019","post-history":"","state":"Draft","vote":"Pending","tcl-version":"9.0","type":"Project","title":"# TIP 543: Eliminate TCL\\_INTERP\\_DESTROYED flag value","author":["Don Porter <[email protected]>"],"is-jest":false},
	"542":{"url":"./tip/542.md","keywords":"Tcl","created":"10-May-2019","post-history":"","state":"Draft","tcl-branch":"utf-max","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 542:Support for switchable (Androwish-compatible) Full Unicode support.","discussions-to":"Tcl Core list","author":["Jan Nijtmans <[email protected]>","Jan Nijtmans <[email protected]>"],"is-jest":false},
	"541":{"url":"./tip/541.md","keywords":"Tcl","created":"8-May-2019","post-history":"","state":"Final","tcl-branch":"bug-2858503fff","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 541: Add support for 'end' index in ttk::combobox current","author":["Francois Vogel <[email protected]>"],"is-jest":false},
................................................................................
	"5":{"url":"./tip/5.md","created":"17-Oct-2000","post-history":"","state":"Final","vote":"Done","tcl-version":"8.4","type":"Project","title":"# TIP 5: Make TkClassProcs and TkSetClassProcs Public and Extensible","author":["Eric Melski <[email protected]>"],"is-jest":false},
	"4":{"url":"./tip/4.md","created":"26-Oct-2000","post-history":"","state":"Draft","vote":"Pending","type":"Informative","title":"# TIP 4: Tcl Release and Distribution Philosophy","discussions-to":"news:comp.lang.tcl","author":["Brent Welch <[email protected]>","Donal K. Fellows <[email protected]>","Larry W. Virden <[email protected]>","Larry W. Virden <[email protected]>"],"is-jest":false},
	"3":{"url":"./tip/3.md","created":"14-Sep-2000","obsoleted-by":"467","post-history":"","state":"Accepted","vote":"Done","type":"Process","title":"# TIP 3: TIP Format","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>"],"is-jest":false},
	"2":{"url":"./tip/2.md","created":"12-Sep-2000","post-history":"","state":"Draft","vote":"Pending","type":"Process","title":"# TIP 2: TIP Guidelines","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>","Don Porter <[email protected]>","Mo DeJong <[email protected]>","Larry W. Virden <[email protected]>","Kevin Kenny <[email protected]>"],"is-jest":false},
	"1":{"url":"./tip/1.md","created":"14-Sep-2000","obsoleted-by":"386","post-history":"","state":"Withdrawn","vote":"No voting","type":"Informational","title":"# TIP 1: TIP Index","author":["TIP Editor <[email protected]>"],"is-jest":false},
	"0":{"url":"./tip/0.md","created":"11-Dec-2000","post-history":"","state":"Final","vote":"Done","type":"Process","title":"# TIP 0: Tcl Core Team Basic Rules","author":["John Ousterhout <[email protected]>"],"is-jest":false},
	"@min": 0,
	"@max": 548
}, "@timestamp": 1559572862}

Changes to index.md. 

203
204
205
206
207
208
209








210
211
212
213
214
215
216

<th>Type</th>
<th>Tcl Version</th>
<th>Status</th>
<th>Title</th>
<th>Impl.</th>
</tr></thead><tbody>









<tr class='state-draft type-project version-87'>
<td valign='top'><a href='./tip/547.md'>547</a></td>
<td valign='top'>Project</td>
<td valign='top'>8.7</td>
<td valign='top'>Draft</td>
<td valign='top' ><a href='./tip/547.md'>New encodings: UTF-16, UCS-2.</a></td>
<td valign='top'><a href='/tcl/timeline?r=tip-547'>Link</a></td>







>
>
>
>
>
>
>
>








203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224

<th>Type</th>
<th>Tcl Version</th>
<th>Status</th>
<th>Title</th>
<th>Impl.</th>
</tr></thead><tbody>

<tr class='state-draft type-project version-87'>
<td valign='top'><a href='./tip/548.md'>548</a></td>
<td valign='top'>Project</td>
<td valign='top'>8.7</td>
<td valign='top'>Draft</td>
<td valign='top' ><a href='./tip/548.md'>Deprecate Tcl\_WinUtfToTChar() and Tcl\_WinTCharToUtf() and provide more flexible replacement functions.</a></td>
<td valign='top'><a href='/tcl/timeline?r=tip-548'>Link</a></td>
</tr>
<tr class='state-draft type-project version-87'>
<td valign='top'><a href='./tip/547.md'>547</a></td>
<td valign='top'>Project</td>
<td valign='top'>8.7</td>
<td valign='top'>Draft</td>
<td valign='top' ><a href='./tip/547.md'>New encodings: UTF-16, UCS-2.</a></td>
<td valign='top'><a href='/tcl/timeline?r=tip-547'>Link</a></td>

Changes to tip/542.md. 

51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
..
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92


93
94
95
96
97
98
99
100
101

   In the latter mode, the Tcl_UniChar type becomes a 32-bit type, but the stub entries for
   the 16-bit Tcl\_UniChar type are present as well. So, most extensions compiled with -DTCL\_UTF\_MAX=3
   will continue to work in either Tcl mode (for caveats, see below).

 * Allow Tcl extensions to be compiled with either -DTCL\_UTF\_MAX=3 (default), or with -DTCL\_UTF\_MAX=4,
   when Tcl is compiled with -DTCL\_UTF\_MAX=4.

 * Enhance the Tcl\_UniCharToUtfDString() function such that the uniLength parameter is allowed to
   have the value -1.
 
 * Deprecate the following functions:

     Tcl\_AppendUnicodeToObj()

     Tcl\_GetUnicode()

     Tcl\_GetUnicodeFromObj()
................................................................................
     Tcl\_UniCharNcmp()

     Tcl\_UniCharNcasecmp()

   If Tcl is compiled with either -DTCL\_UTF\_MAX=4 or -DTCL\_NO\_DEPRECATED, those functions will no longer be available for extensions, as well as in Tcl 9.0.


 * New functions:
     
     Tcl\_Utf16ToUtfDString()
     
     Tcl\_UtfToUtf16DString()

     Tcl\_UtfToUtf16()

     Those are the same as the _UniChar_ variants, but they use the "unsigned short" type in stead of Tcl\_UniChar.



     Those functions can be used if you want your extension to compile with either -DTCL\_UTF\_MAX=3 or -DTCL\_UTF\_MAX=4,
     but still want to use the 16-bit conversions independent on the TCL\_UTF\_MAX setting or Tcl\_UniChar type.

# Compatibility

As long as Tcl is compiled with -DTCL\_UTF\_MAX=3, this is fully upwards compatible.

When Tcl is compiled with -DTCL\_UTF\_MAX=4, this is at the Tcl level, compatible with the Androwish-version







<
<
<







 







|

<
<
<
<


<
>
>

|








51
52
53
54
55
56
57



58
59
60
61
62
63
64
..
74
75
76
77
78
79
80
81
82




83
84

85
86
87
88
89
90
91
92
93
94
95

   In the latter mode, the Tcl_UniChar type becomes a 32-bit type, but the stub entries for
   the 16-bit Tcl\_UniChar type are present as well. So, most extensions compiled with -DTCL\_UTF\_MAX=3
   will continue to work in either Tcl mode (for caveats, see below).

 * Allow Tcl extensions to be compiled with either -DTCL\_UTF\_MAX=3 (default), or with -DTCL\_UTF\_MAX=4,
   when Tcl is compiled with -DTCL\_UTF\_MAX=4.




 * Deprecate the following functions:

     Tcl\_AppendUnicodeToObj()

     Tcl\_GetUnicode()

     Tcl\_GetUnicodeFromObj()
................................................................................
     Tcl\_UniCharNcmp()

     Tcl\_UniCharNcasecmp()

   If Tcl is compiled with either -DTCL\_UTF\_MAX=4 or -DTCL\_NO\_DEPRECATED, those functions will no longer be available for extensions, as well as in Tcl 9.0.


 * New function:
     




     Tcl\_UtfToUtf16()


     This function does almost the same as Tcl\_UtfToUniChar(), but it writes 16-bit Unicode Characters ("unsigned short") independent
     of the value of -DTCL\_UTF\_MAX.

     This function can be used if you want your extension to compile with either -DTCL\_UTF\_MAX=3 or -DTCL\_UTF\_MAX=4,
     but still want to use the 16-bit conversions independent on the TCL\_UTF\_MAX setting or Tcl\_UniChar type.

# Compatibility

As long as Tcl is compiled with -DTCL\_UTF\_MAX=3, this is fully upwards compatible.

When Tcl is compiled with -DTCL\_UTF\_MAX=4, this is at the Tcl level, compatible with the Androwish-version

Added tip/548.md. 












































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

# TIP 548:Deprecate Tcl\_WinUtfToTChar() and Tcl\_WinTCharToUtf() and provide more flexible replacement functions.
	Author:         Jan Nijtmans <[email protected]>
	Author:         Jan Nijtmans <[email protected]>
	State:          Draft
	Type:           Project
	Vote:           Pending
	Created:        3-June-2019
	Post-History:   
	Discussions-To: Tcl Core list
	Keywords:       Tcl
	Tcl-Version:    8.7
	Tcl-Branch:     tip-548
-----

# Abstract

This TIP proposes to deprecate Tcl\_WinUtfToTChar() and Tcl\_WinTCharToUtf() and provide more flexible replacement functions.

# Rationale

The functions Tcl\_WinUtfToTChar() and Tcl\_WinTCharToUtf() originally were functions able to do two different conversions,
depending on the runtime platform: On Windows 95/98/ME they performed conversions between Utf-8 and the Windows default encoding
(usually cp1252), on later Windows versions they convert between Utf-8 and Utf-16. The length parameter of Tcl\_WinTCharToUtf()
always was in bytes, but most other Unicode-related Tcl functions expect their length in Unicode characters.

Since Windows 95/98/ME are not supported any more, it's time to fix this inconsistency.

# Specification

This document proposes:

 * Deprecate the following functions:

     Tcl\_WinUtfToTChar()

     Tcl\_WinTCharToUtf()

   If Tcl is compiled with either -DTCL\_UTF\_MAX=6 (which is not officially supported) or -DTCL\_NO\_DEPRECATED, those functions will
   no longer be available for extensions. In Tcl 9.0, those functions will be completely removed.

 * New functions:
     
     Tcl\_Utf16ToUtfDString()
     
     Tcl\_UtfToUtf16DString()

     Those are the same as the _UniChar_ variants (Tcl\_UniCharToUtfDString/Tcl\_UtfToUniCharDString), but they use an "unsigned short"
     pointer type in their signature stead of a Tcl\_UniChar pointer type.
     In addition, Tcl\_Utf16ToUtfDString() now accepts the value -1 as length parameter.

     Those functions can be used if you want your extension to compile with -DTCL\_UTF\_MAX=3, -DTCL\_UTF\_MAX=4 or -DTCL\_UTF\_MAX=6,
     but still want to use the 16-bit conversions independent on the TCL\_UTF\_MAX setting or Tcl\_UniChar type.

 * Enhance the Tcl\_UniCharToUtfDString() function such that the uniLength parameter is allowed to
   have the value -1.

# Compatibility

This is fully upwards compatible in Tcl 8.x, except if Tcl is compiled with -DTCL\_UTF\_MAX=6 (not officially supported) or
-DTCL\_NO\_DEPRECATED. Starting with Tcl 9.0, the replacement functions should be used in stead.

# Reference Implementation

A reference implementation is available in  the **tip-548** branch.
<https://core.tcl.tk/tcl/timeline?r=tip-548>

# Copyright

This document has been placed in the public domain.