TIP 548:Deprecate <code>Tcl_WinUtfToTChar()</code> and <code>Tcl_WinTCharToUtf()</code> and provide more flexible replacement functions

Login
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.
Author:         Jan Nijtmans <[email protected]>
Author:         Jan Nijtmans <[email protected]>
State:          Draft
Type:           Project
Vote:           Pending
Created:        3-June-2019
Post-History:   
Discussions-To: Tcl Core list
Keywords:       Tcl
Tcl-Version:    8.7
Tcl-Branch:     tip-548

Abstract

This TIP proposes to deprecate Tcl_WinUtfToTChar() and Tcl_WinTCharToUtf() and provide more flexible replacement functions.

Rationale

The functions Tcl_WinUtfToTChar() and Tcl_WinTCharToUtf() originally were functions able to do two different conversions, depending on the runtime platform: On Windows 95/98/ME they performed conversions between Utf-8 and the Windows default encoding (usually cp1252), on later Windows versions they convert between Utf-8 and Utf-16. The length parameter of Tcl_WinTCharToUtf() always was in bytes, but most other Unicode-related Tcl functions expect their length in Unicode characters.

Since Windows 95/98/ME are not supported any more, it's time to fix this inconsistency.

Specification

This document proposes:

If Tcl is compiled with either -DTCL_UTF_MAX=6 (which is not officially supported) or -DTCL_NO_DEPRECATED, those functions will no longer be available. In Tcl 9.0, those functions will be completely removed.

How to upgrade.

In your extension, replace the call:

 `Tcl_WinUtfToTChar(....., bufPtr)`;

with the following two lines:

 `Tcl_DStringInit(bufPtr);`
 `Tcl_UtfToUtf16DString(....., bufPtr);`

And also, replace:

 `Tcl_WinTCharToUtf(....., bufPtr);`

with the following two lines:

 `Tcl_DStringInit(bufPtr);`
 `Tcl_Utf16ToUtfDString(....., bufPtr);`

If the Tcl_WinTCharToUtf() call originally had a "length" parameter not equal to -1, divide it by 2 (or ... don't multiply it by 2 any more).

Compatibility

This is fully upwards compatible in Tcl 8.x, except if Tcl is compiled with -DTCL_UTF_MAX=6 (not officially supported) or -DTCL_NO_DEPRECATED. Starting with Tcl 9.0, the replacement functions should be used in stead.

Reference Implementation

A reference implementation is available in the tip-548 branch. https://core.tcl-lang.org/tcl/timeline?r=tip-548

Copyright

This document has been placed in the public domain.