Author: Jan Nijtmans <[email protected]>
State: Final
Type: Project
Vote: Done
Created: 29-Dec-2017
Post-History:
Keywords: tcl
Tcl-Version: 9.0
Tcl-Branch: memory-API
Abstract
This TIP describes the non-controversial part of the Tcl 9 changes: Make Tcl 9 ready for the 64-bit era.
Rationale
int
is the type of many Tcl API function arguments and return values, and also of many struct fields, but it doesn't provide sufficient room on 64-bit platforms.
Proposal
For hash functions, replace
unsigned int
withsize_t
as the type of the hash value, which allows hash tables to grow beyond 4Gb on 64-bit platforms.Make
size_t
the type of all struct fields representing reference counts or epochs.Change the type of the
size
parameter of each memory-related function, e.g.Tcl_Alloc()
, fromint
tosize_t
and change thechar *
argument tovoid *
.Change the type of the
size
parameter of many functions which fromint
(but NOTint *
) tosize_t
.Change the type of all
ClientData
argumentsvoid *
. This is actually the same type, but it prevents the need for type casts in some situations.Provide a compilation option,
-D_TCL_8_COMPAT
, which provides full source compatibility. Using this option, extensions compile and run fine using either Tcl 8 or Tcl 9 headers. See explanation below.Provide two new macros,
TCL_IO_FAILURE
andTCL_AUTO_LENGTH
, each defined as((size_t)-1)
, to help extensions use the full 64-bit range with Tcl 9, but still compile with Tcl 8 as well (see below). Provide those macros in Tcl 8.7 too, but defined as(-1)
.Modify
Tcl_Alloc()
,Tcl_Free()
, and related functions to redirect to their debugging variants ifTCL_MEM_DEBUG
is defined:Tcl_Alloc()
becomes the same asckalloc()
, ending the general confusion regarding the difference between those two groups of functions: Starting with Tcl 9.0 there no longer any difference.ckalloc()
and related functions are deprecated starting with 9.0, but there are no plans to remove them, and no deprecation warning will be given if extensions use them.
On 32-bit platforms this is all 100% upwards binary compatible provided no internal API is used, since some internal structs might have incompatible but externally invisible changes.
On 64-bit platforms, those changes cause binary incompatibility. Therefore the TCL_STUB_MAGIC
value needs to change, so extensions compiled using Tcl 9 headers will not load in Tcl 8 and reverse.
Implications
Although those changes are binary-compatible on 32-bit platforms, there is a small potential source incompatibility.
Ten functions whose return type was previously int
are changed to size_t
(for now, see below).
This signed/unsigned change might lead to subtle difference in behavior.
The 10 functions are:
Tcl_Gets()
Tcl_GetsObj()
Tcl_Read()
Tcl_ReadChars()
Tcl_ReadRaw()
Tcl_Write()
Tcl_WriteChars()
Tcl_WriteObj()
Tcl_WriteRaw()
Tcl_Ungets()
If the return value of such a function is directly used for comparison, this
could lead cause the operands to be cast to unsigned int
instead of int
for
the comparison.
If an extension is compiled extension with -D_TCL_8_COMPAT
, those ten functions become wrapper macros that make everything behave as if those functions return Tcl_WideInt
. That's the easiest way to resolve this potential problem.
There are two other ways to make this change, without using the TCL_8_COMPAT
macro:
Add
1
to the left and right hand side of the comparison. E.g. like thisDon't compare to
-1
using<
or>
, but always toTCL_IO_FAILURE
using==
or!=
. E.g. like this
Tk and sqlite are already converted to make full use of TIP #494. Other extensions included with Tcl, e.g. tdbc, are unaffected.
Open issues
There has been discussion regarding whether the return type of these ten functions, and possibly other functions, should instead size_t
or maybe some other type.
A Tcl-specific type Tcl_Size
could also be defined for this. Actually, this TIP provides 2 possibilities: Without -D_TCL_8_COMPAT_
the type is size_t
, and with -D_TCL_8_COMPAT_
it is Tcl_WideInt
.
This gives enough room for experimenting which one is best, maybe it would be better to make it ssize_t
or ptrdiff_t
.
There will certainly be more TIPs in the future which propose additional API changes for 9.0. This one is definitely not the last one. Therefore, this TIP doesn't make a final decision yet what will be the final type returned by those ten functions.
Implementation
The implementation of this TIP can be found in the memory-API branch.
Copyright
This document has been placed in the public domain.