Author: Jan Nijtmans <[email protected]> State: Final Type: Project Vote: Done Tcl-Version: 9.0 Tcl-Branch: tip-616 Vote-Summary: Accepted 3/0/2 Votes-For: JN, KBK, KW Votes-Against: none Votes-Present: FV, SL
This TIP proposes to extend the Tcl API for lists and dicts such that they can handle more than 2^31 elements. Also many functions involved with parsing are extended the same way.
The stub table is extended with 7 new functions:
int Tcl_ListObjGetElements(Tcl_Interp *interp, Tcl_Obj *listPtr, Tcl_Size *objcPtr, Tcl_Obj ***objvPtr) int Tcl_ListObjLength(Tcl_Interp *interp, Tcl_Obj *listPtr, Tcl_Size *lengthPtr) int Tcl_DictObjSize(Tcl_Interp *interp, Tcl_Obj *dictPtr, Tcl_Size *sizePtr) int Tcl_SplitList(Tcl_Interp *interp, const char *listStr, Tcl_Size *argcPtr, const char ***argvPtr) void Tcl_SplitPath(const char *path, Tcl_Size *argcPtr, const char ***argvPtr) Tcl_Obj *Tcl_FSSplitPath(Tcl_Obj *pathPtr, Tcl_Size *lenPtr) int Tcl_ParseArgsObjv(Tcl_Interp *interp, const Tcl_ArgvInfo *argTable, Tcl_Size *objcPtr, Tcl_Obj *const *objv, Tcl_Obj ***remObjv)
Also, wrapper macro's are put around those 7 functions, such that - depending in the size of the "Tcl_Size *" argument (either sizeof(int) or sizeof(ptrdiff_t)) the correct stub entry is called, either the original one or the new one. This way, extensions can decide to continue to use an "int" or change the code to use "Tcl_Size" as variable type, supporting an enhanced range for list/dict's. This is how source compatibility is kept.
Currently, the functions
TCL_ERROR when the
the list/dict has invalid syntax. Starting Tcl 9.0, those function can also
sizePtr points to a variable
int and the list/dict has more than 2^31 elements.
In Tcl 8.7, the same functions are added to the stub table, but there the functions are just wrappers calling the original functions. The goal is to allow the new API to be used in 8.7 too, but without actually being able to use more than 2^31 elements in lists/dicts.
The full list of API changes is here
Implementation is in Tcl branch "tip-616".
There is also a minimal Tcl 8.7 implementation in Tcl branch "tip-616-for-8.7"
There is one change in
Tcl_Token that could cause compiler warnings: the extension of the
numTokens fields, which are now unsigned since they can never be negative.
Comparing those values with signed numbers will result in a compiler warning, even though
there is no problem at all: the code keeps functioning fine. Solution: always compare those
fields with unsigned numbers. A type-cast could be also used to silence this warning.
Even though the API changed to allow larger values, internally not everywhere this additional range can be used. For example, commands still don't accept more than 2^31 elements, so lists larger than that cannot be used as commands. Work will continue to allow that in the future, but it's not done in this TIP yet.
After TIP #660 was accepted, a lot of functions changed from using size_t to ptrdiff_t parameters. In order to prevent confusion, this change has been adapted in the TIP text above as well.
The proposed changes for Tcl 9 are source compatible, but NOT binary compatible: All extensions compiled with Tcl 9 headers will have to be recompiled, but no source-code changes are necessary. The additions for Tcl 8.7 are 100% upwards compatible with Tcl 8.6.
This document has been placed in the public domain.