TIP 627: New functions for handling commands > 2^31 elements (for 8.7)

    Author:        Jan Nijtmans <[email protected]>
    State:         Final
    Type:          Project
    Vote:          Done
    Created:       01-06-2022
    Tcl-Version:   8.7
    Keywords:      Tcl
    Tcl-Branch:    tip-627
    Vote-Summary   Accepted 3/0/1
    Votes-For:     JN, KBK, KW
    Votes-Against: none
    Votes-Present: FV


This TIP proposes 4 new functions:

Those functions do the same as Tcl_CreateObjCommand/Tcl_CreateObjTrace/Tcl_NRCreateCommand/Tcl_NRCallObjProc, but their Tcl_ObjCmdProc * argument is now of type Tcl_ObjCmdProc2 *, and Tcl_CmdObjTraceProc * (for Tcl_CreateObjTrace2) is now of type Tcl_CmdObjTraceProc2 *. Those new types can handle more than 2^31 command line arguments when used in Tcl 9.0.


The following new functions are defined:

The definitions of the new types are:

typedef int (Tcl_ObjCmdProc2) (ClientData clientData, Tcl_Interp *interp,
	Tcl_Size objc, struct Tcl_Obj *const *objv);
typedef int (Tcl_CmdObjTraceProc2) (ClientData clientData, Tcl_Interp *interp,
	Tcl_Size level, const char *command, Tcl_Command commandInfo, Tcl_Size objc,
	Tcl_Obj *const *objv);
Note that the objc argument changed type from int to Tcl_Size.

Those functions allow extensions to register commands or trace functions which can handle more than 2^31 elements. Since the Tcl core cannot handle commands with more than 2^31 elements yet, this - for now - is of limited usability: The core should first be extended to handle this everywhere, which is a massive amount of work. Therefore, I am choosing to submit this part separately, so it can be used in Tcl 8.7 as well. A later TIP #626 (for Tcl 9.0), which makes the actual transition making it really useful, can then decided upon separately. Also, if the API is decided upon, the actual transition could even be delayed until Tcl 9.1: As long as the API stays the same, it is considered an upwards compatible change, no matter how it is handled internally.

Starting with Tcl 9.0, the Tcl_CmdInfo struct gets two additional elements:

typedef struct {
    Tcl_ObjCmdProc2 objProc2;	/ Command's object2-based function. /
    void *objClientData2;	/ ClientData for object2 proc. */
} Tcl_CmdInfo;
After calling the function Tcl_GetCommandInfo(), if Tcl_CreateObjCommand2 was used for registering the command, those fields will contain the original function and clientData used in Tcl_CreateObjCommand2. This change cannot be made in Tcl 8.7, since Tcl_CmdInfo is usually allocated on the C-stack: Enlarging a struct size (and writing the additional fields) is not considered binary compatible. In Tcl 9.0, binary compatibility is lost anyway, there we can afford this addition. Also - Tcl 9.0 only -, when Tcl_CreateObjCommand2 is used, the isNativeObjectProc will be set to 2. Current code is normally checking isNativeObjectProc for being !=0, in order to see whether objProc/objClientData is usable. That's still true: if isNativeObjectProc is 2, objProc/objClientData will contain wrapper information, resulting in calling objProc2/objClientData2. So, no current code needs to be adapted for Tcl 9.0: The promise of source compatibility is still kept.


After TIP #665 was accepted, a lot of functions changed from using size_t to Tcl_Size parameters. In order to prevent confusion, this change has been adapted in the TIP text above as well.


See branch tip-627


The proposed change is 100% source and binary compatible with Tcl 8.6, since the new functions are implemented as wrappers around the original functions.


This document has been placed in the public domain.