TIP 627: New functions for handling commands > 2^31 elements (for 8.7)

Bounty program for improvements to Tcl and certain Tcl packages.
    Author:        Jan Nijtmans <[email protected]>
    State:         Draft
    Type:          Project
    Vote:          Pending
    Created:       01-06-2022
    Tcl-Version:   8.7
    Keywords:      Tcl
    Tcl-Branch:    tip-627


This TIP proposes 4 new functions:

Those functions do the same as Tcl_CreateObjCommand/Tcl_CreateObjTrace/Tcl_NRCreateCommand/Tcl_NRCallObjProc, but their Tcl_ObjCmdProc * argument is now of type Tcl_ObjCmdProc2 *, and Tcl_CmdObjTraceProc * (for Tcl_CreateObjTrace2) is now of type Tcl_CmdObjTraceProc2 *. Those new types can handle more than 2^31 command line arguments.


The following new functions are defined:

The definitions of the new types are:

typedef int (Tcl_ObjCmdProc2) (ClientData clientData, Tcl_Interp *interp,
	size_t objc, struct Tcl_Obj *const *objv);
typedef int (Tcl_CmdObjTraceProc2) (ClientData clientData, Tcl_Interp *interp,
	int level, const char *command, Tcl_Command commandInfo, size_t objc,
	struct Tcl_Obj *const *objv);
Note that the objc argument changed type from int to size_t.

Those functions allow extensions to register commands or trace functions which can handle more than 2^31 elements. Since the Tcl core cannot handle commands with more than 2^31 elements yet, this - for now - is of limited usability: The core should first be extended to handle this everywhere, which is a massive amount of work. Therefore, I am choosing to submit this part separately, so it can be used in Tcl 8.7 as well. A later TIP #626 (for Tcl 9.0), which makes the actual transition making it really useful, can then decided upon separately. Also, if the API is decided upon, the actual transition could even be delayed until Tcl 9.1: As long as the API stays the same, it is considered an upwards compatible change, no matter how it is handled internally.

Starting with Tcl 9.0, the Tcl_CmdInfo struct gets two additional elements:

typedef struct {
    Tcl_ObjCmdProc2 objProc2;	/ Command's object2-based function. /
    void *objClientData2;	/ ClientData for object2 proc. */
} Tcl_CmdInfo;
After calling the function Tcl_GetCommandInfo(), if Tcl_CreateObjCommand2 was used for registering the command, those fields will contain the original function and clientData used in Tcl_CreateObjCommand2. This change cannot be made in Tcl 8.7, since Tcl_CmdInfo is usually allocated on the C-stack: Enlarging a struct size (and writing the additional fields) is not considered binary compatible. In Tcl 9.0, binary compatibility is lost anyway, there we can afford this addition. Also - Tcl 9.0 only -, when Tcl_CreateObjCommand2 is used, the isNativeObjectProc will be set to 2. Current code is normally checking isNativeObjectProc for being !=0, in order to see whether objProc/objClientData is usable. That's still true: if isNativeObjectProc is 2, objProc/objClientData will contain wrapper information, resulting in calling objProc2/objClientData2. So, no current code needs to be adapted for Tcl 9.0: The promise of source compatibility is still kept.


See branch tip-627


The proposed change is 100% source and binary compatible with Tcl 8.6, since the new functions are implemented as wrappers around the original functions.


This document has been placed in the public domain.