Tcl Source Code

View Ticket
Login
Ticket UUID: 656fe3c8163aea1bbf10552962fcc79cb49c5bad
Title: Tcl_DictObjPut() - poor usability
Type: Bug Created on: 2019-07-25 16:29:57
Submitter: dgp Assigned to: dgp
Subsystem: 15. Dict Object Severity: Minor
Priority: 6 Last modified: 2024-05-26 14:12:54
Status: Open Closed by: nobody
Resolution: None Closed on:
Version:
Description:
Tcl_DictObjPut(..., dictPtr, keyPtr, valPtr);

Sometimes the dict will store a ref to keyPtr and increment its refcount,
effectively taking responsibility for later release, but not always. Notably,
if an equal key is already in the dict, the old one is kept in preference to
swapping the new one in.

This places more burden on refcounting on the caller, and makes calls like

Tcl_DictObjPut(..., dictPtr, Tcl_NewStringObj(...), valPtr)

unsafe to use -- even though many such examples are in Tcl's own source code.
User Comments:
dgp added on 2019-07-25 16:31:47:
A rule where we always take possession of the new key and swap it in,
and swap out the old (and free it if we're the last holder) sure seems
like it would make calling the routine much friendlier, and I don't see
a significant disadvaantage.

FWIW, Itcl's been leaking memory because it made the wrong assumption.

sebres added on 2019-07-25 17:05:44:

as for "always use new key and swap it ...", may be dangerous depending how it is used in some codes until now... e. g. something like that will segfaults after "fix":

  k = Tcl_NewStringObj(...);
  Tcl_DictObjPut(..., dictPtr, k, ...);  // k->rc = 1
  ...
  k2 = Tcl_NewStringObj(...);
  Tcl_DictObjPut(..., dictPtr, k2, ...); // k2->rc = 1, k->rc = 0 (removed)
  ...
  do_something_with_k(k);                // BOOM

I'll suggest to decrement in case refCount is still 0 after put or even new some functions with flags TCL_OWN_OBJREF like in my proposal to fix [578155d5a1] .


dkf added on 2024-05-25 08:52:59:

This definitely sounds like something for Tcl_BounceRefCount to me.


dkf added on 2024-05-25 12:44:56:

The branch more-dict-c-funcs has some ideas on this.

Branch Description

The branch has four new functions:

int	TclDictGet(Tcl_Interp *interp, Tcl_Obj *dictPtr,
	    const char *key, Tcl_Obj **valuePtrPtr);
int	TclDictPut(Tcl_Interp *interp, Tcl_Obj *dictPtr,
	    const char *key, Tcl_Obj *valuePtr);
int	TclDictPutString(Tcl_Interp *interp, Tcl_Obj *dictPtr,
	    const char *key, const char *value);
int	TclDictRemove(Tcl_Interp *interp, Tcl_Obj *dictPtr,
	    const char *key);

The functions make those operations available without having to explicitly make a key object (and a value object from a classic NUL-terminated C string, for TclDictPutString()) and handle reference counts correctly so that the objects they make internally won't leak. TclDictPut() also will dispose of a zero-ref valuePtr on error; the valuePtr must be distinct from the dictPtr (it always is in our internal uses; the reality is that the Put functions are almost always used with a NULL interpreter and a dictionary that's never been exposed to user scripts). It might be nice to have TclDictPutInt() or something like that — we could certainly make use of it! — but it isn't critical.

They're currently MODULE_SCOPE but bringing them out to tclInt.decls would be acceptable (or to tcl.decls with the obvious name change; a public API would need a TIP but I wouldn't see that as a problem, as I'm sure I have the API reasonably correct).

The branch includes uses of those functions. Sometimes this results in quite a bit more readable code (especially in TclDefaultBgErrorHandlerObjCmd() in tclEvent.c).

I've no plans to do a varargs TclDictGet; I'm not convinced that would be worth it.


dkf added on 2024-05-25 12:49:53:

Note that cases where we've got manual memoization of keys have been left alone. I simply didn't examine whether it was worth converting them.

Also, I've done leak testing with make test with --enable-symbols=mem (on Windows with gcc).


dkf added on 2024-05-26 14:12:54:

Those functions now on primary branches. Decision about exposure more widely remains to be taken, so leaving this issue open.