Tcl Source Code

View Ticket
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.
Ticket UUID: cc1e91552c0ca1817292d951f8c694b28c2b2251
Title: Regression in list expansion
Type: Bug Version: 8.6.9
Submitter: max Created on: 2019-01-07 17:00:06
Subsystem: 14. List Object Assigned To: jan.nijtmans
Priority: 5 Medium Severity: Minor
Status: Closed Last Modified: 2019-01-20 19:20:19
Resolution: Fixed Closed By: fvogel
    Closed on: 2019-01-20 19:20:19
  *  tcl-8.6.8 and earlier:
% list [list {*}[set a " "]]

  *  tcl-8.6.9:
% list [list {*}[set a " "]]
{ }

This breaks some 20 tests in the SQLite test suite.
User Comments: fvogel added on 2019-01-20 19:20:19:
Apologies, please just forget about what I wrote below, it works. Not sure why I said otherwise. My repo must not have been up-to-date.

fvogel added on 2019-01-20 19:17:02:

Could you please revisit this?

I still see the problem at least with core-8-branch (didn't check with trunk or any other branch).

It appears that the fix [58c46e74b931d3a1] that I confirmed to work got changed later (see [e94a8f9c8f9a6950]) and this changed fix is what was merged. However it does not work for me.

sebres added on 2019-01-11 16:37:49:

So since [1671ba2d5c] the second regression (crash) is also covered now...

Everybody has now new test-facility "testpurebytesobj" allowing creation of pure bytes object without internal representations (NULL's).

Using this it's pretty simple to cover similar things, as well as to reproduce the crash, noticed by François (of course, only on branch before the Jan's fix [58c46e74b931d3a1]):

load tcltest86 Tcltest
lrange [testpurebytesobj] 0 1

sebres added on 2019-01-11 15:25:07:

> Your assumption that TclListObjGetElements() always results in a list object is wrong: In case of an empty string representation the empty string is returned.

You are right Jan, Thx.

#define TclListObjGetElements(interp, listPtr, objcPtr, objvPtr) \
    (((listPtr)->typePtr == &tclListType) \
	    ? ((ListObjGetElements((listPtr), *(objcPtr), *(objvPtr))), TCL_OK)\
	    : Tcl_ListObjGetElements((interp), (listPtr), (objcPtr), (objvPtr)))
How one can see here, the object either already a list or (my assumtion was) it will be to list immeadiatelly after invocation of Tcl_ListObjGetElements.

Just I really overlooked this block in scope it is not a list - Tcl_ListObjGetElements.

Thank you again. I'll try to construct a test-case for exacly this situation.

oehhar added on 2019-01-11 14:57:33:

For the records, here is the post by Francois reporting the regression:

I'm trying to run the Tk test suite with the tip of core-8-6-branch in both Tcl and Tk repositories.

Observation: tktest.exe crashes because it fails in loading the tcl86tg.dll

This happens all with the tip of core-8-6-branch of Tk, i.e.:

It crashes with the tip of Tcl core-8-6-branch, that is with:

but this same tktest.exe does not crash when trying to load the Tcl dll built from the commit just before in the Tcl repository, i.e. with:

Looking at the apparently faulty commit [9d24a1b9db1ebb45] I have no idea about the reason for this failure to load the Tcl dll.

Is anyone seeing the cause perhaps?

jan.nijtmans added on 2019-01-11 14:36:57:

Well, I managed to reproduce the problem. Fix committed here: [58c46e74b931d3a1].

Your assumption that TclListObjGetElements() always results in a list object is wrong: In case of an empty string representation the empty string is returned.

sebres added on 2019-01-11 12:07:02:

Well, actually I doubt that the problem of François would be caused directly by this fix (possibly just an after-effect), because if I'm not wrong it can crash in two cases:

1. in call of ListObjIsCanonical, but it is a nonsense, because either valuePtr is inaccessible and don't think so, because TclListObjGetElements above by retrieving of list length (objc), or by check of the canonical flag (canonicalFlag) of list and it would be even weird, even because TclListObjGetElements always convert valuePtr to the list (tclListType) with proper list representation.

2. within goto, but it is also hard to imagine (below INST_LIST_RANGE_IMM does this also for several cases (e. g. if indices are out of range).

So my assumption there are the possible after-effects like:

- Ref-Counting issues, for example now it returns new object (refCount=0), where previously refCount was >=1. In this case the Tk miss the refCount increment somewhere.

- Type-safety like conversion to another type in the in-between operations (hard to imagine, now as well as previously it is/was always a list in this case).

- Another UB.

Anyway, I'll try to find the reason.

@François, is there any test-case I could use to reproduce the crash? Can you possibly provide a call-stack by the segfault.

jan.nijtmans added on 2019-01-11 08:06:56:

The fix commit appeared to cause a regression, reported by François Vogel. I'm wondering if the *goto* is correct at all. Therefore, reopening.

sebres added on 2019-01-09 10:13:35:
merged in all branches >= 8.6

sebres added on 2019-01-08 13:40:28:

Fixed now in [e364d7f9ffcd2827], branch bug-cc1e91552c. I'll merge it later if tested and no objections follow.

max added on 2019-01-08 10:51:28:

Bisecting core-8-6-branch turned up [c9318a708c] as having caused the regression.

Also, kbk found out that only the bytecoded list command is affected by the regression, but not the interpreted one:

% list [list {*}[set a " "]]
{ }
% set c list
% list [$c {*}[set a " "]]