| Ticket UUID: | bc7ddc7944ec60be761ad6ee54d31110926f85a8 | ||
| Title: | Error: "can't convert value to type substcode" | ||
| Type: | Bug | Created on: | 2023-08-08 21:36:51 |
| Submitter: | pointsman | Assigned to: | griffin |
| Subsystem: | 69. Other | Severity: | Severe |
| Priority: | 5 Medium | Last modified: | 2023-08-24 17:53:50 |
| Status: | Closed | Closed by: | jan.nijtmans |
| Resolution: | Fixed | Closed on: | 2023-08-24 17:53:50 |
| Version: | trunk | ||
| Description: | ||||
On Windows and Linux I get reproducible a Tcl Error which comes out of [https://core.tcl-lang.org/tcl/file?udc=1&ln=963-965&ci=tip&name=generic%2FtclObj.c] and I'm somewhat certain this should not be this way. Bisecing points to [91c2f411e7]. Since then I see this error: @|can't convert value to type substcode @| while executing @|"foreach file $filesMade { @| if {[file exists $file]} { @| DebugDo 1 {Warn "cleanupTests deleting $file..."} @| catch {file delete -force -- $file} @| ..." @| (procedure "::tcltest::cleanupTests" line 33) (The script place referenced by this error msg is [https://core.tcl-lang.org/tcl/file?udc=1&ln=2535&ci=978b4c8a0a42db53&name=library%2Ftcltest%2Ftcltest.tcl]) This is done only by script code; no extension whatsoever in place. Unfortunately its a lot and involved code needed to trigger that and I didn't succeed to reduce this to a simple code example. I extracted it out of the tcllib context (in which I stumbled over this) but it is still lots of code. I tried some standard means from valgrind over gut feelings until fprintf(stder,"") but can only share a few more things. I seems infact to blow up in the line foreach file $filesMade { At this place $filesMade has an internal representation like (by tcl::unsupported::representation: value is a substcode with a refcount of 74, [...] Simples way to reproduce is: Step 0: Install a build of current Tcl trunk somewhere Step 0.5: Get the tcllib fossil repository: fossil clone https://core.tcl-lang.org/tcllib tcllib.fossil Step 1: Create a test directory, cd into and do fossil open path/to/your/tcllib.fossil tcl9-testarea-rde Step 2: Run tclsh sak.tcl test run -v --shell /path/to/your/tclsh9.0 debug (The tclsh in the cmd line above may be any recent Tcl sh you've in path.) In the output of this you should see the error. | ||||
| User Comments: | ||||
griffin added on 2023-08-10 01:59:23:
The problem is in SetDuplicatePureObj. A source Obj with an empty value is passed in, but it appears it is a reused Obj that still has it's typePtr set. In this case, it is a type that does not have a SetFromAny method, consequently causing the Convert operation to fail. But, since the value is the Empty string, a conversion isn't necessary. The bug branch [bug-bc7ddc7944] has a proposed fix. I was not able to track down the cause of the typePtr having a value for a "new", "empty" (pure?) Obj. This part seems sketchy. apnadkarni added on 2023-08-10 03:06:51:
The problem is also fixed by reverting the use of TclDuplicatePureObj back to the original Tcl_DuplicateObj. I have had concerns with the introduction of TclDuplicatePureObj that has led to multiple hard to track down bugs in seemingly unrelated code. And on a cursory look, I do not see any actual benefit compared to the code prior to its introduction. (In theory, it is a "generalization" of the original list representation copying code to all Tcl_Obj but it does not seem to be used anywhere except for lists which makes the generalization not very useful.) Any opinion on reverting its use? griffin added on 2023-08-10 15:32:27:
There is 1 use of DictType, but who's counting. +1 on reverting it's use. jan.nijtmans added on 2023-08-10 16:06:08:
I'm +1 on reverting the use of TclDuplicatePureObj. It's possible a better optimization is possible, but TclDuplicatePureObj isn't it. Sorry, Nathan griffin added on 2023-08-10 16:21:33:
I've just started removing this code on this ticket branch. apnadkarni added on 2023-08-11 01:36:32:
For the record, my more detailed take on the matter. the following bugs were triggered by TclDuplicatePureObj : [bc7ddc7944] (this), [8c52e78738], [a34733451b], [c405bdd1f0]. This does not necessarily mean TclDuplicatePureObj itself was buggy but possibly it was used where its apparent semantic difference from Tcl_DuplicateObj made it unsuitable for use in those instances. Even more concerning is that the semantics of the function are different between 8.7 and 9.0 in subtle but important ways (use AttemptInitStringRep vs InitStringRep, the different typePtr used for conversion). This is confusing and just asking for trouble. Until this is sorted out, I'm in favour of reverting even if the original idea has merit. /Ashok apnadkarni added on 2023-08-11 01:44:56:
Brian, with regard to reverting changes, in case you were not aware, note that many (most?) TclDuplicatePureObj calls need to be replaced by ListObjCopy, not Tcl_DuplicateObj for efficiency reasons. ListObjCopy has been around forever and afaik DGP introduced it for list performance reasons. Nathan replaced those calls as well with the generalized TclDuplicatePureObj. Reverting those to Tcl_DuplicateObj would likely be a performance regression. I think it would be easier to first revert to ListObjCopy and then Tcl_DuplicateObj. I can do that later as well if you are too far along. /Ashok griffin added on 2023-08-11 18:01:19:
Thanks Ashok! I've replaced DuplicateObj with ListObjCopy in a couple places. There has been a lot of change since DuplicatePureObj was introduced. It would be good if other eyes reviewed the changes in this branch. apnadkarni added on 2023-08-12 15:41:42:
Let me know when you're done and I'll review for anything that might have been missed. /Ashok griffin added on 2023-08-12 18:00:55:
I think I'm done with changes. Everything appears to be reverted. apnadkarni added on 2023-08-14 12:29:31:
I've reviewed the changes and didn't find any errors of omission or commission :-) griffin added on 2023-08-14 15:52:03:
Thanks Ashok! The fix (reverting TclDuplicatePureObj back to ListObjCopy) has been committed. griffin added on 2023-08-18 01:55:07:
Fixed with commit [35275c285b]. pointsman added on 2023-08-20 00:57:37:
It seems that [35275c285b] has broken something else. I'm sorry that I could not track it further down atm (lacking time). To reproduce the apparently introduced problem by [35275c285b] do: Step 0: Install a build of current Tcl trunk somewhere Step 0.5: Get the tcllib fossil repository: fossil clone https://core.tcl-lang.org/tcllib tcllib.fossil Step 1: Create a test directory, cd into and do fossil open path/to/your/tcllib.fossil tcl9-testarea-rde Step 2: Run tclsh sak.tcl test run -v --shell /path/to/your/tclsh9.0 clay Prior to [35275c285b] (and with Tcl 8.6) this runs without failing tests. With that commit 91 tests fail. apnadkarni added on 2023-08-20 11:24:12:
I don't think it's that commit, I don't see any errors. The errors show up with [2a01c62b1a] (dict length performance regression fix) at least on my system. apnadkarni added on 2023-08-20 13:29:46:
Closing as confirmed that the regression was introduced with [2a01c62b1a] and not [35275c285b].
Reopened [f5c01e42a7] corresponding to the breaking commit.
jan.nijtmans added on 2023-08-24 17:53:50:
Many thanks to Don and Paul for the "revert-dup-pure" branch (and to Paul and Ashok for doing the revert on trunk!). I had a look, and it looks good to me, +1 for merging "revert-dup-pure" to core-8-branch. Clearly, TclDuplicatePureObj() (as currently implemented) was a mistake, and my attempt trying to backport it to 8.7 made it even worse! Sorry for that, I totally agree on all actions done! Thanks! | ||||
