Tcl Source Code

View Ticket
Login
Ticket UUID: 408568
Title: Variable substitution parsing limited to ASCII alphanumerics
Type: Bug Created on: 2001-03-14 19:08:43
Submitter: kennykb Assigned to: sebres
Subsystem: 45. Parsing and Eval Severity: Minor
Priority: 8 Last modified: 2019-03-11 17:05:15
Status: Open Closed by: nobody
Resolution: None Closed on:
Version: obsolete: 8.3.2
Description:
Dollar-substitution breaks down in the presence of characters that are not in ISO-8859-1.

The problem is that tclParse.c (line 1740 in 8.4a2) has the code:

    c = UCHAR(ch);
    if (isalnum(c) || (c == '_')) { /* INTL: ISO only, UCHAR. */

I think this should almost certainly be replaced with a call to Tcl_UniCharIsAlnum.
User Comments:
hobbs added on 2001-08-07 05:49:22:
Logged In: YES 
user_id=72656

More is necessary than just adjusting that one line, but 
there is also the question of whether that is the intention 
or not.  The problem is that vars written in \u sequence 
won't get interped inside ${...}, but who is doing that?

kennykb added on 2008-05-26 12:11:35:
Logged In: YES 
user_id=99768
Originator: YES

dgp claimed on the chat to be looking at the possibility of
recognizing non-Latin1 names

sebres added on 2019-02-06 19:17:02:
Assigned to me in order to provide a branch of me, solved this for >= 8.5 (just for record).
I'll reassign this back to Don, once I get it pushed (rebased).

sebres added on 2019-03-08 00:55:37:

Well, I rebased my fix in branch bug-408568, where first commit illustrates the "bug" and second fixes it.

Although this is based on another branch, because it covers similar utf-8 regression (which was already fixed in 8.6), but also solves too large discrepancy for lot of code compared to 8.6 (it makes just more simpler to me working on some 8.5/8.6 mixed branches on my side, because avoids many conflicts there).

Assigned back to Don as promised :).


sebres added on 2019-03-08 09:56:51:

This had a certain conflict within test-case "parse-12.26", but it is wanted here (it is variable now), therefore test is adjusted in the branch.

Also note [d2ffcca163] for related subject (non-ascii variable handling).


sebres added on 2019-03-11 16:55:12:

Other similar ticket [1511357] (as noticed by dgp in chat) which could be simply resolved now using new function TclFindBarewordEndUni.


sebres added on 2019-03-11 17:05:15:

Just for the record - example illustrating a certain inconsistency:

proc ::tcl::mathfunc::пи {} {return 3.14}
puts [::tcl::mathfunc::пи]
puts expr:[expr {пи()}]