Tcl Source Code

View Ticket
Login
Ticket UUID: 3393714
Title: overflow in toupper delta
Type: Bug Created on: 2011-08-18 09:28:33
Submitter: jan.nijtmans Assigned to: jan.nijtmans
Subsystem: 44. UTF-8 Strings Severity:
Priority: 5 Medium Last modified: 2014-02-21 19:11:51
Status: Open Closed by:
Resolution: None Closed on:
Version: obsolete: 8.6b2
Description:
From the Unicode table"
0265;LATIN SMALL LETTER TURNED H;Ll;0;L;;;;;N;;;A78D;;A78D
A78D;LATIN CAPITAL LETTER TURNED H;Lu;0;L;;;;;N;;;;0265;

This means that [string toupper \u0265 should give \uA78D, but
in the groups[] table (tclUniData.c) only 9 bits are reserved to
store the case delta. In thos case, we need 17 bits for
the delta (16 bits + sign bit)
User Comments:
nijtmans added on 2011-08-18 18:40:41:
Overflow problem fixed in all open branches.

However, there are 11 unicode characters in Unicode 6.0
(2 capital and 9 small letters), which have a 2-byte
UTF-8 representation for the original character but
a 3-byte representation for the converted character.
Currently the Tcl_UtfTo(Upper|Lower|Title) have
a protection to prevent inflation of the string,
which prevents the conversion in this case.

Here are the affected Unicode characters:
023A;LATIN CAPITAL LETTER A WITH STROKE;Lu;0;L;;;;;N;;;;2C65;
023E;LATIN CAPITAL LETTER T WITH DIAGONAL STROKE;Lu;0;L;;;;;N;;;;2C66;
023F;LATIN SMALL LETTER S WITH SWASH TAIL;Ll;0;L;;;;;N;;;2C7E;;2C7E
0240;LATIN SMALL LETTER Z WITH SWASH TAIL;Ll;0;L;;;;;N;;;2C7F;;2C7F
0250;LATIN SMALL LETTER TURNED A;Ll;0;L;;;;;N;;;2C6F;;2C6F
0251;LATIN SMALL LETTER ALPHA;Ll;0;L;;;;;N;LATIN SMALL LETTER SCRIPT A;;2C6D;;2C6D
0252;LATIN SMALL LETTER TURNED ALPHA;Ll;0;L;;;;;N;LATIN SMALL LETTER TURNED SCRIPT A;;2C70;;2C70
0265;LATIN SMALL LETTER TURNED H;Ll;0;L;;;;;N;;;A78D;;A78D
026B;LATIN SMALL LETTER L WITH MIDDLE TILDE;Ll;0;L;;;;;N;;;2C62;;2C62
0271;LATIN SMALL LETTER M WITH HOOK;Ll;0;L;;;;;N;LATIN SMALL LETTER M HOOK;;2C6E;;2C6E
027D;LATIN SMALL LETTER R WITH TAIL;Ll;0;L;;;;;N;LATIN SMALL LETTER R HOOK;;2C64;;2C64

For the first two capitals, [string tolower] will
result in the original characters. For the other
characters, [string toupper] and [string totitle]
give the wrong result.

Therefore this issue cannot be closed yet.

nijtmans added on 2011-12-08 22:57:17:
In Unicode 6.1, there will be one additional character
for which the same holds:

0266;LATIN SMALL LETTER H WITH HOOK;Ll;0;L;;;;;N;LATIN SMALL LETTER H HOOK;;A7AA;;A7AA