| Ticket UUID: | 219297 | ||
| Title: | euc-jp encoding ignores euc-3 extension | ||
| Type: | Bug | Created on: | 2000-10-26 05:10:19 |
| Submitter: | nobody | Assigned to: | hobbs |
| Subsystem: | 10. Objects | Severity: | |
| Priority: | 4 | Last modified: | 2002-04-20 08:46:33 |
| Status: | Open | Closed by: | |
| Resolution: | None | Closed on: | |
| Version: | final: 8.1.1 | ||
| Description: | ||||
OriginalBugID: 2609 Bug
Version: 8.1.1
SubmitDate: '1999-08-31'
LastModified: '1999-10-21'
Severity: LOW
Status: UnAssn
Submitter: techsupp
ChangedBy: hobbs
OS: Linux-Red Hat
Machine: Other
FixedDate: '2000-10-25'
ClosedDate: '2000-10-25'
Name:
Roger E Critchlow Jr
ReproducibleScript:
scan [encoding convertfrom euc-jp "\217\260\241 3021 U4e02"] %c%c char1
char2
if {$char1 != 0x4e02} {
puts "tcl euc-jp translated three byte encoding for U+4e02 into
[format {\\x%02x \\u%04x} $char1 $char2]"
exit 1
} else {
exit 0
}
ObservedBehavior:
tcl euc-jp encoding ignores the euc-3 extension used in Japanese to
encode 5800 kanji.
DesiredBehavior:
tcl euc-jp encoding implements the euc-jp encoding.
| ||||
| User Comments: | ||||
hobbs added on 2001-10-13 03:14:06:
Logged In: YES user_id=72656 We have to verify that this "euc-3 extension" is meant to be part of the standard euc-jp or not (since our euc-jp is auto-generated from a former standard script from Unicode.org). haible added on 2002-04-20 06:04:07:
Logged In: YES
user_id=5923
What the submitter meant was the 3-byte area
(0x8F 0x{A1..FE} 0x{A1..FE} of EUC-JP that encodes the
JISX0212
character set. You find it summarized in
http://www.ksky.ne.jp/~smile4me/charcode/japanese.htm
hobbs added on 2002-04-20 08:03:45:
Logged In: YES user_id=72656 The above octal is this in hex: \x8f\xb0\xa1 To reiterate the page Bruno mentions, these characters are encoded as 3-byte chars (0x8FA1A1-0x8FFEFE) and represent the supplemental jisx0212 chars. A couple other good resources: http://www.io.com/~kazushi/encoding/ujis.html http://web.lfw.org/text/jp.html Basically the problem seems to be that the way that we originally did euc-jp was only for two-byte support. That isn't sufficient of course ... hobbs added on 2002-04-20 08:24:12:
Logged In: YES user_id=72656 Perhaps the perl uses of the Tcl encoding stuff have provided a possible solution for us: http://archive.develooper.com/perl- [email protected]/msg00448.html haible added on 2002-04-20 08:39:30:
Logged In: YES user_id=5923 If Perl has really copied the Tcl .enc files, then yes that patch should be good for Tcl as well. Except for one thing: the name of the encoding. There are not two encodings euc-jp and euc-jp-0212, one without and one with JISX0212. There is only one encoding EUC-JP, and it includes JISX0212. Those who think that EUC-JP does not include JISX0212 are misled by the standard Japanese encoding on Windows - Shift_JIS. This one does not contain JISX0212. See Ken Lunde's book "CJKV Information Processing" if you want the gory details of all Japanese encodings (probably more details than you can bear :-)). hobbs added on 2002-04-20 08:46:33:
Logged In: YES user_id=72656 If you look at the person's recommended patch, it appears that he uses the euc-jp-0212 part as just a special case name. He turns euc-jp into an escape encoding to handle the jis0212 3-byte case. It's a bit of a hack, but the essence is that using "euc-jp" would work correctly. | ||||
