Check-in [b9ed346070]

	"353":{"url":"./tip/353.md","tcl-ticket":"2823282","created":"29-Jul-2009","post-history":"","state":"Final","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 353: NR-enabled Expressions for Extensions","author":["Don Porter <[email protected]>"],"is-jest":false},
	"352":{"url":"./tip/352.md","created":"14-Jul-2009","post-history":"","state":"Draft","vote":"Pending","type":"Informative","title":"# TIP 352: Tcl Style Guide","author":["Ray Johnson <[email protected]>","Donal K. Fellows <[email protected]>","Mark Janssen <[email protected]>"],"is-jest":false},
	"351":{"url":"./tip/351.md","created":"09-Jul-2009","post-history":"","state":"Final","tcl-branch":"tip-351","tcl-version":"8.7","vote":"Done","type":"Project","title":"# TIP 351: Add Striding Support to lsearch","author":["Peter da Silva <[email protected]>","Donal K. Fellows <[email protected]>","Harald Oehlmann <[email protected]>","Andreas Leitgeb <[email protected]>"],"is-jest":false},
	"350":{"url":"./tip/350.md","obsoletes":"308","created":"18-Apr-2009","post-history":"","state":"Draft","vote":"Pending","type":"Informative","title":"# TIP 350: Tcl Database Connectivity - Corrigenda","author":["Kevin B. Kenny <[email protected]>"],"is-jest":false},
	"349":{"url":"./tip/349.md","created":"17-Apr-2009","post-history":"","state":"Draft","vote":"Pending","tcl-version":"8.7","type":"Project","title":"# TIP 349: New \"-cargo\" option for every Tk widget","author":["Zbigniew Baniewski <[email protected]>"],"is-jest":false},
	"348":{"url":"./tip/348.md","tcl-ticket":"2868499","keywords":"Tcl,debugging","created":"26-Feb-2009","post-history":"","state":"Final","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 348: Substituted 'errorstack' / 'traceback'","author":["Alexandre Ferrieux <[email protected]>"],"is-jest":false},
	"347":{"url":"./tip/347.md","keywords":"Tcl","created":"09-Feb-2009","obsoleted-by":"395","post-history":"","state":"Withdrawn","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 347: Align 'string is ...' to Type-Conversion Functions in 'expr'","discussions-to":"news:comp.lang.tcl","author":["Jos Decoster <[email protected]>"],"is-jest":false},
	"346":{"url":"./tip/346.md","tcl-ticket":"1665628","keywords":"Tcl,encoding,convertto,strict,Unicode,String,ByteArray","created":"02-Feb-2009","post-history":"","state":"Draft","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 346: Error on Failed String Encodings","author":["Alexandre Ferrieux <[email protected]>"],"is-jest":false},
	"345":{"url":"./tip/345.md","tcl-ticket":"2564363","keywords":"Tcl,encoding,invalid UTF-8","created":"05-Feb-2009","post-history":"","state":"Final","tcl-version":"8.7","vote":"Done","type":"Project","title":"# TIP 345: Kill the 'identity' Encoding","discussions-to":"Tcl Core List","author":["Alexandre Ferrieux <[email protected]>"],"is-jest":false},
	"344":{"url":"./tip/344.md","created":"31-Dec-2008","post-history":"","state":"Draft","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 344: Bring TCP_NODELAY and SO_KEEPALIVE to socket options","author":["Alexandre Ferrieux <[email protected]>"],"is-jest":false},
	"343":{"url":"./tip/343.md","tcl-ticket":"2368084","keywords":"Tcl,binary","created":"03-Dec-2008","post-history":"","state":"Final","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 343: A Binary Specifier for [format/scan]","author":["Alexandre Ferrieux <[email protected]>"],"is-jest":false},
	"342":{"url":"./tip/342.md","keywords":"dictionary, default value","state":"Final","type":"Project","author":["Lars Hellström <[email protected]>"],"votes-for":"DKF, JN, DGP, SL, AK","votes-against":"none","created":"27-Nov-2008","title":"# TIP 342: Dict Get With Default","post-history":"","vote":"Done","tcl-version":"8.7","tcl-branch":"tip-342","votes-present":"FV","is-jest":false},
	"341":{"url":"./tip/341.md","tcl-ticket":"2370575","keywords":"Tcl, set intersection","created":"27-Nov-2008","post-history":"","state":"Final","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 341: Multiple 'dict filter' Patterns","author":["Lars Hellström <[email protected]>"],"is-jest":false},
	"340":{"url":"./tip/340.md","keywords":"Tcl_SetResult","created":"14-Nov-2008","post-history":"","state":"Withdrawn","tcl-version":"8.7","vote":"Pending","type":"Project","title":"# TIP 340: Const Qualification of Tcl_SetResult's Argument","author":["Jan Nijtmans <[email protected]>"],"is-jest":false},
	"339":{"url":"./tip/339.md","created":"14-Nov-2008","post-history":"","state":"Rejected","tcl-version":"8.6","vote":"Done","type":"Project","title":"# TIP 339: Case-Insensitive Package Names","author":["Andreas Kupries <[email protected]>"],"is-jest":false},

	"4":{"url":"./tip/4.md","created":"26-Oct-2000","post-history":"","state":"Draft","vote":"Pending","type":"Informative","title":"# TIP 4: Tcl Release and Distribution Philosophy","discussions-to":"news:comp.lang.tcl","author":["Brent Welch <[email protected]>","Donal K. Fellows <[email protected]>","Larry W. Virden <[email protected]>","Larry W. Virden <[email protected]>"],"is-jest":false},
	"3":{"url":"./tip/3.md","created":"14-Sep-2000","obsoleted-by":"467","post-history":"","state":"Accepted","vote":"Done","type":"Process","title":"# TIP 3: TIP Format","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>"],"is-jest":false},
	"2":{"url":"./tip/2.md","created":"12-Sep-2000","post-history":"","state":"Draft","vote":"Pending","type":"Process","title":"# TIP 2: TIP Guidelines","author":["Andreas Kupries <[email protected]>","Donal K. Fellows <[email protected]>","Don Porter <[email protected]>","Mo DeJong <[email protected]>","Larry W. Virden <[email protected]>","Kevin Kenny <[email protected]>"],"is-jest":false},
	"1":{"url":"./tip/1.md","created":"14-Sep-2000","obsoleted-by":"386","post-history":"","state":"Withdrawn","vote":"No voting","type":"Informational","title":"# TIP 1: TIP Index","author":["TIP Editor <[email protected]>"],"is-jest":false},
	"0":{"url":"./tip/0.md","created":"11-Dec-2000","post-history":"","state":"Final","vote":"Done","type":"Process","title":"# TIP 0: Tcl Core Team Basic Rules","author":["John Ousterhout <[email protected]>"],"is-jest":false},
	"@min": 0,
	"@max": 549
}, "@timestamp": 1559927027}

</tr>
<tr class='state-draft type-project version-87'>
<td valign='top'><a href='./tip/346.md'>346</a></td>
<td valign='top'>Project</td>
<td valign='top'>8.7</td>
<td valign='top'>Draft</td>
<td valign='top' ><a href='./tip/346.md'>Error on Failed String Encodings</a></td>
<td valign='top'><a href='/tcl/tktview/1665628'>Link</a></td>
</tr>
<tr class='state-final type-project version-87'>
<td valign='top'><a href='./tip/345.md'>345</a></td>
<td valign='top'>Project</td>
<td valign='top'>8.7</td>
<td valign='top'>Final</td>
<td valign='top' ><a href='./tip/345.md'>Kill the &apos;identity&apos; Encoding</a></td>

# TIP 346: Error on Failed String Encodings
	Author:         Alexandre Ferrieux <[email protected]>
	State:          Draft
	Type:           Project
	Vote:           Pending
	Created:        02-Feb-2009
	Post-History:   
	Keywords:       Tcl,encoding,convertto,strict,Unicode,String,ByteArray
	Tcl-Version:    8.7
	Tcl-Ticket:     1665628
-----

# Abstract

This TIP proposes to raise an error when an encoding-based conversion
loses information.

# Background

Encoding-based conversions occur e.g. when writing a string to a
channel. In doing so, Unicode characters are converted to sequences of bytes
according to the channel's encoding. Similarly, a conversion can occur
on request of the ByteArray internal representation of an object, the target
encoding being ISO8859-1. In both cases, for some
combinations of Unicode char and target encoding, the mapping is lossy
\(non-injective\). For example, the "e acute" character, and many of its
cousins, is mapped to a "?" in the 'ascii' target encoding. Also, Unicode chars above \\u00FF get 'projected' onto their low byte in the ISO8859-1 ByteArray conversion.

This loss of information, in the first case, introduces unnoticed i18n
mishandlings. In the second case, it makes it unreliable to do pure-ByteArray
operations on objects unless they have no string representation. This induces
unwanted and hard-to-debug performance hits on bytearray manipulations when
people add debugging **puts**.

# Proposed Change

This TIP proposes to make this loss conspicuous.

For the first use case, the idea is to introduce a **-strict** option to
**encoding convertto**, that would raise an explicit error when non-mappable
characters are met. Lossy conversions during channel I/O would also fail if a **-strictencoding true** [fconfigure option] is set.
  For the second case, we simply want the conversion to
fail, like does the Listification of an ill-formed list. In both cases, the
change consists of letting the proper internal conversion routine like **SetByteArrayFromAny** return TCL\_ERROR.

# Rationale

The second case does imply a Potential Incompatibility, since currently SBFA is documented to always return TCL\_OK. However, it is felt
that virtually all cases that are sensitive to this, are actually half-working
in a completely hidden manner. Hence the global effect is a healthy one.

# Reference Example

See Bug 1665628 <https://sourceforge.net/support/tracker.php?aid=1665628> .

# Copyright

This document has been placed in the public domain.