TIP 607: -failindex option for encoding convertto/convertfrom

Login
Bounty program for improvements to Tcl and certain Tcl packages.
Author:         Jan Nijtmans <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        30-Jul-2021
Post-History:
Keywords:       Tcl
Tcl-Version:    8.7
Tcl-Branch:     tip607-encoding-failindex
Vote-Summary:   Accepted 5/0/0
Votes-For:      AK, FV, JN, KW, SL
Votes-Against:  none
Votes-Present:  none

Abstract

This TIP proposes to add a -failindex option to encoding convertto/convertfrom. The implementation brings TIP [601] to the script level. In case of untransformable data, the error location and the so far transformed string may be retrieved.

Rationale

Please refer to TIP [601] for usage examples and use-cases. This tip was extracted from there, but the rationale and many descriptions also hold for this TIP.

Remark, that the wish in use-case 1 to return the so far encoded data is fullfilled by this TIP.

Option name

The option name -failindex is inspired by the option of the string is command with the same name and similar functionality.

Distinguish between error types "incomplete multi-byte sequence" and "not encodable character"

See TIP [601] Example 1 and 2 for the explanation of the two error types.

The two error types are bound to the used command:

In consequence, no feed-back of the error type is required. The error position is sufficient.

TCL 8.7 and TCL 9.0

In TCL 8.7, this interface is the only way to get informed about encoding errors.

In TCL 9.0, the default behaviour is to fail on any encoding errors. So, this interface may also be helpful to prepare TCL 8.7 scripts for TCL 9.0 and to check where TCL 9.0 would fail.

Specification

New option

The command is extended by a -failindex option:

encoding convertfrom ?-failindex posvar? ?encoding? data
encoding convertto ?-failindex posvar? ?encoding? data

The distinct behaviour is as follows:

The definition of the value written by the -failindex option is given in TIP [601] as "Error position".

This option may not be used together with the TCL encoding option -nocomplain of TIP [601]. Any attempt to use -nocomplain and -failindex simultaneously is an error case.

Credits

The proposal was initiated by a post by Andreas Leitgeb 2021-05-12 on the core list.

Copyright

This document has been placed in the public domain.