# TIP 502: Index Value Reform
	Author:         Don Porter <dgp@users.sourceforge.net>
	State:          Final
	Type:           Project
	Vote:           Done
	Created:        26-Feb-2018
	Post-History:   
	Tcl-Version:	8.7
	Tcl-Branch:     tip-502
	Vote-Results:   5/0/0 accepted
	Votes-For:      DKF, JN, JD, KBK, SL
	Votes-Against:  none
	Votes-Present:  none
-----

# Abstract

Proposes reformed handling of Tcl index values.

# Background

Many Tcl programmers may be surprised by this result

	% lindex {a b} -4294967295
	b

This is the consequence of two features of incumbent Tcl
functionality. First, passing the value **-4294967295** to
_Tcl\_GetInt\(FromObj\)_ results in a successful extraction
of the C __int__ value __1__. Generally any supported formatting
of any value between __-UINT\_MAX__ and __UINT\_MAX__ is accepted,
but transformed into a value between __INT\_MIN__ and __INT\_MAX__.
Second, the internal routine/macro _TclGetIntForIndex\(M\)_
implements a definition of Tcl index values where any value acceptable
to _Tcl\_GetInt\(FromObj\)_ is also acceptable as an index of the
same value.  Both of these features are of questionable merit, but
the first has a fair bit of compatibility constraint attached to it
because it involves the behavior of public routines.  The second
should be more open to potential revisions.

Another source of possible surprise is demonstrated:

	% lindex {a b} 4294967295
	% lindex {a b} 4294967295+1
	a
	% lindex {a b} 4294967296
	bad index "4294967296": must be integer?[+-]integer? or end?[+-]integer?

These examples illustrate overflow effects in index arithmetic, and a
rejection of apparently valid integer values that are larger than
__UINT\_MAX__. Neither of these effects is in agreement with handling of
integers by **expr** which makes the results surprising.

# Proposal

Revise the parsing of values used as index values to accept all
integer representations acceptable to **expr** as either complete
index values or as operands in the end-offset or index arithmetic
formats.

Although this suggests an unlimited range of valid index values,
in practice all index values will be truncated internally to the
signed wide integer range, __LLONG\_MIN__ to __LLONG\_MAX__, and
in the Tcl 8 series of releases, further truncated to the signed
int range, __INT\_MIN__ to __INT\_MAX__, since no container can
be larger than that range in Tcl 8.

# Compatibility

Examples like the ones in the Background above will incompatibly
change from one successful outcome to a different successful outcome.
This is a true incompatibility, but it is difficult to believe anyone
actually desires the outlier behaviors illustrated above, much less has
code that relies on them.

# Implementation

Drafted on the tip-502 branch.

Note that the core routine of the implementation, _GetWideForIndex_,
produces a *Tcl_WideInt* result from a successful parse. I expect
this functionality will be useful in the Tcl 9 codebase as an enabling
step for containers with sizes larger than __INT\_MAX__. The existing
internal routine _TclGetIntForIndex_ continues to produce the more
limited range suitable for Tcl 8.

# Copyright

This document has been placed in the public domain.

