Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Establish 4 symbols for categories of parsed index values:
TCL_INDEX_START = 0 The start index.
TCL_INDEX_END = -2 The "end" index.
TCL_INDEX_BEFORE = -1 All indices less than start.
TCL_INDEX_AFTER = INT_MAX All indices greater than "end".
Then use these symbols among callers of TclGetIndexFromToken() so that index value parsing can directly implement the callers sense of when out of range indices ought to be treated the same as start or end positions. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | bug-db36fa5122 |
Files: | files | file ages | folders |
SHA3-256: |
3f026007bd14c9d26b047fe199b5fd14 |
User & Date: | dgp 2018-03-07 18:39:44.439 |
Context
2018-03-07
| ||
19:55 | Express INST_LIST_INDEX_IMM index processinig in terms of TCL_INDEX_END so that consistencies are ma... check-in: 31a078bec7 user: dgp tags: bug-db36fa5122 | |
18:39 | Establish 4 symbols for categories of parsed index values: TCL_INDEX_START = 0 The start index.... check-in: 3f026007bd user: dgp tags: bug-db36fa5122 | |
17:02 | Rework TclGetIndexFromToken to make use of TclGetEndOffsetFromObj, and to lay out the index value en... check-in: 8cadbda624 user: dgp tags: bug-db36fa5122 | |
Changes
Changes to generic/tclAssembly.c.
︙ | ︙ | |||
2254 2255 2256 2257 2258 2259 2260 | /* General operand validity check */ if (GetNextOperand(assemEnvPtr, tokenPtrPtr, &value) != TCL_OK) { return TCL_ERROR; } Tcl_DecrRefCount(value); /* Convert to an integer, advance to the next token and return. */ | > > > > > | | | 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 | /* General operand validity check */ if (GetNextOperand(assemEnvPtr, tokenPtrPtr, &value) != TCL_OK) { return TCL_ERROR; } Tcl_DecrRefCount(value); /* Convert to an integer, advance to the next token and return. */ /* * NOTE: Indexing a list with an index before it yields the * same result as indexing after it, and might be more easily portable * when list size limits grow. */ status = TclGetIndexFromToken(tokenPtr, result, TCL_INDEX_BEFORE, TCL_INDEX_BEFORE); *tokenPtrPtr = TokenAfter(tokenPtr); if (status == TCL_ERROR && interp) { Tcl_SetObjResult(interp, Tcl_ObjPrintf("bad index \"%.*s\"", tokenPtr->size, tokenPtr->start)); Tcl_SetErrorCode(interp, "TCL", "ASSEM", "BADINDEX", NULL); } return status; |
︙ | ︙ |
Changes to generic/tclCompCmdsGR.c.
︙ | ︙ | |||
38 39 40 41 42 43 44 | * compile time. Two cases are possible. The compile time value * of the token might be parsed as an absolute index value * in the C signed int range. Note that this includes index * values that are integers as presented as well as index * arithmetic expressions that can be fully computed at compile * time. The absolute index values that can be directly meaningful * as an index into either a list or a string are those integer | > | | | | | | | > > | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | * compile time. Two cases are possible. The compile time value * of the token might be parsed as an absolute index value * in the C signed int range. Note that this includes index * values that are integers as presented as well as index * arithmetic expressions that can be fully computed at compile * time. The absolute index values that can be directly meaningful * as an index into either a list or a string are those integer * values >= TCL_INDEX_START (0) and < TCL_INDEX_AFTER (INT_MAX). * The largest string supported in Tcl 8 has bytelength INT_MAX. * This means the largest character supported length is also INT_MAX, * and the index of the last character in a string of length INT_MAX * is INT_MAX-1. * * Any absolute index value parsed outside that range is encoded * using the minBoundary and maxBounday values passed in by the * caller as the encoding to use for indices that are either * less than or greater than the usable index range. TCL_INDEX_AFTER * is available as a good choice for most callers to use for * maxBoundary. Likewise, the value TCL_INDEX_BEFORE is good for * most callers to use for minBoundary. Other values are possible * when the caller knows it is helpful in producing its own behavior * for indices before and after the indexed item. * * A token can also be parsed as an end-relative index expression. * All end-relative expressions that indicate an index larger * than end (end+2, end--5) point beyond the end of the indexed * collection, and can be encoded as maxBoundary. The end-relative * expressions that indicate an index less than or equal to end * are encoded relative to the value TCL_INDEX_END (-2). The |
︙ | ︙ | |||
97 98 99 100 101 102 103 | return TCL_ERROR; } result = TclGetIntFromObj(NULL, tmpObj, &idx); if (result == TCL_OK) { /* We parsed a value in the range INT_MIN...INT_MAX */ integerEncode: | | | 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | return TCL_ERROR; } result = TclGetIntFromObj(NULL, tmpObj, &idx); if (result == TCL_OK) { /* We parsed a value in the range INT_MIN...INT_MAX */ integerEncode: if (idx < TCL_INDEX_START) { /* All negative absolute indices are "before the beginning" */ idx = minBoundary; } else if (idx == INT_MAX) { /* This index value is always "after the end" */ idx = maxBoundary; } /* usual case, the absolute index value encodes itself */ |
︙ | ︙ | |||
119 120 121 122 123 124 125 | if (idx > 0) { /* * All end+postive or end-negative expressions * always indicate "after the end". */ idx = maxBoundary; } else if (idx < INT_MIN - TCL_INDEX_END) { | | | 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 | if (idx > 0) { /* * All end+postive or end-negative expressions * always indicate "after the end". */ idx = maxBoundary; } else if (idx < INT_MIN - TCL_INDEX_END) { /* These indices always indicate "before the beginning */ idx = minBoundary; } else { /* Encoded end-positive (or end+negative) are offset */ idx += TCL_INDEX_END; } } else { result = TclGetIntForIndexM(NULL, tmpObj, 0, &idx); |
︙ | ︙ | |||
210 211 212 213 214 215 216 | for (i=1; i<numWords; varTokenPtr = TokenAfter(varTokenPtr),i++) { localIndex = IndexTailVarIfKnown(interp, varTokenPtr, envPtr); if (localIndex < 0) { return TCL_ERROR; } | | | 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 | for (i=1; i<numWords; varTokenPtr = TokenAfter(varTokenPtr),i++) { localIndex = IndexTailVarIfKnown(interp, varTokenPtr, envPtr); if (localIndex < 0) { return TCL_ERROR; } /* TODO: Consider what value can pass through the * IndexTailVarIfKnown() screen. Full CompileWord() * likely does not apply here. Push known value instead. */ CompileWord(envPtr, varTokenPtr, interp, i); TclEmitInstInt4( INST_NSUPVAR, localIndex, envPtr); } /* |
︙ | ︙ | |||
1170 1171 1172 1173 1174 1175 1176 | valTokenPtr = TokenAfter(parsePtr->tokenPtr); if (numWords != 3) { goto emitComplexLindex; } idxTokenPtr = TokenAfter(valTokenPtr); | | | | | > | | < < | 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 | valTokenPtr = TokenAfter(parsePtr->tokenPtr); if (numWords != 3) { goto emitComplexLindex; } idxTokenPtr = TokenAfter(valTokenPtr); if (TclGetIndexFromToken(idxTokenPtr, &idx, TCL_INDEX_BEFORE, TCL_INDEX_BEFORE) == TCL_OK) { /* * The idxTokenPtr parsed as a valid index value and was * encoded as expected by INST_LIST_INDEX_IMM. * * NOTE: that we rely on indexing before a list producing the * same result as indexing after a list. */ CompileWord(envPtr, valTokenPtr, interp, 1); TclEmitInstInt4( INST_LIST_INDEX_IMM, idx, envPtr); return TCL_OK; } |
︙ | ︙ | |||
1399 1400 1401 1402 1403 1404 1405 | int idx1, idx2; if (parsePtr->numWords != 4) { return TCL_ERROR; } listTokenPtr = TokenAfter(parsePtr->tokenPtr); | < < < < < < | > > > > > | > < > | < > > | 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 | int idx1, idx2; if (parsePtr->numWords != 4) { return TCL_ERROR; } listTokenPtr = TokenAfter(parsePtr->tokenPtr); tokenPtr = TokenAfter(listTokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx1, TCL_INDEX_START, TCL_INDEX_AFTER) != TCL_OK) { return TCL_ERROR; } /* * Token was an index value, and we treat all "first" indices * before the list same as the start of the list. */ tokenPtr = TokenAfter(tokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx2, TCL_INDEX_BEFORE, TCL_INDEX_END) != TCL_OK) { return TCL_ERROR; } /* * Token was an index value, and we treat all "last" indices * after the list same as the end of the list. */ /* * Issue instructions. It's not safe to skip doing the LIST_RANGE, as * we've not proved that the 'list' argument is really a list. Not that it * is worth trying to do that given current knowledge. */ |
︙ | ︙ | |||
1466 1467 1468 1469 1470 1471 1472 | /* * Parse the index. Will only compile if it is constant and not an * _integer_ less than zero (since we reserve negative indices here for * end-relative indexing) or an end-based index greater than 'end' itself. */ tokenPtr = TokenAfter(listTokenPtr); | > > > > > > > > | > | 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 | /* * Parse the index. Will only compile if it is constant and not an * _integer_ less than zero (since we reserve negative indices here for * end-relative indexing) or an end-based index greater than 'end' itself. */ tokenPtr = TokenAfter(listTokenPtr); /* * NOTE: This command treats all inserts at indices before the list * the same as inserts at the start of the list, and all inserts * after the list the same as inserts at the end of the list. We * make that transformation here so we can use the optimized bytecode * as much as possible. */ if (TclGetIndexFromToken(tokenPtr, &idx, TCL_INDEX_START, TCL_INDEX_END) != TCL_OK) { return TCL_ERROR; } /* * There are four main cases. If there are no values to insert, this is * just a confirm-listiness check. If the index is '0', this is a prepend. * If the index is 'end' (== TCL_INDEX_END), this is an append. Otherwise, |
︙ | ︙ | |||
1490 1491 1492 1493 1494 1495 1496 | for (i=3 ; i<parsePtr->numWords ; i++) { tokenPtr = TokenAfter(tokenPtr); CompileWord(envPtr, tokenPtr, interp, i); } TclEmitInstInt4( INST_LIST, i-3, envPtr); | | | | 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 | for (i=3 ; i<parsePtr->numWords ; i++) { tokenPtr = TokenAfter(tokenPtr); CompileWord(envPtr, tokenPtr, interp, i); } TclEmitInstInt4( INST_LIST, i-3, envPtr); if (idx == TCL_INDEX_START) { TclEmitInstInt4( INST_REVERSE, 2, envPtr); TclEmitOpcode( INST_LIST_CONCAT, envPtr); } else if (idx == TCL_INDEX_END) { TclEmitOpcode( INST_LIST_CONCAT, envPtr); } else { /* * Here we handle two ranges for idx. First when idx > 0, we * want the first half of the split to end at index idx-1 and * the second half to start at index idx. * Second when idx < TCL_INDEX_END, indicating "end-N" indexing, |
︙ | ︙ | |||
1554 1555 1556 1557 1558 1559 1560 | int idx1, idx2, i, offset, offset2; if (parsePtr->numWords < 4) { return TCL_ERROR; } listTokenPtr = TokenAfter(parsePtr->tokenPtr); | < < < < < < | > | > | > > > > | | | < > > > > | > | > > < < < < < < | 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 | int idx1, idx2, i, offset, offset2; if (parsePtr->numWords < 4) { return TCL_ERROR; } listTokenPtr = TokenAfter(parsePtr->tokenPtr); tokenPtr = TokenAfter(listTokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx1, TCL_INDEX_START, TCL_INDEX_AFTER) != TCL_OK) { return TCL_ERROR; } tokenPtr = TokenAfter(tokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx2, TCL_INDEX_BEFORE, TCL_INDEX_END) != TCL_OK) { return TCL_ERROR; } /* * idx1, idx2 are the conventional encoded forms of the tokens parsed * as all forms of index values. Values of idx1 that come before the * list are treated the same as if they were the start of the list. * Values of idx2 that come after the list are treated the same as if * they were the end of the list. */ if (idx1 == TCL_INDEX_AFTER) { /* * [lreplace] treats idx1 value end+1 differently from end+2, etc. * The operand encoding cannot distinguish them, so we must bail * out to direct evaluation. */ return TCL_ERROR; } /* TODO: ...... */ /* * Compilation fails when one index is end-based but the other isn't. * Fixing this will require more bytecodes, but this is a workaround for * now. [Bug 47ac84309b] */ if ((idx1 <= TCL_INDEX_END) != (idx2 <= TCL_INDEX_END)) { /* * NOTE: when idx1 == 0 and idx2 == TCL_INDEX_END, * we bail out here! Yet, down below */ return TCL_ERROR; |
︙ | ︙ | |||
3045 3046 3047 3048 3049 3050 3051 | localIndex = IndexTailVarIfKnown(interp, varTokenPtr, envPtr); if (localIndex < 0) { return TCL_ERROR; } | | | 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 | localIndex = IndexTailVarIfKnown(interp, varTokenPtr, envPtr); if (localIndex < 0) { return TCL_ERROR; } /* TODO: Consider what value can pass through the * IndexTailVarIfKnown() screen. Full CompileWord() * likely does not apply here. Push known value instead. */ CompileWord(envPtr, varTokenPtr, interp, i); TclEmitInstInt4( INST_VARIABLE, localIndex, envPtr); if (i+1 < numWords) { /* |
︙ | ︙ |
Changes to generic/tclCompCmdsSZ.c.
︙ | ︙ | |||
930 931 932 933 934 935 936 | fromTokenPtr = TokenAfter(stringTokenPtr); toTokenPtr = TokenAfter(fromTokenPtr); /* * Parse the two indices. */ | | > > > > > | > < < < > > > > | 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 | fromTokenPtr = TokenAfter(stringTokenPtr); toTokenPtr = TokenAfter(fromTokenPtr); /* * Parse the two indices. */ if (TclGetIndexFromToken(fromTokenPtr, &idx1, TCL_INDEX_START, TCL_INDEX_AFTER) != TCL_OK) { goto nonConstantIndices; } /* * Token parsed as an index expression. We treat all indices before * the string the same as the start of the string. */ if (TclGetIndexFromToken(toTokenPtr, &idx2, TCL_INDEX_BEFORE, TCL_INDEX_END) != TCL_OK) { goto nonConstantIndices; } /* * Token parsed as an index expression. We treat all indices after * the string the same as the end of the string. */ /* * Push the operand onto the stack and then the substring operation. */ CompileWord(envPtr, stringTokenPtr, interp, 1); OP44( STR_RANGE_IMM, idx1, idx2); |
︙ | ︙ | |||
983 984 985 986 987 988 989 | valueTokenPtr = TokenAfter(parsePtr->tokenPtr); if (parsePtr->numWords == 5) { tokenPtr = TokenAfter(valueTokenPtr); tokenPtr = TokenAfter(tokenPtr); replacementTokenPtr = TokenAfter(tokenPtr); } | < < < < < < < | > > > > > | > > | | < > | | | 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 | valueTokenPtr = TokenAfter(parsePtr->tokenPtr); if (parsePtr->numWords == 5) { tokenPtr = TokenAfter(valueTokenPtr); tokenPtr = TokenAfter(tokenPtr); replacementTokenPtr = TokenAfter(tokenPtr); } tokenPtr = TokenAfter(valueTokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx1, TCL_INDEX_START, TCL_INDEX_AFTER) != TCL_OK) { goto genericReplace; } /* * Token parsed as an index value. Indices before the string are * treated as index of start of string. */ tokenPtr = TokenAfter(tokenPtr); if (TclGetIndexFromToken(tokenPtr, &idx2, TCL_INDEX_BEFORE, TCL_INDEX_END) != TCL_OK) { goto genericReplace; } /* * Token parsed as an index value. Indices after the string are * treated as index of end of string. */ /* TODO...... */ /* * We handle these replacements specially: first character (where * idx1=idx2=0) and last character (where idx1=idx2=TCL_INDEX_END). Anything * else and the semantics get rather screwy. * * TODO: These seem to be very narrow cases. They are not even * covered by the test suite, and any programming that ends up |
︙ | ︙ |
Changes to generic/tclCompile.h.
︙ | ︙ | |||
1686 1687 1688 1689 1690 1691 1692 | #define TCL_NO_ELEMENT 2 /* Do not push the array element. */ /* * Special value used by TclGetIndexFromToken to encoding the "end" index. */ #define TCL_INDEX_END (-2) | > | > | 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 | #define TCL_NO_ELEMENT 2 /* Do not push the array element. */ /* * Special value used by TclGetIndexFromToken to encoding the "end" index. */ #define TCL_INDEX_END (-2) #define TCL_INDEX_BEFORE (-1) #define TCL_INDEX_START (0) #define TCL_INDEX_AFTER (INT_MAX) /* * DTrace probe macros (NOPs if DTrace support is not enabled). */ /* * Define the following macros to enable debug logging of the DTrace proc, |
︙ | ︙ |