Tcl Source Code

Check-in [37927cede6]
Login
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2019 Conference, Houston/TX, US, Nov 4-8
Send your abstracts to [email protected]
or submit via the online form by Sep 9.

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:New internal routines TclScanElement() and TclConvertElement(). Rewritten guts of machinery to produce string rep of lists. [Bug 3173086]
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | core-8-5-branch
Files: files | file ages | folders
SHA1: 37927cede6a57636dd2c6f3ed267a8c8843a6d7a
User & Date: dgp 2011-05-10 16:52:35
Context
2011-05-12
06:07
If your compiler already defines _WIN64, assume --enable-64bit check-in: bec8d10e17 user: jan.nijtmans tags: core-8-5-branch
2011-05-10
17:22
New internal routines TclScanElement() and TclConvertElement(). Rewritten guts of machinery to produ... check-in: 7720fbb825 user: dgp tags: trunk
16:52
New internal routines TclScanElement() and TclConvertElement(). Rewritten guts of machinery to produ... check-in: 37927cede6 user: dgp tags: core-8-5-branch
16:05
Completed patch with mucho comments. Merge 8.5. Closed-Leaf check-in: 8de9137c51 user: dgp tags: bug-3173086
2011-05-09
13:53
Revise empty string tests so that we avoid potentially expensive string rep generations, especially ... check-in: 765f5fcb20 user: dgp tags: core-8-5-branch
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to ChangeLog.














1
2
3
4
5
6
7












2011-05-09  Don Porter  <[email protected]>

	* generic/tclListObj.c:	Revise empty string tests so that we avoid
	potentially expensive string rep generations, especially for dicts.

2011-05-07  Miguel Sofer  <[email protected]>

>
>
>
>
>
>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2011-05-10  Don Porter  <[email protected]>

	* generic/tclInt.h:     New internal routines TclScanElement() and
	* generic/tclUtil.c:    TclConvertElement() are rewritten guts of
	machinery to produce string rep of lists.  The new routines avoid
	and correct [Bug 3173086].  See comments for much more detail.

	* generic/tclDictObj.c:         Update all callers.
	* generic/tclIndexObj.c:
	* generic/tclListObj.c:
	* generic/tclUtil.c:
	* tests/list.test:

2011-05-09  Don Porter  <[email protected]>

	* generic/tclListObj.c:	Revise empty string tests so that we avoid
	potentially expensive string rep generations, especially for dicts.

2011-05-07  Miguel Sofer  <[email protected]>

Changes to generic/tcl.decls.

770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
declare 216 generic {
    void Tcl_Release(ClientData clientData)
}
declare 217 generic {
    void Tcl_ResetResult(Tcl_Interp *interp)
}
declare 218 generic {
    int Tcl_ScanElement(CONST char *str, int *flagPtr)
}
declare 219 generic {
    int Tcl_ScanCountedElement(CONST char *str, int length, int *flagPtr)
}
# Obsolete
declare 220 generic {
    int Tcl_SeekOld(Tcl_Channel chan, int offset, int mode)
}
declare 221 generic {
    int Tcl_ServiceAll(void)






|


|







770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
declare 216 generic {
    void Tcl_Release(ClientData clientData)
}
declare 217 generic {
    void Tcl_ResetResult(Tcl_Interp *interp)
}
declare 218 generic {
    int Tcl_ScanElement(CONST char *src, int *flagPtr)
}
declare 219 generic {
    int Tcl_ScanCountedElement(CONST char *src, int length, int *flagPtr)
}
# Obsolete
declare 220 generic {
    int Tcl_SeekOld(Tcl_Channel chan, int offset, int mode)
}
declare 221 generic {
    int Tcl_ServiceAll(void)

Changes to generic/tcl.h.

952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
 * Flag values passed to Tcl_ConvertElement.
 * TCL_DONT_USE_BRACES forces it not to enclose the element in braces, but to
 *	use backslash quoting instead.
 * TCL_DONT_QUOTE_HASH disables the default quoting of the '#' character. It
 *	is safe to leave the hash unquoted when the element is not the first
 *	element of a list, and this flag can be used by the caller to indicate
 *	that condition.
 * (Careful! If you change these flag values be sure to change the definitions
 * at the front of tclUtil.c).
 */

#define TCL_DONT_USE_BRACES	1
#define TCL_DONT_QUOTE_HASH	8

/*
 * Flag that may be passed to Tcl_GetIndexFromObj to force it to disallow






<
<







952
953
954
955
956
957
958


959
960
961
962
963
964
965
 * Flag values passed to Tcl_ConvertElement.
 * TCL_DONT_USE_BRACES forces it not to enclose the element in braces, but to
 *	use backslash quoting instead.
 * TCL_DONT_QUOTE_HASH disables the default quoting of the '#' character. It
 *	is safe to leave the hash unquoted when the element is not the first
 *	element of a list, and this flag can be used by the caller to indicate
 *	that condition.


 */

#define TCL_DONT_USE_BRACES	1
#define TCL_DONT_QUOTE_HASH	8

/*
 * Flag that may be passed to Tcl_GetIndexFromObj to force it to disallow

Changes to generic/tclDecls.h.

1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
....
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
#define Tcl_ResetResult_TCL_DECLARED
/* 217 */
EXTERN void		Tcl_ResetResult(Tcl_Interp *interp);
#endif
#ifndef Tcl_ScanElement_TCL_DECLARED
#define Tcl_ScanElement_TCL_DECLARED
/* 218 */
EXTERN int		Tcl_ScanElement(CONST char *str, int *flagPtr);
#endif
#ifndef Tcl_ScanCountedElement_TCL_DECLARED
#define Tcl_ScanCountedElement_TCL_DECLARED
/* 219 */
EXTERN int		Tcl_ScanCountedElement(CONST char *str, int length,
				int *flagPtr);
#endif
#ifndef Tcl_SeekOld_TCL_DECLARED
#define Tcl_SeekOld_TCL_DECLARED
/* 220 */
EXTERN int		Tcl_SeekOld(Tcl_Channel chan, int offset, int mode);
#endif
................................................................................
    void (*tcl_RegisterObjType) (Tcl_ObjType *typePtr); /* 211 */
    Tcl_RegExp (*tcl_RegExpCompile) (Tcl_Interp *interp, CONST char *pattern); /* 212 */
    int (*tcl_RegExpExec) (Tcl_Interp *interp, Tcl_RegExp regexp, CONST char *text, CONST char *start); /* 213 */
    int (*tcl_RegExpMatch) (Tcl_Interp *interp, CONST char *text, CONST char *pattern); /* 214 */
    void (*tcl_RegExpRange) (Tcl_RegExp regexp, int index, CONST84 char **startPtr, CONST84 char **endPtr); /* 215 */
    void (*tcl_Release) (ClientData clientData); /* 216 */
    void (*tcl_ResetResult) (Tcl_Interp *interp); /* 217 */
    int (*tcl_ScanElement) (CONST char *str, int *flagPtr); /* 218 */
    int (*tcl_ScanCountedElement) (CONST char *str, int length, int *flagPtr); /* 219 */
    int (*tcl_SeekOld) (Tcl_Channel chan, int offset, int mode); /* 220 */
    int (*tcl_ServiceAll) (void); /* 221 */
    int (*tcl_ServiceEvent) (int flags); /* 222 */
    void (*tcl_SetAssocData) (Tcl_Interp *interp, CONST char *name, Tcl_InterpDeleteProc *proc, ClientData clientData); /* 223 */
    void (*tcl_SetChannelBufferSize) (Tcl_Channel chan, int sz); /* 224 */
    int (*tcl_SetChannelOption) (Tcl_Interp *interp, Tcl_Channel chan, CONST char *optionName, CONST char *newValue); /* 225 */
    int (*tcl_SetCommandInfo) (Tcl_Interp *interp, CONST char *cmdName, CONST Tcl_CmdInfo *infoPtr); /* 226 */






|




|







 







|
|







1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
....
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
#define Tcl_ResetResult_TCL_DECLARED
/* 217 */
EXTERN void		Tcl_ResetResult(Tcl_Interp *interp);
#endif
#ifndef Tcl_ScanElement_TCL_DECLARED
#define Tcl_ScanElement_TCL_DECLARED
/* 218 */
EXTERN int		Tcl_ScanElement(CONST char *src, int *flagPtr);
#endif
#ifndef Tcl_ScanCountedElement_TCL_DECLARED
#define Tcl_ScanCountedElement_TCL_DECLARED
/* 219 */
EXTERN int		Tcl_ScanCountedElement(CONST char *src, int length,
				int *flagPtr);
#endif
#ifndef Tcl_SeekOld_TCL_DECLARED
#define Tcl_SeekOld_TCL_DECLARED
/* 220 */
EXTERN int		Tcl_SeekOld(Tcl_Channel chan, int offset, int mode);
#endif
................................................................................
    void (*tcl_RegisterObjType) (Tcl_ObjType *typePtr); /* 211 */
    Tcl_RegExp (*tcl_RegExpCompile) (Tcl_Interp *interp, CONST char *pattern); /* 212 */
    int (*tcl_RegExpExec) (Tcl_Interp *interp, Tcl_RegExp regexp, CONST char *text, CONST char *start); /* 213 */
    int (*tcl_RegExpMatch) (Tcl_Interp *interp, CONST char *text, CONST char *pattern); /* 214 */
    void (*tcl_RegExpRange) (Tcl_RegExp regexp, int index, CONST84 char **startPtr, CONST84 char **endPtr); /* 215 */
    void (*tcl_Release) (ClientData clientData); /* 216 */
    void (*tcl_ResetResult) (Tcl_Interp *interp); /* 217 */
    int (*tcl_ScanElement) (CONST char *src, int *flagPtr); /* 218 */
    int (*tcl_ScanCountedElement) (CONST char *src, int length, int *flagPtr); /* 219 */
    int (*tcl_SeekOld) (Tcl_Channel chan, int offset, int mode); /* 220 */
    int (*tcl_ServiceAll) (void); /* 221 */
    int (*tcl_ServiceEvent) (int flags); /* 222 */
    void (*tcl_SetAssocData) (Tcl_Interp *interp, CONST char *name, Tcl_InterpDeleteProc *proc, ClientData clientData); /* 223 */
    void (*tcl_SetChannelBufferSize) (Tcl_Channel chan, int sz); /* 224 */
    int (*tcl_SetChannelOption) (Tcl_Interp *interp, Tcl_Channel chan, CONST char *optionName, CONST char *newValue); /* 225 */
    int (*tcl_SetCommandInfo) (Tcl_Interp *interp, CONST char *cmdName, CONST Tcl_CmdInfo *infoPtr); /* 226 */

Changes to generic/tclDictObj.c.

456
457
458
459
460
461
462
463
464
465
466
467
468

469
470
471
472
473
474
475







476
477
478
479
480
481
482


483
484
485
486
487
488
489
490
491
492

493
494
495
496


497


498
499
500
501


502





503
504
505
506
507

508
509
510

511
512
513
514
515
516

517
518
519
520
521
522


523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
 */

static void
UpdateStringOfDict(
    Tcl_Obj *dictPtr)
{
#define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr;
    Dict *dict = dictPtr->internalRep.otherValuePtr;
    ChainEntry *cPtr;
    Tcl_Obj *keyPtr, *valuePtr;
    int numElems, i, length;
    char *elem, *dst;


    /*
     * This field is the most useful one in the whole hash structure, and it
     * is not exposed by any API function...
     */

    numElems = dict->table.numEntries * 2;








    /*
     * Pass 1: estimate space, gather flags.
     */

    if (numElems <= LOCAL_SIZE) {
	flagPtr = localFlags;


    } else {
	flagPtr = (int *) ckalloc((unsigned) numElems*sizeof(int));
    }
    dictPtr->length = 1;
    for (i=0,cPtr=dict->entryChainHead; i<numElems; i+=2,cPtr=cPtr->nextPtr) {
	/*
	 * Assume that cPtr is never NULL since we know the number of array
	 * elements already.
	 */


	keyPtr = (Tcl_Obj *) Tcl_GetHashKey(&dict->table, &cPtr->entry);
	elem = TclGetStringFromObj(keyPtr, &length);
	dictPtr->length += Tcl_ScanCountedElement(elem, length,
		&flagPtr[i]) + 1;





	valuePtr = Tcl_GetHashValue(&cPtr->entry);
	elem = TclGetStringFromObj(valuePtr, &length);
	dictPtr->length += Tcl_ScanCountedElement(elem, length,
		&flagPtr[i+1]) + 1;


    }






    /*
     * Pass 2: copy into string rep buffer.
     */


    dictPtr->bytes = ckalloc((unsigned) dictPtr->length);
    dst = dictPtr->bytes;
    for (i=0,cPtr=dict->entryChainHead; i<numElems; i+=2,cPtr=cPtr->nextPtr) {

	keyPtr = (Tcl_Obj *) Tcl_GetHashKey(&dict->table, &cPtr->entry);
	elem = TclGetStringFromObj(keyPtr, &length);
	dst += Tcl_ConvertCountedElement(elem, length, dst,
		flagPtr[i] | (i==0 ? 0 : TCL_DONT_QUOTE_HASH));
	*(dst++) = ' ';


	valuePtr = Tcl_GetHashValue(&cPtr->entry);
	elem = TclGetStringFromObj(valuePtr, &length);
	dst += Tcl_ConvertCountedElement(elem, length, dst,
		flagPtr[i+1] | TCL_DONT_QUOTE_HASH);
	*(dst++) = ' ';
    }


    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }
    if (dst == dictPtr->bytes) {
	*dst = 0;
    } else {
	*(--dst) = 0;
    }
    dictPtr->length = dst - dictPtr->bytes;
}
 
/*
 *----------------------------------------------------------------------
 *
 * SetDictFromAny --
 *






|



|

>






|
>
>
>
>
>
>
>







>
>



<






>


|
<
>
>
|
>
>


|
<
>
>
|
>
>
>
>
>





>
|


>


|
<
|

>


|
<
|

>
>



<
<
<
<
<
<







456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495

496
497
498
499
500
501
502
503
504
505

506
507
508
509
510
511
512
513

514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534

535
536
537
538
539
540

541
542
543
544
545
546
547






548
549
550
551
552
553
554
 */

static void
UpdateStringOfDict(
    Tcl_Obj *dictPtr)
{
#define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr = NULL;
    Dict *dict = dictPtr->internalRep.otherValuePtr;
    ChainEntry *cPtr;
    Tcl_Obj *keyPtr, *valuePtr;
    int i, length, bytesNeeded = 0;
    char *elem, *dst;
    const int maxFlags = UINT_MAX / sizeof(int);

    /*
     * This field is the most useful one in the whole hash structure, and it
     * is not exposed by any API function...
     */

    int numElems = dict->table.numEntries * 2;

    /* Handle empty list case first, simplifies what follows */
    if (numElems == 0) {
	dictPtr->bytes = tclEmptyStringRep;
	dictPtr->length = 0;
	return;
    }

    /*
     * Pass 1: estimate space, gather flags.
     */

    if (numElems <= LOCAL_SIZE) {
	flagPtr = localFlags;
    } else if (numElems > maxFlags) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    } else {
	flagPtr = (int *) ckalloc((unsigned) numElems*sizeof(int));
    }

    for (i=0,cPtr=dict->entryChainHead; i<numElems; i+=2,cPtr=cPtr->nextPtr) {
	/*
	 * Assume that cPtr is never NULL since we know the number of array
	 * elements already.
	 */

	flagPtr[i] = ( i ? TCL_DONT_QUOTE_HASH : 0 );
	keyPtr = (Tcl_Obj *) Tcl_GetHashKey(&dict->table, &cPtr->entry);
	elem = TclGetStringFromObj(keyPtr, &length);
	bytesNeeded += TclScanElement(elem, length, flagPtr+i);

	if (bytesNeeded < 0) {
	    Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
	}

	flagPtr[i+1] = TCL_DONT_QUOTE_HASH;
	valuePtr = Tcl_GetHashValue(&cPtr->entry);
	elem = TclGetStringFromObj(valuePtr, &length);
	bytesNeeded += TclScanElement(elem, length, flagPtr+i+1);

	if (bytesNeeded < 0) {
	    Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
	}
    }
    if (bytesNeeded > INT_MAX - numElems + 1) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    }
    bytesNeeded += numElems;

    /*
     * Pass 2: copy into string rep buffer.
     */

    dictPtr->length = bytesNeeded - 1;
    dictPtr->bytes = ckalloc((unsigned) bytesNeeded);
    dst = dictPtr->bytes;
    for (i=0,cPtr=dict->entryChainHead; i<numElems; i+=2,cPtr=cPtr->nextPtr) {
	flagPtr[i] |= ( i ? TCL_DONT_QUOTE_HASH : 0 );
	keyPtr = (Tcl_Obj *) Tcl_GetHashKey(&dict->table, &cPtr->entry);
	elem = TclGetStringFromObj(keyPtr, &length);
	dst += TclConvertElement(elem, length, dst, flagPtr[i]);

	*dst++ = ' ';

	flagPtr[i+1] |= TCL_DONT_QUOTE_HASH;
	valuePtr = Tcl_GetHashValue(&cPtr->entry);
	elem = TclGetStringFromObj(valuePtr, &length);
	dst += TclConvertElement(elem, length, dst, flagPtr[i+1]);

	*dst++ = ' ';
    }
    dictPtr->bytes[dictPtr->length] = '\0';

    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }






}
 
/*
 *----------------------------------------------------------------------
 *
 * SetDictFromAny --
 *

Changes to generic/tclIndexObj.c.

533
534
535
536
537
538
539

540
541
542
543
544
545
546
547
548
549
550
551
552
...
587
588
589
590
591
592
593

594
595
596
597
598
599
600
601
602
603
604
605
606
			origObjv[i]->internalRep.otherValuePtr;

		elementStr = ecrPtr->fullSubcmdName;
		elemLen = strlen(elementStr);
	    } else {
		elementStr = TclGetStringFromObj(origObjv[i], &elemLen);
	    }

	    len = Tcl_ScanCountedElement(elementStr, elemLen, &flags);

	    if (MAY_QUOTE_WORD && len != elemLen) {
		char *quotedElementStr = TclStackAlloc(interp, (unsigned)len);

		len = Tcl_ConvertCountedElement(elementStr, elemLen,
			quotedElementStr, flags);
		Tcl_AppendToObj(objPtr, quotedElementStr, len);
		TclStackFree(interp, quotedElementStr);
	    } else {
		Tcl_AppendToObj(objPtr, elementStr, elemLen);
	    }

................................................................................
	    Tcl_AppendStringsToObj(objPtr, ecrPtr->fullSubcmdName, NULL);
	} else {
	    /*
	     * Quote the argument if it contains spaces (Bug 942757).
	     */

	    elementStr = TclGetStringFromObj(objv[i], &elemLen);

	    len = Tcl_ScanCountedElement(elementStr, elemLen, &flags);

	    if (MAY_QUOTE_WORD && len != elemLen) {
		char *quotedElementStr = TclStackAlloc(interp,(unsigned) len);

		len = Tcl_ConvertCountedElement(elementStr, elemLen,
			quotedElementStr, flags);
		Tcl_AppendToObj(objPtr, quotedElementStr, len);
		TclStackFree(interp, quotedElementStr);
	    } else {
		Tcl_AppendToObj(objPtr, elementStr, elemLen);
	    }
	}






>
|




|







 







>
|




|







533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
...
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
			origObjv[i]->internalRep.otherValuePtr;

		elementStr = ecrPtr->fullSubcmdName;
		elemLen = strlen(elementStr);
	    } else {
		elementStr = TclGetStringFromObj(origObjv[i], &elemLen);
	    }
	    flags = 0;
	    len = TclScanElement(elementStr, elemLen, &flags);

	    if (MAY_QUOTE_WORD && len != elemLen) {
		char *quotedElementStr = TclStackAlloc(interp, (unsigned)len);

		len = TclConvertElement(elementStr, elemLen,
			quotedElementStr, flags);
		Tcl_AppendToObj(objPtr, quotedElementStr, len);
		TclStackFree(interp, quotedElementStr);
	    } else {
		Tcl_AppendToObj(objPtr, elementStr, elemLen);
	    }

................................................................................
	    Tcl_AppendStringsToObj(objPtr, ecrPtr->fullSubcmdName, NULL);
	} else {
	    /*
	     * Quote the argument if it contains spaces (Bug 942757).
	     */

	    elementStr = TclGetStringFromObj(objv[i], &elemLen);
	    flags = 0;
	    len = TclScanElement(elementStr, elemLen, &flags);

	    if (MAY_QUOTE_WORD && len != elemLen) {
		char *quotedElementStr = TclStackAlloc(interp,(unsigned) len);

		len = TclConvertElement(elementStr, elemLen,
			quotedElementStr, flags);
		Tcl_AppendToObj(objPtr, quotedElementStr, len);
		TclStackFree(interp, quotedElementStr);
	    } else {
		Tcl_AppendToObj(objPtr, elementStr, elemLen);
	    }
	}

Changes to generic/tclInt.h.

2569
2570
2571
2572
2573
2574
2575


2576
2577
2578
2579
2580
2581
2582
....
2777
2778
2779
2780
2781
2782
2783


2784
2785
2786
2787
2788
2789
2790
MODULE_SCOPE ContLineLoc* TclContinuationsEnter(Tcl_Obj *objPtr, int num,
			    int *loc);
MODULE_SCOPE void	TclContinuationsEnterDerived(Tcl_Obj *objPtr,
			    int start, int *clNext);
MODULE_SCOPE ContLineLoc* TclContinuationsGet(Tcl_Obj *objPtr);
MODULE_SCOPE void	TclContinuationsCopy(Tcl_Obj *objPtr,
			    Tcl_Obj *originObjPtr);


MODULE_SCOPE void	TclDeleteNamespaceVars(Namespace *nsPtr);
/* TIP #280 - Modified token based evulation, with line information. */
MODULE_SCOPE int	TclEvalEx(Tcl_Interp *interp, const char *script,
			    int numBytes, int flags, int line,
			    int *clNextOuter, CONST char *outerScript);
MODULE_SCOPE int	TclFileAttrsCmd(Tcl_Interp *interp,
			    int objc, Tcl_Obj *const objv[]);
................................................................................
MODULE_SCOPE size_t	TclpThreadGetStackSize(void);
MODULE_SCOPE void	TclRememberCondition(Tcl_Condition *mutex);
MODULE_SCOPE void	TclRememberJoinableThread(Tcl_ThreadId id);
MODULE_SCOPE void	TclRememberMutex(Tcl_Mutex *mutex);
MODULE_SCOPE void	TclRemoveScriptLimitCallbacks(Tcl_Interp *interp);
MODULE_SCOPE int	TclReToGlob(Tcl_Interp *interp, const char *reStr,
			    int reStrLen, Tcl_DString *dsPtr, int *flagsPtr);


MODULE_SCOPE void	TclSetBgErrorHandler(Tcl_Interp *interp,
			    Tcl_Obj *cmdPrefix);
MODULE_SCOPE void	TclSetBignumIntRep(Tcl_Obj *objPtr,
			    mp_int *bignumValue);
MODULE_SCOPE void	TclSetCmdNameObj(Tcl_Interp *interp, Tcl_Obj *objPtr,
			    Command *cmdPtr);
MODULE_SCOPE void	TclSetProcessGlobalValue(ProcessGlobalValue *pgvPtr,






>
>







 







>
>







2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
....
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
MODULE_SCOPE ContLineLoc* TclContinuationsEnter(Tcl_Obj *objPtr, int num,
			    int *loc);
MODULE_SCOPE void	TclContinuationsEnterDerived(Tcl_Obj *objPtr,
			    int start, int *clNext);
MODULE_SCOPE ContLineLoc* TclContinuationsGet(Tcl_Obj *objPtr);
MODULE_SCOPE void	TclContinuationsCopy(Tcl_Obj *objPtr,
			    Tcl_Obj *originObjPtr);
MODULE_SCOPE int	TclConvertElement(CONST char *src, int length,
			    char *dst, int flags);
MODULE_SCOPE void	TclDeleteNamespaceVars(Namespace *nsPtr);
/* TIP #280 - Modified token based evulation, with line information. */
MODULE_SCOPE int	TclEvalEx(Tcl_Interp *interp, const char *script,
			    int numBytes, int flags, int line,
			    int *clNextOuter, CONST char *outerScript);
MODULE_SCOPE int	TclFileAttrsCmd(Tcl_Interp *interp,
			    int objc, Tcl_Obj *const objv[]);
................................................................................
MODULE_SCOPE size_t	TclpThreadGetStackSize(void);
MODULE_SCOPE void	TclRememberCondition(Tcl_Condition *mutex);
MODULE_SCOPE void	TclRememberJoinableThread(Tcl_ThreadId id);
MODULE_SCOPE void	TclRememberMutex(Tcl_Mutex *mutex);
MODULE_SCOPE void	TclRemoveScriptLimitCallbacks(Tcl_Interp *interp);
MODULE_SCOPE int	TclReToGlob(Tcl_Interp *interp, const char *reStr,
			    int reStrLen, Tcl_DString *dsPtr, int *flagsPtr);
MODULE_SCOPE int	TclScanElement(CONST char *string, int length,
			    int *flagPtr);
MODULE_SCOPE void	TclSetBgErrorHandler(Tcl_Interp *interp,
			    Tcl_Obj *cmdPrefix);
MODULE_SCOPE void	TclSetBignumIntRep(Tcl_Obj *objPtr,
			    mp_int *bignumValue);
MODULE_SCOPE void	TclSetCmdNameObj(Tcl_Interp *interp, Tcl_Obj *objPtr,
			    Command *cmdPtr);
MODULE_SCOPE void	TclSetProcessGlobalValue(ProcessGlobalValue *pgvPtr,

Changes to generic/tclListObj.c.

1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819

1820










1821
1822
1823
1824
1825
1826
1827
1828

1829
1830
1831
1832
1833

1834
1835


1836
1837
1838
1839
1840
1841
1842
1843
1844

1845
1846
1847
1848
1849

1850
1851
1852

1853
1854
1855
1856
1857
1858


1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
 */

static void
UpdateStringOfList(
    Tcl_Obj *listPtr)		/* List object with string rep to update. */
{
#   define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr;
    List *listRepPtr = ListRepPtr(listPtr);
    int numElems = listRepPtr->elemCount;
    register int i;
    char *elem, *dst;
    int length;
    Tcl_Obj **elemPtrs;

    /*
     * Convert each element of the list to string form and then convert it to
     * proper list element form, adding it to the result buffer.

     */











    /*
     * Pass 1: estimate space, gather flags.
     */

    if (numElems <= LOCAL_SIZE) {
	flagPtr = localFlags;
    } else {

	flagPtr = (int *) ckalloc((unsigned) numElems * sizeof(int));
    }
    listPtr->length = 1;
    elemPtrs = &listRepPtr->elements;
    for (i = 0; i < numElems; i++) {

	elem = TclGetStringFromObj(elemPtrs[i], &length);
	listPtr->length += Tcl_ScanCountedElement(elem, length, flagPtr+i)+1;



	/*
	 * Check for continued sanity. [Bug 1267380]
	 */

	if (listPtr->length < 1) {
	    Tcl_Panic("string representation size exceeds sane bounds");
	}
    }


    /*
     * Pass 2: copy into string rep buffer.
     */


    listPtr->bytes = ckalloc((unsigned) listPtr->length);
    dst = listPtr->bytes;
    for (i = 0; i < numElems; i++) {

	elem = TclGetStringFromObj(elemPtrs[i], &length);
	dst += Tcl_ConvertCountedElement(elem, length, dst,
		flagPtr[i] | (i==0 ? 0 : TCL_DONT_QUOTE_HASH));
	*dst = ' ';
	dst++;
    }


    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }
    if (dst == listPtr->bytes) {
	*dst = 0;
    } else {
	dst--;
	*dst = 0;
    }
    listPtr->length = dst - listPtr->bytes;

    /*
     * Mark the list as being canonical; although it has a string rep, it is
     * one we derived through proper "canonical" quoting and so it's known to
     * be free from nasties relating to [concat] and [eval].
     */

    listRepPtr->canonicalFlag = 1;
}
 
/*
 * Local Variables:
 * mode: c
 * c-basic-offset: 4
 * fill-column: 78
 * End:
 */






|


|

<



|
|
>

>
>
>
>
>
>
>
>
>
>








>


<


>

|
>
>
|
<
<
<
|
|
|
|
<
>





>
|


>

|
<
|
<

>
>



<
<
<
<
<
<
<
<
<
<
<
<
<
<
<









1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813

1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841

1842
1843
1844
1845
1846
1847
1848
1849



1850
1851
1852
1853

1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866

1867

1868
1869
1870
1871
1872
1873















1874
1875
1876
1877
1878
1879
1880
1881
1882
 */

static void
UpdateStringOfList(
    Tcl_Obj *listPtr)		/* List object with string rep to update. */
{
#   define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr = NULL;
    List *listRepPtr = ListRepPtr(listPtr);
    int numElems = listRepPtr->elemCount;
    int i, length, bytesNeeded = 0;
    char *elem, *dst;

    Tcl_Obj **elemPtrs;

    /*
     * Mark the list as being canonical; although it will now have a string
     * rep, it is one we derived through proper "canonical" quoting and so
     * it's known to be free from nasties relating to [concat] and [eval].
     */

    listRepPtr->canonicalFlag = 1;

    /* Handle empty list case first, so rest of the routine is simpler */

    if (numElems == 0) {
	listPtr->bytes = tclEmptyStringRep;
	listPtr->length = 0;
	return;
    }

    /*
     * Pass 1: estimate space, gather flags.
     */

    if (numElems <= LOCAL_SIZE) {
	flagPtr = localFlags;
    } else {
	/* We know numElems <= LIST_MAX, so this is safe. */
	flagPtr = (int *) ckalloc((unsigned) numElems * sizeof(int));
    }

    elemPtrs = &listRepPtr->elements;
    for (i = 0; i < numElems; i++) {
	flagPtr[i] = ( i ? TCL_DONT_QUOTE_HASH : 0 );
	elem = TclGetStringFromObj(elemPtrs[i], &length);
	bytesNeeded += TclScanElement(elem, length, flagPtr+i);
	if (bytesNeeded < 0) {
	    Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
	}



    }
    if (bytesNeeded > INT_MAX - numElems + 1) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    }

    bytesNeeded += numElems;

    /*
     * Pass 2: copy into string rep buffer.
     */

    listPtr->length = bytesNeeded - 1;
    listPtr->bytes = ckalloc((unsigned) bytesNeeded);
    dst = listPtr->bytes;
    for (i = 0; i < numElems; i++) {
	flagPtr[i] |= ( i ? TCL_DONT_QUOTE_HASH : 0 );
	elem = TclGetStringFromObj(elemPtrs[i], &length);
	dst += TclConvertElement(elem, length, dst, flagPtr[i]);

	*dst++ = ' ';

    }
    listPtr->bytes[listPtr->length] = '\0';

    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }















}
 
/*
 * Local Variables:
 * mode: c
 * c-basic-offset: 4
 * fill-column: 78
 * End:
 */

Changes to generic/tclUtil.c.

22
23
24
25
26
27
28
29
30
31
32
33
34
35


36
37
38

39
40
41
42
43
44
45
46
47
48
49






































50
51
52
53





54
55
56
57
58
59
60
..
82
83
84
85
86
87
88









































































































































































































































89
90
91
92
93
94
95
...
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
...
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617




618
619

620
621
622
623

624
625
626
627
628
629




630
631
632










633
634
635
636
637
638


639
640
641
642
643
644
645

646
647
648
649
650
651
652
653
654
655
656
657
658

659
660
661
662












663
664
665




666
667
668

669
670








671
672


673
674




675
676
677




678
679
680


681
682










683
684
685
686
687
688
689
690
691
692




693
694






695
696
697
698






699
700
701
702




703




704



705


706





707

































708
709
























710

711


712
713
714


715




716
717




















718
719
720
721
722
723
724
...
771
772
773
774
775
776
777
778
779

780

781
782
783


















784
785
786
787








788






789








790
791
792
793




794












795
796
797
798
799




800
801
802



803
804
805
806
807
808
809
810
811
812
813
814


815
816
817
818
819
820
821
822
823
824
825

826
827
828
829
830
831
832




833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855



856
857

858

859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889



890









891
892
893
894
895
896
897
898
899
900
901
902
903
...
918
919
920
921
922
923
924
925
926
927
928
929










930
931
932
933
934
935
936














937
938
939
940
941

942


943





944
945
946
947
948
949
950
951

952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
....
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
....
1994
1995
1996
1997
1998
1999
2000

2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022

2023
2024
2025
2026
2027
2028
2029
 */

static ProcessGlobalValue executableName = {
    0, 0, NULL, NULL, NULL, NULL, NULL
};

/*
 * The following values are used in the flags returned by Tcl_ScanElement and
 * used by Tcl_ConvertElement. The values TCL_DONT_USE_BRACES and
 * TCL_DONT_QUOTE_HASH are defined in tcl.h; make sure neither value overlaps
 * with any of the values below.
 *
 * TCL_DONT_USE_BRACES -	1 means the string mustn't be enclosed in
 *				braces (e.g. it contains unmatched braces, or


 *				ends in a backslash character, or user just
 *				doesn't want braces); handle all special
 *				characters by adding backslashes.

 * USE_BRACES -			1 means the string contains a special
 *				character that can be handled simply by
 *				enclosing the entire argument in braces.
 * BRACES_UNMATCHED -		1 means that braces aren't properly matched in
 *				the argument.
 * TCL_DONT_QUOTE_HASH -	1 means the caller insists that a leading hash
 * 				character ('#') should *not* be quoted. This
 * 				is appropriate when the caller can guarantee
 * 				the element is not the first element of a
 * 				list, so [eval] cannot mis-parse the element
 * 				as a comment.






































 */

#define USE_BRACES		2
#define BRACES_UNMATCHED	4






/*
 * The following key is used by Tcl_PrintDouble and TclPrecTraceProc to
 * access the precision to be used for double formatting.
 */

static Tcl_ThreadDataKey precisionKey;
................................................................................
    "end-offset",			/* name */
    NULL,				/* freeIntRepProc */
    NULL,				/* dupIntRepProc */
    UpdateStringOfEndOffset,		/* updateStringProc */
    SetEndOffsetFromAny
};
 









































































































































































































































/*
 *----------------------------------------------------------------------
 *
 * TclMaxListLength --
 *
 *	Given 'bytes' pointing to 'numBytes' bytes, scan through them and
 *	count the number of whitespace runs that could be list element
................................................................................
 *
 *	If TCL_OK is returned, then *elementPtr will be set to point to the
 *	first element of list, and *nextPtr will be set to point to the
 *	character just after any white space following the last character
 *	that's part of the element. If this is the last argument in the list,
 *	then *nextPtr will point just after the last character in the list
 *	(i.e., at the character at list+listLength). If sizePtr is non-NULL,
 *	*sizePtr is filled in with the number of characters in the element. If
 *	the element is in braces, then *elementPtr will point to the character
 *	after the opening brace and *sizePtr will not include either of the
 *	braces. If there isn't an element in the list, *sizePtr will be zero,
 *	and both *elementPtr and *nextPtr will point just after the last
 *	character in the list. If literalPtr is non-NULL, *literalPtr is set
 *	to a boolean value indicating whether the substring returned as
 *	the values of **elementPtr and *sizePtr is the literal value of
................................................................................
 * Tcl_ScanElement --
 *
 *	This function is a companion function to Tcl_ConvertElement. It scans
 *	a string to see what needs to be done to it (e.g. add backslashes or
 *	enclosing braces) to make the string into a valid Tcl list element.
 *
 * Results:
 *	The return value is an overestimate of the number of characters that
 *	will be needed by Tcl_ConvertElement to produce a valid list element
 *	from string. The word at *flagPtr is filled in with a value needed by
 *	Tcl_ConvertElement when doing the actual conversion.
 *
 * Side effects:
 *	None.
 *
 *----------------------------------------------------------------------
 */

int
Tcl_ScanElement(
    register CONST char *string,/* String to convert to list element. */
    register int *flagPtr)	/* Where to store information to guide
				 * Tcl_ConvertCountedElement. */
{
    return Tcl_ScanCountedElement(string, -1, flagPtr);
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_ScanCountedElement --
 *
 *	This function is a companion function to Tcl_ConvertCountedElement. It
 *	scans a string to see what needs to be done to it (e.g. add
 *	backslashes or enclosing braces) to make the string into a valid Tcl
 *	list element. If length is -1, then the string is scanned up to the
 *	first null byte.
 *
 * Results:
 *	The return value is an overestimate of the number of characters that
 *	will be needed by Tcl_ConvertCountedElement to produce a valid list
 *	element from string. The word at *flagPtr is filled in with a value
 *	needed by Tcl_ConvertCountedElement when doing the actual conversion.
 *
 * Side effects:
 *	None.
 *
 *----------------------------------------------------------------------
 */

int
Tcl_ScanCountedElement(
    CONST char *string,		/* String to convert to Tcl list element. */
    int length,			/* Number of bytes in string, or -1. */
    int *flagPtr)		/* Where to store information to guide
				 * Tcl_ConvertElement. */
{
    int flags, nestingLevel;
    register CONST char *p, *lastChar;





    /*
     * This function and Tcl_ConvertElement together do two things:

     *
     * 1. They produce a proper list, one that will yield back the argument
     *	  strings when evaluated or when disassembled with Tcl_SplitList. This
     *	  is the most important thing.

     *
     * 2. They try to produce legible output, which means minimizing the use
     *	  of backslashes (using braces instead). However, there are some
     *	  situations where backslashes must be used (e.g. an element like
     *	  "{abc": the leading brace will have to be backslashed. For each
     *	  element, one of three things must be done:




     *
     * 	  (a) Use the element as-is (it doesn't contain any special
     *	      characters). This is the most desirable option.










     *
     *	  (b) Enclose the element in braces, but leave the contents alone.
     *	      This happens if the element contains embedded space, or if it
     *	      contains characters with special interpretation ($, [, ;, or \),
     *	      or if it starts with a brace or double-quote, or if there are no
     *	      characters in the element.


     *
     *	  (c) Don't enclose the element in braces, but add backslashes to
     *	      prevent special interpretation of special characters. This is a
     *	      last resort used when the argument would normally fall under
     *	      case (b) but contains unmatched braces. It also occurs if the
     *	      last character of the argument is a backslash or if the element
     *	      contains a backslash followed by newline.

     *
     * The function figures out how many bytes will be needed to store the
     * result (actually, it overestimates). It also collects information about
     * the element in the form of a flags word.
     *
     * Note: list elements produced by this function and
     * Tcl_ConvertCountedElement must have the property that they can be
     * enclosing in curly braces to make sub-lists. This means, for example,
     * that we must not leave unmatched curly braces in the resulting list
     * element. This property is necessary in order for functions like
     * Tcl_DStringStartSublist to work.
     */


    nestingLevel = 0;
    flags = 0;
    if (string == NULL) {
	string = "";












    }
    if (length == -1) {
	length = strlen(string);




    }
    lastChar = string + length;
    p = string;

    if ((p == lastChar) || (*p == '{') || (*p == '"')) {
	flags |= USE_BRACES;








    }
    for (; p < lastChar; p++) {


	switch (*p) {
	case '{':




	    nestingLevel++;
	    break;
	case '}':




	    nestingLevel--;
	    if (nestingLevel < 0) {
		flags |= TCL_DONT_USE_BRACES|BRACES_UNMATCHED;


	    }
	    break;










	case '[':
	case '$':
	case ';':
	case ' ':
	case '\f':
	case '\n':
	case '\r':
	case '\t':
	case '\v':
	    flags |= USE_BRACES;




	    break;
	case '\\':






	    if ((p+1 == lastChar) || (p[1] == '\n')) {
		flags = TCL_DONT_USE_BRACES | BRACES_UNMATCHED;
	    } else {
		int size;







		TclParseBackslash(p, lastChar - p, &size, NULL);
		p += size-1;
		flags |= USE_BRACES;




	    }




	    break;



	}


    }





    if (nestingLevel != 0) {

































	flags = TCL_DONT_USE_BRACES | BRACES_UNMATCHED;
    }
























    *flagPtr = flags;




    /*
     * Allow enough space to backslash every character plus leave two spaces
     * for braces.


     */





    return 2*(p-string) + 2;




















}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_ConvertElement --
 *
................................................................................
int
Tcl_ConvertCountedElement(
    register CONST char *src,	/* Source information for list element. */
    int length,			/* Number of bytes in src, or -1. */
    char *dst,			/* Place to put list-ified element. */
    int flags)			/* Flags produced by Tcl_ScanElement. */
{
    register char *p = dst;
    register CONST char *lastChar;



    /*
     * See the comment block at the beginning of the Tcl_ScanElement code for
     * details of how this works.


















     */

    if (src && length == -1) {
	length = strlen(src);








    }






    if ((src == NULL) || (length == 0)) {








	p[0] = '{';
	p[1] = '}';
	p[2] = 0;
	return 2;




    }












    lastChar = src + length;
    if ((*src == '#') && !(flags & TCL_DONT_QUOTE_HASH)) {
	flags |= USE_BRACES;
    }
    if ((flags & USE_BRACES) && !(flags & TCL_DONT_USE_BRACES)) {




	*p = '{';
	p++;
	for (; src != lastChar; src++, p++) {



	    *p = *src;
	}
	*p = '}';
	p++;
    } else {
	if (*src == '{') {
	    /*
	     * Can't have a leading brace unless the whole element is enclosed
	     * in braces. Add a backslash before the brace. Furthermore, this
	     * may destroy the balance between open and close braces, so set
	     * BRACES_UNMATCHED.
	     */



	    p[0] = '\\';
	    p[1] = '{';
	    p += 2;
	    src++;
	    flags |= BRACES_UNMATCHED;
	} else if ((*src == '#') && !(flags & TCL_DONT_QUOTE_HASH)) {
	    /*
	     * Leading '#' could be seen by [eval] as the start of a comment,
	     * if on the first element of a list, so quote it.
	     */


	    p[0] = '\\';
	    p[1] = '#';
	    p += 2;
	    src++;
	}
	for (; src != lastChar; src++) {




	    switch (*src) {
	    case ']':
	    case '[':
	    case '$':
	    case ';':
	    case ' ':
	    case '\\':
	    case '"':
		*p = '\\';
		p++;
		break;
	    case '{':
	    case '}':
		/*
		 * It may not seem necessary to backslash braces, but it is.
		 * The reason for this is that the resulting list element may
		 * actually be an element of a sub-list enclosed in braces
		 * (e.g. if Tcl_DStringStartSublist has been invoked), so
		 * there may be a brace mismatch if the braces aren't
		 * backslashed.
		 */

		if (flags & BRACES_UNMATCHED) {



		    *p = '\\';
		    p++;

		}

		break;
	    case '\f':
		*p = '\\';
		p++;
		*p = 'f';
		p++;
		continue;
	    case '\n':
		*p = '\\';
		p++;
		*p = 'n';
		p++;
		continue;
	    case '\r':
		*p = '\\';
		p++;
		*p = 'r';
		p++;
		continue;
	    case '\t':
		*p = '\\';
		p++;
		*p = 't';
		p++;
		continue;
	    case '\v':
		*p = '\\';
		p++;
		*p = 'v';
		p++;
		continue;



	    }









	    *p = *src;
	    p++;
	}
    }
    *p = '\0';
    return p-dst;
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_Merge --
 *
................................................................................

char *
Tcl_Merge(
    int argc,			/* How many strings to merge. */
    CONST char * CONST *argv)	/* Array of string values. */
{
#   define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr;
    int numChars;
    char *result;
    char *dst;
    int i;











    /*
     * Pass 1: estimate space, gather flags.
     */

    if (argc <= LOCAL_SIZE) {
	flagPtr = localFlags;














    } else {
	flagPtr = (int *) ckalloc((unsigned) argc*sizeof(int));
    }
    numChars = 1;
    for (i = 0; i < argc; i++) {

	numChars += Tcl_ScanElement(argv[i], &flagPtr[i]) + 1;


    }






    /*
     * Pass two: copy into the result area.
     */

    result = (char *) ckalloc((unsigned) numChars);
    dst = result;
    for (i = 0; i < argc; i++) {

	numChars = Tcl_ConvertElement(argv[i], dst,
		flagPtr[i] | (i==0 ? 0 : TCL_DONT_QUOTE_HASH));
	dst += numChars;
	*dst = ' ';
	dst++;
    }
    if (dst == result) {
	*dst = 0;
    } else {
	dst[-1] = 0;
    }

    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }
    return result;
}
 
................................................................................

char *
Tcl_DStringAppendElement(
    Tcl_DString *dsPtr,		/* Structure describing dynamic string. */
    CONST char *element)	/* String to append. Must be
				 * null-terminated. */
{
    int newSize, flags, strSize;
    char *dst;

    strSize = ((element== NULL) ? 0 : strlen(element));
    newSize = Tcl_ScanCountedElement(element, strSize, &flags)
	+ dsPtr->length + 1;

    /*
     * Allocate a larger buffer for the string if the current one isn't large
     * enough. Allocate extra space in the new buffer so that there will be
     * room to grow before we have to allocate again. SPECIAL NOTE: must use
     * memcpy, not strcpy, to copy the string to a larger buffer, since there
     * may be embedded NULLs in the string in some cases.
................................................................................

	    memcpy(newString, dsPtr->string, (size_t) dsPtr->length);
	    dsPtr->string = newString;
	} else {
	    dsPtr->string = (char *) ckrealloc((void *) dsPtr->string,
		    (size_t) dsPtr->spaceAvl);
	}

    }

    /*
     * Convert the new string to a list element and copy it into the buffer at
     * the end, with a space, if needed.
     */

    dst = dsPtr->string + dsPtr->length;
    if (TclNeedSpace(dsPtr->string, dst)) {
	*dst = ' ';
	dst++;
	dsPtr->length++;

	/*
	 * If we need a space to separate this element from preceding stuff,
	 * then this element will not lead a list, and need not have it's
	 * leading '#' quoted.
	 */

	flags |= TCL_DONT_QUOTE_HASH;
    }
    dsPtr->length += Tcl_ConvertCountedElement(element, strSize, dst, flags);

    return dsPtr->string;
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_DStringSetLength --






|
|
|
<

|
<
>
>
|
|
<
>
|
|
|
<
<






>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


|
<
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|







 







|

|










|



|










|
|


|

|










|
|



|
|

>
>
>
>
|
<
>
|
<
<
<
>
|
<
<
|
|
|
>
>
>
>
|
<
<
>
>
>
>
>
>
>
>
>
>
|
<
<
<
<
<
>
>
|
<
<
<
<
<
<
>
|
<
<
<
|
<
|
|
|
|
|
|
|
>
|
<
|
|
>
>
>
>
>
>
>
>
>
>
>
>
|
<
<
>
>
>
>

<
<
>
|
<
>
>
>
>
>
>
>
>

<
>
>


>
>
>
>



>
>
>
>


<
>
>


>
>
>
>
>
>
>
>
>
>









|
>
>
>
>


>
>
>
>
>
>
|
<
<
<
>
>
>
>
>
>
|
<
<
<
>
>
>
>

>
>
>
>

>
>
>
|
>
>
|
>
>
>
>
>

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
|
>
>
|
<
<
>
>
|
>
>
>
>
|
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
|
>
|
>
|
<
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|

<
<
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
|
|
<
|
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
|
<
<
|
<
>
>
>
>


<
>
>
>
|
|
<
<
|
<
<
<
<
<
<
<
>
>
|
|
<
<
|
<
<
<
<
<
<
>
|
<
<
<
<
|
<
>
>
>
>
|
|
|
|
|
|
|
|
|
|
|
|
|
<
<
<
<
<
<
<
<
<
<
>
>
>
|
|
>
|
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>
>
>

>
>
>
>
>
>
>
>
>
|
|
|
<
<
|







 







|
|
|
|
<
>
>
>
>
>
>
>
>
>
>







>
>
>
>
>
>
>
>
>
>
>
>
>
>



<

>
|
>
>
|
>
>
>
>
>





|


>
|
<
<



<
<
<
|
<







 







|
|
|
|
|
<







 







>







|
<












|
>







22
23
24
25
26
27
28
29
30
31

32
33

34
35
36
37

38
39
40
41


42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98
99
100
...
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
...
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
...
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895

896
897



898
899


900
901
902
903
904
905
906
907


908
909
910
911
912
913
914
915
916
917
918





919
920
921






922
923



924

925
926
927
928
929
930
931
932
933

934
935
936
937
938
939
940
941
942
943
944
945
946
947
948


949
950
951
952
953


954
955

956
957
958
959
960
961
962
963
964

965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981

982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018



1019
1020
1021
1022
1023
1024
1025



1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113


1114
1115
1116
1117
1118
1119
1120
1121

1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
....
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207


1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227


1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253

1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272


1273

1274
1275
1276
1277
1278
1279

1280
1281
1282
1283
1284


1285







1286
1287
1288
1289


1290






1291
1292




1293

1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310










1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365


1366
1367
1368
1369
1370
1371
1372
1373
....
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398

1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432

1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453


1454
1455
1456



1457

1458
1459
1460
1461
1462
1463
1464
....
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474

2475
2476
2477
2478
2479
2480
2481
....
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503

2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
 */

static ProcessGlobalValue executableName = {
    0, 0, NULL, NULL, NULL, NULL, NULL
};

/*
 * The following values are used in the flags arguments of Tcl*Scan*Element and
 * Tcl*Convert*Element. The values TCL_DONT_USE_BRACES and TCL_DONT_QUOTE_HASH
 * are defined in tcl.h, like so:

 *
#define TCL_DONT_USE_BRACES     1

#define TCL_DONT_QUOTE_HASH     8
 *
 * Those are public flag bits which callers of the public routines
 * Tcl_Convert*Element() can use to indicate:

 *
 * TCL_DONT_USE_BRACES -	1 means the caller is insisting that brace
 *				quoting not be used when converting the list
 *				element.


 * TCL_DONT_QUOTE_HASH -	1 means the caller insists that a leading hash
 * 				character ('#') should *not* be quoted. This
 * 				is appropriate when the caller can guarantee
 * 				the element is not the first element of a
 * 				list, so [eval] cannot mis-parse the element
 * 				as a comment.
 *
 * The remaining values which can be carried by the flags of these routines
 * are for internal use only.  Make sure they do not overlap with the public
 * values above.
 *
 * The Tcl*Scan*Element() routines make a determination which of 4 modes of
 * conversion is most appropriate for Tcl*Convert*Element() to perform, and
 * sets two bits of the flags value to indicate the mode selected.
 *
 * CONVERT_NONE		The element needs no quoting.  Its literal string
 *			is suitable as is.
 * CONVERT_BRACE	The conversion should be enclosing the literal string
 *			in braces.
 * CONVERT_ESCAPE	The conversion should be using backslashes to escape
 *			any characters in the string that require it.
 * CONVERT_MASK		A mask value used to extract the conversion mode from
 *			the flags argument.
 *			Also indicates a strange conversion mode where all
 *			special characters are escaped with backslashes 
 *			*except for braces*.  This is a strange and unnecessary
 *			case, but it's part of the historical way in which
 *			lists have been formatted in Tcl.  To experiment with
 *			removing this case, set the value of COMPAT to 0.
 *
 * One last flag value is used only by callers of TclScanElement().  The flag
 * value produced by a call to Tcl*Scan*Element() will never leave this bit
 * set.
 *
 * CONVERT_ANY		The caller of TclScanElement() declares it can make
 *			no promise about what public flags will be passed to
 *			the matching call of TclConvertElement().  As such,
 *			TclScanElement() has to determine the worst case
 *			destination buffer length over all possibilities, and
 *			in other cases this means an overestimate of the
 *			required size.
 *
 * For more details, see the comments on the Tcl*Scan*Element and 
 * Tcl*Convert*Element routines.
 */

#define COMPAT 1

#define CONVERT_NONE	0
#define CONVERT_BRACE	2
#define CONVERT_ESCAPE	4
#define CONVERT_MASK	(CONVERT_BRACE | CONVERT_ESCAPE)
#define CONVERT_ANY	16

/*
 * The following key is used by Tcl_PrintDouble and TclPrecTraceProc to
 * access the precision to be used for double formatting.
 */

static Tcl_ThreadDataKey precisionKey;
................................................................................
    "end-offset",			/* name */
    NULL,				/* freeIntRepProc */
    NULL,				/* dupIntRepProc */
    UpdateStringOfEndOffset,		/* updateStringProc */
    SetEndOffsetFromAny
};
 
/*
 *	*	STRING REPRESENTATION OF LISTS	*	*	*
 *
 * The next several routines implement the conversions of strings to and
 * from Tcl lists.  To understand their operation, the rules of parsing
 * and generating the string representation of lists must be known.  Here
 * we describe them in one place.
 *
 * A list is made up of zero or more elements.  Any string is a list if
 * it is made up of alternating substrings of element-separating ASCII
 * whitespace and properly formatted elements.
 *
 * The ASCII characters which can make up the whitespace between list
 * elements are:
 *
 *	\u0009	\t	TAB
 *	\u000A	\n	NEWLINE
 *	\u000B	\v	VERTICAL TAB
 *	\u000C	\f	FORM FEED
 * 	\u000D	\r	CARRIAGE RETURN
 *	\u0020		SPACE
 *
 * NOTE: differences between this and other places where Tcl defines a role
 * for "whitespace".
 *
 *	* Unlike command parsing, here NEWLINE is just another whitespace
 *	  character; its role as a command terminator in a script has no
 *	  importance here.
 *
 *	* Unlike command parsing, the BACKSLASH NEWLINE sequence is not
 *	  considered to be a whitespace character.
 *
 *	* Other Unicode whitespace characters (recognized by
 *	  [string is space] or Tcl_UniCharIsSpace()) do not play any role
 *	  as element separators in Tcl lists.
 *
 *	* The NUL byte ought not appear, as it is not in strings properly
 *	  encoded for Tcl, but if it is present, it is not treated as
 *	  separating whitespace, or a string terminator.  It is just
 *	  another character in a list element.
 *
 * The interpretaton of a formatted substring as a list element follows
 * rules similar to the parsing of the words of a command in a Tcl script.
 * Backslash substitution plays a key role, and is defined exactly as it is
 * in command parsing.  The same routine, TclParseBackslash() is used in both
 * command parsing and list parsing.  
 *
 * NOTE:  This means that if and when backslash substitution rules ever
 * change for command parsing, the interpretation of strings as lists also
 * changes.
 * 
 * Backslash substitution replaces an "escape sequence" of one or more
 * characters starting with
 *		\u005c	\	BACKSLASH
 * with a single character.  The one character escape sequent case happens
 * only when BACKSLASH is the last character in the string.  In all other
 * cases, the escape sequence is at least two characters long.
 *
 * The formatted substrings are interpreted as element values according to
 * the following cases:
 *
 * * If the first character of a formatted substring is
 *		\u007b	{	OPEN BRACE
 *   then the end of the substring is the matching 
 *		\u007d	}	CLOSE BRACE
 *   character, where matching is determined by counting nesting levels,
 *   and not including any brace characters that are contained within a
 *   backslash escape sequence in the nesting count.  Having found the
 *   matching brace, all characters between the braces are the string
 *   value of the element.  If no matching close brace is found before the
 *   end of the string, the string is not a Tcl list.  If the character
 *   following the close brace is not an element separating whitespace
 *   character, or the end of the string, then the string is not a Tcl list.
 *
 *   NOTE: this differs from a brace-quoted word in the parsing of a
 *   Tcl command only in its treatment of the backslash-newline sequence.
 *   In a list element, the literal characters in the backslash-newline
 *   sequence become part of the element value.  In a script word,
 *   conversion to a single SPACE character is done.
 *
 *   NOTE: Most list element values can be represented by a formatted
 *   substring using brace quoting.  The exceptions are any element value
 *   that includes an unbalanced brace not in a backslash escape sequence,
 *   and any value that ends with a backslash not itself in a backslash
 *   escape sequence.
 * 
 * * If the first character of a formatted substring is
 *		\u0022	"	QUOTE
 *   then the end of the substring is the next QUOTE character, not counting
 *   any QUOTE characters that are contained within a backslash escape
 *   sequence.  If no next QUOTE is found before the end of the string, the
 *   string is not a Tcl list.  If the character following the closing QUOTE
 *   is not an element separating whitespace character, or the end of the
 *   string, then the string is not a Tcl list.  Having found the limits
 *   of the substring, the element value is produced by performing backslash
 *   substitution on the character sequence between the open and close QUOTEs.
 *
 *   NOTE: Any element value can be represented by this style of formatting,
 *   given suitable choice of backslash escape sequences.
 *
 * * All other formatted substrings are terminated by the next element
 *   separating whitespace character in the string.  Having found the limits
 *   of the substring, the element value is produced by performing backslash
 *   substitution on it.
 *
 *   NOTE:  Any element value can be represented by this style of formatting,
 *   given suitable choice of backslash escape sequences, with one exception.
 *   The empty string cannot be represented as a list element without the use
 *   of either braces or quotes to delimit it.
 *
 * This collection of parsing rules is implemented in the routine
 * TclFindElement().
 *
 * In order to produce lists that can be parsed by these rules, we need
 * the ability to distinguish between characters that are part of a list
 * element value from characters providing syntax that define the structure
 * of the list.  This means that our code that generates lists must at a
 * minimum be able to produce escape sequences for the 10 characters
 * identified above that have significance to a list parser.
 *
 *	*	*	CANONICAL LISTS	*	*	*	*	*	
 *
 * In addition to the basic rules for parsing strings into Tcl lists, there
 * are additional properties to be met by the set of list values that are
 * generated by Tcl.  Such list values are often said to be in "canonical
 * form":
 *
 * * When any canonical list is evaluated as a Tcl script, it is a script
 *   of either zero commands (an empty list) or exactly one command.  The
 *   command word is exactly the first element of the list, and each argument
 *   word is exactly one of the following elements of the list.  This means
 *   that any characters that have special meaning during script evaluation
 *   need special treatment when canonical lists are produced:
 *
 *	* Whitespace between elements may not include NEWLINE.
 *	* The command terminating character,
 *		\u003b	;	SEMICOLON
 *	  must be BRACEd, QUOTEd, or escaped so that it does not terminate
 * 	  the command prematurely.
 *	* Any of the characters that begin substitutions in scripts,
 *		\u0024	$	DOLLAR
 *		\u005b	[	OPEN BRACKET
 *		\u005c	\	BACKSLASH
 *	  need to be BRACEd or escaped.
 *	* In any list where the first character of the first element is
 *		\u0023	#	HASH
 *	  that HASH character must be BRACEd, QUOTEd, or escaped so that it
 *	  does not convert the command into a comment.
 *	* Any list element that contains the character sequence
 *	  BACKSLASH NEWLINE cannot be formatted with BRACEs.  The
 *	  BACKSLASH character must be represented by an escape
 *	  sequence, and unless QUOTEs are used, the NEWLINE must
 *	  be as well.
 *
 * * It is also guaranteed that one can use a canonical list as a building
 *   block of a larger script within command substitution, as in this example:
 *	set script "puts \[[list $cmd $arg]]"; eval $script
 *   To support this usage, any appearance of the character
 *		\u005d	]	CLOSE BRACKET
 *   in a list element must be BRACEd, QUOTEd, or escaped.
 *
 * * Finally it is guaranteed that enclosing a canonical list in braces
 *   produces a new value that is also a canonical list.  This new list has
 *   length 1, and its only element is the original canonical list.  This
 *   same guarantee also makes it possible to construct scripts where an
 *   argument word is given a list value by enclosing the canonical form
 *   of that list in braces:
 *	set script "puts {[list $one $two $three]}"; eval $script
 *   This sort of coding was once fairly common, though it's become more
 *   idiomatic to see the following instead:
 *	set script [list puts [list $one $two $three]]; eval $script
 *   In order to support this guarantee, every canonical list must have 
 *   balance when counting those braces that are not in escape sequences.
 *
 * Within these constraints, the canonical list generation routines
 * TclScanElement() and TclConvertElement() attempt to generate the string
 * for any list that is easiest to read.  When an element value is itself
 * acceptable as the formatted substring, it is usually used (CONVERT_NONE).
 * When some quoting or escaping is required, use of BRACEs (CONVERT_BRACE)
 * is usually preferred over the use of escape sequences (CONVERT_ESCAPE).
 * There are some exceptions to both of these preferences for reasons of
 * code simplicity, efficiency, and continuation of historical habits.
 * Canonical lists never use the QUOTE formatting to delimit their elements
 * because that form of quoting does not nest, which makes construction of
 * nested lists far too much trouble.  Canonical lists always use only a
 * single SPACE character for element-separating whitespace.
 *
 *	*	*	FUTURE CONSIDERATIONS	*	*	*
 *
 * When a list element requires quoting or escaping due to a CLOSE BRACKET
 * character or an internal QUOTE character, a strange formatting mode is
 * recommended.  For example, if the value "a{b]c}d" is converted by the
 * usual modes:
 *
 *	CONVERT_BRACE:	a{b]c}d		=> {a{b]c}d}
 *	CONVERT_ESCAPE:	a{b]c}d		=> a\{b\]c\}d
 *
 * we get perfectly usable formatted list elements.  However, this is not
 * what Tcl releases have been producing.  Instead, we have:
 *
 *	CONVERT_MASK:	a{b]c}d		=> a{b\]c}d
 *
 * where the CLOSE BRACKET is escaped, but the BRACEs are not.  The same
 * effect can be seen replacing ] with " in this example.  There does not
 * appear to be any functional or aesthetic purpose for this strange
 * additional mode.  The sole purpose I can see for preserving it is to
 * keep generating the same formatted lists programmers have become accustomed
 * to, and perhaps written tests to expect.  That is, compatibility only.
 * The additional code complexity required to support this mode is significant.
 * The lines of code supporting it are delimited in the routines below with
 * #if COMPAT directives.  This makes it easy to experiment with eliminating
 * this formatting mode simply with "#define COMPAT 0" above.  I believe
 * this is worth considering.
 * 
 * Another consideration is the treatment of QUOTE characters in list elements.
 * TclConvertElement() must have the ability to produce the escape sequence
 * \" so that when a list element begins with a QUOTE we do not confuse
 * that first character with a QUOTE used as list syntax to define list
 * structure.  However, that is the only place where QUOTE characters need
 * quoting.  In this way, handling QUOTE could really be much more like
 * the way we handle HASH which also needs quoting and escaping only in
 * particular situations.  Following up this could increase the set of
 * list elements that can use the CONVERT_NONE formatting mode.
 *
 * More speculative is that the demands of canonical list form require brace
 * balance for the list as a whole, while the current implementation achieves
 * this by establishing brace balance for every element.
 *
 * Finally, a reminder that the rules for parsing and formatting lists are
 * closely tied together with the rules for parsing and evaluating scripts,
 * and will need to evolve in sync.
 */
 
/*
 *----------------------------------------------------------------------
 *
 * TclMaxListLength --
 *
 *	Given 'bytes' pointing to 'numBytes' bytes, scan through them and
 *	count the number of whitespace runs that could be list element
................................................................................
 *
 *	If TCL_OK is returned, then *elementPtr will be set to point to the
 *	first element of list, and *nextPtr will be set to point to the
 *	character just after any white space following the last character
 *	that's part of the element. If this is the last argument in the list,
 *	then *nextPtr will point just after the last character in the list
 *	(i.e., at the character at list+listLength). If sizePtr is non-NULL,
 *	*sizePtr is filled in with the number of bytes in the element. If
 *	the element is in braces, then *elementPtr will point to the character
 *	after the opening brace and *sizePtr will not include either of the
 *	braces. If there isn't an element in the list, *sizePtr will be zero,
 *	and both *elementPtr and *nextPtr will point just after the last
 *	character in the list. If literalPtr is non-NULL, *literalPtr is set
 *	to a boolean value indicating whether the substring returned as
 *	the values of **elementPtr and *sizePtr is the literal value of
................................................................................
 * Tcl_ScanElement --
 *
 *	This function is a companion function to Tcl_ConvertElement. It scans
 *	a string to see what needs to be done to it (e.g. add backslashes or
 *	enclosing braces) to make the string into a valid Tcl list element.
 *
 * Results:
 *	The return value is an overestimate of the number of bytes that
 *	will be needed by Tcl_ConvertElement to produce a valid list element
 *	from src. The word at *flagPtr is filled in with a value needed by
 *	Tcl_ConvertElement when doing the actual conversion.
 *
 * Side effects:
 *	None.
 *
 *----------------------------------------------------------------------
 */

int
Tcl_ScanElement(
    register CONST char *src,	/* String to convert to list element. */
    register int *flagPtr)	/* Where to store information to guide
				 * Tcl_ConvertCountedElement. */
{
    return Tcl_ScanCountedElement(src, -1, flagPtr);
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_ScanCountedElement --
 *
 *	This function is a companion function to Tcl_ConvertCountedElement. It
 *	scans a string to see what needs to be done to it (e.g. add
 *	backslashes or enclosing braces) to make the string into a valid Tcl
 *	list element. If length is -1, then the string is scanned from src up
 *	to the first null byte.
 *
 * Results:
 *	The return value is an overestimate of the number of bytes that
 *	will be needed by Tcl_ConvertCountedElement to produce a valid list
 *	element from src. The word at *flagPtr is filled in with a value
 *	needed by Tcl_ConvertCountedElement when doing the actual conversion.
 *
 * Side effects:
 *	None.
 *
 *----------------------------------------------------------------------
 */

int
Tcl_ScanCountedElement(
    CONST char *src,		/* String to convert to Tcl list element. */
    int length,			/* Number of bytes in src, or -1. */
    int *flagPtr)		/* Where to store information to guide
				 * Tcl_ConvertElement. */
{
    int flags = CONVERT_ANY;
    int numBytes = TclScanElement(src, length, &flags);

    *flagPtr = flags;
    return numBytes;
}
 
/*

 *----------------------------------------------------------------------
 *



 * TclScanElement --
 *


 *	This function is a companion function to TclConvertElement. It
 *	scans a string to see what needs to be done to it (e.g. add
 *	backslashes or enclosing braces) to make the string into a valid Tcl
 *	list element. If length is -1, then the string is scanned from src up
 *	to the first null byte.  A NULL value for src is treated as an
 *	empty string.  The incoming value of *flagPtr is a report from the
 *	caller what additional flags it will pass to TclConvertElement().
 *


 * Results:
 *	The recommended formatting mode for the element is determined and
 *	a value is written to *flagPtr indicating that recommendation.  This
 *	recommendation is combined with the incoming flag values in *flagPtr
 *	set by the caller to determine how many bytes will be needed by
 *	TclConvertElement() in which to write the formatted element following
 *	the recommendation modified by the flag values.  This number of bytes
 *	is the return value of the routine.  In some situations it may be
 *	an overestimate, but so long as the caller passes the same flags
 *	to TclConvertElement(), it will be large enough.
 *





 * Side effects:
 *	None.
 *






 *----------------------------------------------------------------------
 */





int
TclScanElement(
    CONST char *src,		/* String to convert to Tcl list element. */
    int length,			/* Number of bytes in src, or -1. */
    int *flagPtr)		/* Where to store information to guide
				 * Tcl_ConvertElement. */
{
    CONST char *p = src;
    int nestingLevel = 0;	/* Brace nesting count */

    int forbidNone = 0;		/* Do not permit CONVERT_NONE mode. Something
				   needs protection or escape. */
    int requireEscape = 0;	/* Force use of CONVERT_ESCAPE mode.  For some
				 * reason bare or brace-quoted form fails. */
    int extra = 0;		/* Count of number of extra bytes needed for
				 * formatted element, assuming we use escape
				 * sequences in formatting. */
    int bytesNeeded;		/* Buffer length computed to complete the
				 * element formatting in the selected mode. */
#if COMPAT
    int preferEscape = 0;	/* Use preferences to track whether to use */
    int preferBrace = 0;	/* CONVERT_MASK mode. */
    int braceCount = 0;		/* Count of all braces '{' '}' seen. */
#endif
    


    if ((p == NULL) || (length == 0) || ((*p == '\0') && (length == -1))) {
	/* Empty string element must be brace quoted. */
	*flagPtr = CONVERT_BRACE;
	return 2;
    }



    if ((*p == '{') || (*p == '"')) {

	/*
	 * Must escape or protect so leading character of value is not
	 * misinterpreted as list element delimiting syntax.
	 */
	forbidNone = 1;
#if COMPAT
	preferBrace = 1;
#endif
    }


    while (length) {
	switch (*p) {
	case '{':
#if COMPAT
	    braceCount++;
#endif
	    extra++;				/* Escape '{' => '\{' */
	    nestingLevel++;
	    break;
	case '}':
#if COMPAT
	    braceCount++;
#endif
	    extra++;				/* Escape '}' => '\}' */
	    nestingLevel--;
	    if (nestingLevel < 0) {

		/* Unbalanced braces!  Cannot format with brace quoting. */
		requireEscape = 1;
	    }
	    break;
	case ']':
	case '"':
#if COMPAT
	    forbidNone = 1;
	    extra++;		/* Escapes all just prepend a backslash */
	    preferEscape = 1;
	    break;
#else
	    /* FLOW THROUGH */
#endif
	case '[':
	case '$':
	case ';':
	case ' ':
	case '\f':
	case '\n':
	case '\r':
	case '\t':
	case '\v':
	    forbidNone = 1;
	    extra++;		/* Escape sequences all one byte longer. */
#if COMPAT
	    preferBrace = 1;
#endif
	    break;
	case '\\':
	    extra++;				/* Escape '\' => '\\' */
	    if ((length == 1) || ((length == -1) && (p[1] == '\0'))) {
		/* Final backslash. Cannot format with brace quoting. */
		requireEscape = 1;		
		break;
	    }
	    if (p[1] == '\n') {



		extra++;	/* Escape newline => '\n', one byte longer */
		/* Backslash newline sequence.  Brace quoting not permitted. */
		requireEscape = 1;
		length -= (length > 0);
		p++;
		break;
	    }



	    if ((p[1] == '{') || (p[1] == '}') || (p[1] == '\\')) {
		extra++;	/* Escape sequences all one byte longer. */
		length -= (length > 0);
		p++;
	    }
	    forbidNone = 1;
#if COMPAT
	    preferBrace = 1;
#endif
	    break;
	case '\0':
	    if (length == -1) {
		goto endOfString;
	    }
	    /* TODO: Panic on improper encoding? */
	    break;
	}
	length -= (length > 0);
	p++;
    }

    endOfString:
    if (nestingLevel != 0) {
	/* Unbalanced braces!  Cannot format with brace quoting. */
	requireEscape = 1;
    }

    /* We need at least as many bytes as are in the element value... */
    bytesNeeded = p - src;

    if (requireEscape) {
	/*
	 * We must use escape sequences.  Add all the extra bytes needed
	 * to have room to create them.
	 */
	bytesNeeded += extra;
	/* Make room to escape leading #, if needed. */
	if ((*src == '#') && !(*flagPtr & TCL_DONT_QUOTE_HASH)) {
	    bytesNeeded++;
	}
	*flagPtr = CONVERT_ESCAPE;
	goto overflowCheck;
    }
    if (*flagPtr & CONVERT_ANY) {
	/*
	 * The caller has not let us know what flags it will pass to
	 * TclConvertElement() so compute the max size we might need for
	 * any possible choice.  Normally the formatting using escape
	 * sequences is the longer one, and a minimum "extra" value of 2
	 * makes sure we don't request too small a buffer in those edge
	 * cases where that's not true.
	 */
	if (extra < 2) {
	    extra = 2;
	}
	*flagPtr &= ~CONVERT_ANY;
	*flagPtr |= TCL_DONT_USE_BRACES;
    }
    if (forbidNone) {
	/* We must request some form of quoting of escaping... */
#if COMPAT
	if (preferEscape && !preferBrace) {
	    /*
	     * If we are quoting solely due to ] or internal " characters
	     * use the CONVERT_MASK mode where we escape all special 
	     * characters except for braces.  "extra" counted space needed
	     * to escape braces too, so substract "braceCount" to get our
	     * actual needs.
	     */
	    bytesNeeded += (extra - braceCount);
	    /* Make room to escape leading #, if needed. */
	    if ((*src == '#') && !(*flagPtr & TCL_DONT_QUOTE_HASH)) {
		bytesNeeded++;
	    }
	    /*
	     * If the caller reports it will direct TclConvertElement() to
	     * use full escapes on the element, add back the bytes needed to
	     * escape the braces.
	     */
	    if (*flagPtr & TCL_DONT_USE_BRACES) {
		bytesNeeded += braceCount;
	    }
	    *flagPtr = CONVERT_MASK;
	    goto overflowCheck;
	}
#endif
	if (*flagPtr & TCL_DONT_USE_BRACES) {
	    /*


	     * If the caller reports it will direct TclConvertElement() to
	     * use escapes, add the extra bytes needed to have room for them.
	     */
	    bytesNeeded += extra;
	    /* Make room to escape leading #, if needed. */
	    if ((*src == '#') && !(*flagPtr & TCL_DONT_QUOTE_HASH)) {
		bytesNeeded++;
	    }

	} else {
	    /* Add 2 bytes for room for the enclosing braces. */
	    bytesNeeded += 2;
	}
	*flagPtr = CONVERT_BRACE;
	goto overflowCheck;
    }

    /* So far, no need to quote or escape anything. */
    if ((*src == '#') && !(*flagPtr & TCL_DONT_QUOTE_HASH)) {
	/* If we need to quote a leading #, make room to enclose in braces. */
	bytesNeeded += 2;
    }
    *flagPtr = CONVERT_NONE;

    overflowCheck:
    if (bytesNeeded < 0) {
	Tcl_Panic("TclScanElement: string length overflow");
    }
    return bytesNeeded;
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_ConvertElement --
 *
................................................................................
int
Tcl_ConvertCountedElement(
    register CONST char *src,	/* Source information for list element. */
    int length,			/* Number of bytes in src, or -1. */
    char *dst,			/* Place to put list-ified element. */
    int flags)			/* Flags produced by Tcl_ScanElement. */
{
    int numBytes = TclConvertElement(src, length, dst, flags);
    dst[numBytes] = '\0';
    return numBytes;
}
 
/*


 *----------------------------------------------------------------------
 *
 * TclConvertElement --
 *
 *	This is a companion function to TclScanElement. Given the
 *	information produced by TclScanElement, this function converts
 *	a string to a list element equal to that string.
 *
 * Results:
 *	Information is copied to *dst in the form of a list element identical
 *	to src (i.e. if Tcl_SplitList is applied to dst it will produce a
 *	string identical to src). The return value is a count of the number of
 *	characters copied (not including the terminating NULL character).
 *
 * Side effects:
 *	None.
 *
 *----------------------------------------------------------------------
 */



int TclConvertElement(
    register CONST char *src,	/* Source information for list element. */
    int length,			/* Number of bytes in src, or -1. */
    char *dst,			/* Place to put list-ified element. */
    int flags)			/* Flags produced by Tcl_ScanElement. */
{
    int conversion = flags & CONVERT_MASK;
    char *p = dst;

    /* Let the caller demand we use escape sequences rather than braces. */
    if ((flags & TCL_DONT_USE_BRACES) && (conversion & CONVERT_BRACE)) {
	conversion = CONVERT_ESCAPE;
    }

    /* No matter what the caller demands, empty string must be braced! */
    if ((src == NULL) || (length == 0) || ((*src == '\0') && (length == -1))) {
	src = tclEmptyStringRep;
	length = 0;
	conversion = CONVERT_BRACE;
    }

    /* Escape leading hash as needed and requested. */
    if ((*src == '#') && !(flags & TCL_DONT_QUOTE_HASH)) {
	if (conversion == CONVERT_ESCAPE) {
	    p[0] = '\\';
	    p[1] = '#';

	    p += 2;
	    src++;
	    length--;
	} else {
	    conversion = CONVERT_BRACE;
	}
    }

    /* No escape or quoting needed.  Copy the literal string value. */
    if (conversion == CONVERT_NONE) {
	if (length == -1) {
	    /* TODO: INT_MAX overflow? */
	    while (*src) {
		*p++ = *src++;
	    }
	    return p - dst;
	} else {
	    memcpy(dst, src, length);
	    return length;


	}

    }

    /* Formatted string is original string enclosed in braces. */
    if (conversion == CONVERT_BRACE) {
	*p = '{';
	p++;

	if (length == -1) {
	    /* TODO: INT_MAX overflow? */
	    while (*src) {
		*p++ = *src++;
	    }


	} else {







	    memcpy(p, src, length);
	    p += length;
	}
	*p = '}';


	p++;






	return p - dst;
    }






    /* conversion == CONVERT_ESCAPE or CONVERT_MASK */

    /* Formatted string is original string converted to escape sequences. */
    for ( ; length; src++, length -= (length > 0)) {
	switch (*src) {
	case ']':
	case '[':
	case '$':
	case ';':
	case ' ':
	case '\\':
	case '"':
	    *p = '\\';
	    p++;
	    break;
	case '{':
	case '}':










#if COMPAT
	    if (conversion == CONVERT_ESCAPE) {
#endif
		*p = '\\';
		p++;
#if COMPAT
	    }
#endif
	    break;
	case '\f':
	    *p = '\\';
	    p++;
	    *p = 'f';
	    p++;
	    continue;
	case '\n':
	    *p = '\\';
	    p++;
	    *p = 'n';
	    p++;
	    continue;
	case '\r':
	    *p = '\\';
	    p++;
	    *p = 'r';
	    p++;
	    continue;
	case '\t':
	    *p = '\\';
	    p++;
	    *p = 't';
	    p++;
	    continue;
	case '\v':
	    *p = '\\';
	    p++;
	    *p = 'v';
	    p++;
	    continue;
	case '\0':
	    if (length == -1) {
		return p - dst;
	    }
	    /* 
	     * If we reach this point, there's an embedded NULL in the
	     * string range being processed, which should not happen when
	     * the encoding rules for Tcl strings are properly followed.
	     * If the day ever comes when we stop tolerating such things,
	     * this is where to put the Tcl_Panic().
	     */
	    break;
	}
	*p = *src;
	p++;
    }


    return p - dst;
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_Merge --
 *
................................................................................

char *
Tcl_Merge(
    int argc,			/* How many strings to merge. */
    CONST char * CONST *argv)	/* Array of string values. */
{
#   define LOCAL_SIZE 20
    int localFlags[LOCAL_SIZE], *flagPtr = NULL;
    int i, bytesNeeded = 0;
    char *result, *dst;
    const int maxFlags = UINT_MAX / sizeof(int);


    if (argc == 0) {
	/*
	 * Handle empty list case first, so logic of the general case
	 * can be simpler.
	 */
	result = ckalloc(1);
	result[0] = '\0';
	return result;
    }

    /*
     * Pass 1: estimate space, gather flags.
     */

    if (argc <= LOCAL_SIZE) {
	flagPtr = localFlags;
    } else if (argc > maxFlags) {
	/*
	 * We cannot allocate a large enough flag array to format this
	 * list in one pass.  We could imagine converting this routine
	 * to a multi-pass implementation, but for sizeof(int) == 4, 
	 * the limit is a max of 2^30 list elements and since each element
	 * is at least one byte formatted, and requires one byte space
	 * between it and the next one, that a minimum space requirement
	 * of 2^31 bytes, which is already INT_MAX. If we tried to format
	 * a list of > maxFlags elements, we're just going to overflow
	 * the size limits on the formatted string anyway, so just issue
	 * that same panic early.
	 */
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    } else {
	flagPtr = (int *) ckalloc((unsigned) argc*sizeof(int));
    }

    for (i = 0; i < argc; i++) {
	flagPtr[i] = ( i ? TCL_DONT_QUOTE_HASH : 0 );
	bytesNeeded += TclScanElement(argv[i], -1, &flagPtr[i]);
	if (bytesNeeded < 0) {
	    Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
	}
    }
    if (bytesNeeded > INT_MAX - argc + 1) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    }
    bytesNeeded += argc;

    /*
     * Pass two: copy into the result area.
     */

    result = ckalloc((unsigned) bytesNeeded);
    dst = result;
    for (i = 0; i < argc; i++) {
	flagPtr[i] |= ( i ? TCL_DONT_QUOTE_HASH : 0 );
	dst += TclConvertElement(argv[i], -1, dst, flagPtr[i]);


	*dst = ' ';
	dst++;
    }



    dst[-1] = 0;


    if (flagPtr != localFlags) {
	ckfree((char *) flagPtr);
    }
    return result;
}
 
................................................................................

char *
Tcl_DStringAppendElement(
    Tcl_DString *dsPtr,		/* Structure describing dynamic string. */
    CONST char *element)	/* String to append. Must be
				 * null-terminated. */
{
    char *dst = dsPtr->string + dsPtr->length;
    int needSpace = TclNeedSpace(dsPtr->string, dst);
    int flags = needSpace ? TCL_DONT_QUOTE_HASH : 0;
    int newSize = dsPtr->length + needSpace
	    + TclScanElement(element, -1, &flags);


    /*
     * Allocate a larger buffer for the string if the current one isn't large
     * enough. Allocate extra space in the new buffer so that there will be
     * room to grow before we have to allocate again. SPECIAL NOTE: must use
     * memcpy, not strcpy, to copy the string to a larger buffer, since there
     * may be embedded NULLs in the string in some cases.
................................................................................

	    memcpy(newString, dsPtr->string, (size_t) dsPtr->length);
	    dsPtr->string = newString;
	} else {
	    dsPtr->string = (char *) ckrealloc((void *) dsPtr->string,
		    (size_t) dsPtr->spaceAvl);
	}
	dst = dsPtr->string + dsPtr->length;
    }

    /*
     * Convert the new string to a list element and copy it into the buffer at
     * the end, with a space, if needed.
     */

    if (needSpace) {

	*dst = ' ';
	dst++;
	dsPtr->length++;

	/*
	 * If we need a space to separate this element from preceding stuff,
	 * then this element will not lead a list, and need not have it's
	 * leading '#' quoted.
	 */

	flags |= TCL_DONT_QUOTE_HASH;
    }
    dsPtr->length += TclConvertElement(element, -1, dst, flags);
    dsPtr->string[dsPtr->length] = '\0';
    return dsPtr->string;
}
 
/*
 *----------------------------------------------------------------------
 *
 * Tcl_DStringSetLength --

Changes to tests/list.test.

100
101
102
103
104
105
106




107
108
109
110
	set last [expr $last-1]
    }
    return [concat $result $list]
}
test list-3.1 {SetListFromAny and lrange/concat results} {
    slowsort {fred julie alex carol bill annie}
} {alex annie bill carol fred julie}





# cleanup
::tcltest::cleanupTests
return






>
>
>
>




100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
	set last [expr $last-1]
    }
    return [concat $result $list]
}
test list-3.1 {SetListFromAny and lrange/concat results} {
    slowsort {fred julie alex carol bill annie}
} {alex annie bill carol fred julie}

test list-4.1 {Bug 3173086} {
    string is list "{[list \\\\\}]}"
} 1

# cleanup
::tcltest::cleanupTests
return