Tcl Source Code

Changes On Branch tip-587
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch tip-587 Excluding Merge-Ins

This is equivalent to a diff from ecc1efae3a to 8b5badfb92

2020-10-22
10:38
TIP #587: Default utf-8 for source command check-in: ad4d3b63ba user: jan.nijtmans tags: trunk
2020-09-26
18:42
Merge 8.7 check-in: f205897f50 user: jan.nijtmans tags: trunk
2020-09-25
14:36
Merge trunk Closed-Leaf check-in: 8b5badfb92 user: jan.nijtmans tags: tip-587
14:25
Merge 8.7 check-in: ecc1efae3a user: jan.nijtmans tags: trunk
14:25
Merge 8.6 check-in: a34d4f2927 user: jan.nijtmans tags: core-8-branch
12:26
Merge trunk check-in: 7e222f7c97 user: jan.nijtmans tags: tip-587
12:10
Merge 8.7 check-in: cfcf76d616 user: jan.nijtmans tags: trunk

Changes to doc/FileSystem.3.

410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
accumulates the return values in a list which is returned to the
caller (with a reference count of 0).
.PP
\fBTcl_FSEvalFileEx\fR reads the file given by \fIpathPtr\fR using
the encoding identified by \fIencodingName\fR and evaluates
its contents as a Tcl script. It returns the same information as
\fBTcl_EvalObjEx\fR.
If \fIencodingName\fR is NULL, the system encoding is used for
reading the file contents.
If the file could not be read then a Tcl error is returned to describe
why the file could not be read.
The eofchar for files is
.QW \e32
(^Z) for all platforms.
If you require a
.QW ^Z
in code for string comparison, you can use
.QW \e032
or
.QW \eu001a ,
which will be safely substituted by the Tcl interpreter into
.QW ^Z .
\fBTcl_FSEvalFile\fR is a simpler version of
\fBTcl_FSEvalFileEx\fR that always uses the system encoding
when reading the file.
.PP
\fBTcl_FSLoadFile\fR dynamically loads a binary code file into memory and
returns the addresses of two procedures within that file, if they are
defined. The appropriate function for the filesystem to which \fIpathPtr\fR
belongs will be called. If that filesystem does not implement this
function (most virtual filesystems will not, because of OS limitations







|















|







410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
accumulates the return values in a list which is returned to the
caller (with a reference count of 0).
.PP
\fBTcl_FSEvalFileEx\fR reads the file given by \fIpathPtr\fR using
the encoding identified by \fIencodingName\fR and evaluates
its contents as a Tcl script. It returns the same information as
\fBTcl_EvalObjEx\fR.
If \fIencodingName\fR is NULL, the utf-8 encoding is used for
reading the file contents.
If the file could not be read then a Tcl error is returned to describe
why the file could not be read.
The eofchar for files is
.QW \e32
(^Z) for all platforms.
If you require a
.QW ^Z
in code for string comparison, you can use
.QW \e032
or
.QW \eu001a ,
which will be safely substituted by the Tcl interpreter into
.QW ^Z .
\fBTcl_FSEvalFile\fR is a simpler version of
\fBTcl_FSEvalFileEx\fR that always uses the utf-8 encoding
when reading the file.
.PP
\fBTcl_FSLoadFile\fR dynamically loads a binary code file into memory and
returns the addresses of two procedures within that file, if they are
defined. The appropriate function for the filesystem to which \fIpathPtr\fR
belongs will be called. If that filesystem does not implement this
function (most virtual filesystems will not, because of OS limitations

Changes to doc/InitSubSyst.3.

19
20
21
22
23
24
25
26
27
28
29
30
31
.PP
The \fBTcl_InitSubsystems\fR procedure initializes the Tcl
library. This procedure is typically invoked as the very
first thing in the application's main program.
.PP
\fBTcl_InitSubsystems\fR is very similar in use to
\fBTcl_FindExecutable\fR. It can be used when Tcl is
used as utility library, no other encodings than utf8,
iso8859-1 or unicode are used, and no interest exists in the
value of \fBinfo nameofexecutable\fR. The system encoding will not
be extracted from the environment, but falls back to iso8859-1.
.SH KEYWORDS
binary, executable file







|


|


19
20
21
22
23
24
25
26
27
28
29
30
31
.PP
The \fBTcl_InitSubsystems\fR procedure initializes the Tcl
library. This procedure is typically invoked as the very
first thing in the application's main program.
.PP
\fBTcl_InitSubsystems\fR is very similar in use to
\fBTcl_FindExecutable\fR. It can be used when Tcl is
used as utility library, no other encodings than utf-8,
iso8859-1 or unicode are used, and no interest exists in the
value of \fBinfo nameofexecutable\fR. The system encoding will not
be extracted from the environment, but falls back to utf-8.
.SH KEYWORDS
binary, executable file

Changes to doc/source.n.

43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
which will be safely substituted by the Tcl interpreter into
.QW ^Z .
.PP
A leading BOM (Byte order mark) contained in the file is ignored for unicode encodings (utf-8, utf-16, ucs-2).
.PP
The \fB\-encoding\fR option is used to specify the encoding of
the data stored in \fIfileName\fR.  When the \fB\-encoding\fR option
is omitted, the system encoding is assumed.
.SH EXAMPLE
.PP
Run the script in the file \fBfoo.tcl\fR and then the script in the
file \fBbar.tcl\fR:
.PP
.CS
\fBsource\fR foo.tcl







|







43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
which will be safely substituted by the Tcl interpreter into
.QW ^Z .
.PP
A leading BOM (Byte order mark) contained in the file is ignored for unicode encodings (utf-8, utf-16, ucs-2).
.PP
The \fB\-encoding\fR option is used to specify the encoding of
the data stored in \fIfileName\fR.  When the \fB\-encoding\fR option
is omitted, the utf-8 encoding is assumed.
.SH EXAMPLE
.PP
Run the script in the file \fBfoo.tcl\fR and then the script in the
file \fBbar.tcl\fR:
.PP
.CS
\fBsource\fR foo.tcl

Changes to generic/tclIOUtil.c.

1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
int
Tcl_FSEvalFileEx(
    Tcl_Interp *interp,		/* Interpreter that evaluates the script. */
    Tcl_Obj *pathPtr,		/* Pathname of the file to process.
				 * Tilde-substitution is performed on this
				 * pathname. */
    const char *encodingName)	/* Either the name of an encoding or NULL to
				   use the system encoding. */
{
    size_t length;
	int result = TCL_ERROR;
    Tcl_StatBuf statBuf;
    Tcl_Obj *oldScriptFile;
    Interp *iPtr;
    const char *string;







|







1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
int
Tcl_FSEvalFileEx(
    Tcl_Interp *interp,		/* Interpreter that evaluates the script. */
    Tcl_Obj *pathPtr,		/* Pathname of the file to process.
				 * Tilde-substitution is performed on this
				 * pathname. */
    const char *encodingName)	/* Either the name of an encoding or NULL to
				   use the utf-8 encoding. */
{
    size_t length;
	int result = TCL_ERROR;
    Tcl_StatBuf statBuf;
    Tcl_Obj *oldScriptFile;
    Interp *iPtr;
    const char *string;
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730


1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
     * uses it on every platform to allow for scripted documents. [Bug: 2040]
     */

    Tcl_SetChannelOption(interp, chan, "-eofchar", "\32 {}");

    /*
     * If the encoding is specified, set the channel to that encoding.
     * Otherwise don't touch it, leaving things up to the system encoding.  If
     * the encoding is unknown report an error.
     */

    if (encodingName != NULL) {


	if (Tcl_SetChannelOption(interp, chan, "-encoding", encodingName)
		!= TCL_OK) {
	    Tcl_CloseEx(interp,chan,0);
	    return result;
	}
    }

    TclNewObj(objPtr);
    Tcl_IncrRefCount(objPtr);

    /*
     * Read first character of stream to check for utf-8 BOM







<
|


|
>
>
|
|
|
|
<







1719
1720
1721
1722
1723
1724
1725

1726
1727
1728
1729
1730
1731
1732
1733
1734
1735

1736
1737
1738
1739
1740
1741
1742
     * uses it on every platform to allow for scripted documents. [Bug: 2040]
     */

    Tcl_SetChannelOption(interp, chan, "-eofchar", "\32 {}");

    /*
     * If the encoding is specified, set the channel to that encoding.

     * Otherwise use utf-8.  If the encoding is unknown report an error.
     */

    if (encodingName == NULL) {
	encodingName = "utf-8";
    }
    if (Tcl_SetChannelOption(interp, chan, "-encoding", encodingName)
	    != TCL_OK) {
	Tcl_CloseEx(interp,chan,0);
	return result;

    }

    TclNewObj(objPtr);
    Tcl_IncrRefCount(objPtr);

    /*
     * Read first character of stream to check for utf-8 BOM
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
int
TclNREvalFile(
    Tcl_Interp *interp,		/* Interpreter in which to evaluate the script. */
    Tcl_Obj *pathPtr,		/* Pathname of a file containing the script to
				 * evaluate. Tilde-substitution is performed on
				 * this pathname. */
    const char *encodingName)	/* The name of an encoding to use, or NULL to
				 *  use the system encoding. */
{
    Tcl_StatBuf statBuf;
    Tcl_Obj *oldScriptFile, *objPtr;
    Interp *iPtr;
    Tcl_Channel chan;
    const char *string;








|







1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
int
TclNREvalFile(
    Tcl_Interp *interp,		/* Interpreter in which to evaluate the script. */
    Tcl_Obj *pathPtr,		/* Pathname of a file containing the script to
				 * evaluate. Tilde-substitution is performed on
				 * this pathname. */
    const char *encodingName)	/* The name of an encoding to use, or NULL to
				 *  use the utf-8 encoding. */
{
    Tcl_StatBuf statBuf;
    Tcl_Obj *oldScriptFile, *objPtr;
    Interp *iPtr;
    Tcl_Channel chan;
    const char *string;

1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866


1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
     * uses it on every platform to allow for scripted documents. [Bug: 2040]
     */

    Tcl_SetChannelOption(interp, chan, "-eofchar", "\32 {}");

    /*
     * If the encoding is specified, set the channel to that encoding.
     * Otherwise don't touch it, leaving things up to the system encoding.  If
     * the encoding is unknown report an error.
     */

    if (encodingName != NULL) {


	if (Tcl_SetChannelOption(interp, chan, "-encoding", encodingName)
		!= TCL_OK) {
	    Tcl_CloseEx(interp, chan, 0);
	    return TCL_ERROR;
	}
    }

    TclNewObj(objPtr);
    Tcl_IncrRefCount(objPtr);

    /*
     * Read first character of stream to check for utf-8 BOM







<
|


|
>
>
|
|
|
|
<







1855
1856
1857
1858
1859
1860
1861

1862
1863
1864
1865
1866
1867
1868
1869
1870
1871

1872
1873
1874
1875
1876
1877
1878
     * uses it on every platform to allow for scripted documents. [Bug: 2040]
     */

    Tcl_SetChannelOption(interp, chan, "-eofchar", "\32 {}");

    /*
     * If the encoding is specified, set the channel to that encoding.

     * Otherwise use utf-8.  If the encoding is unknown report an error.
     */

    if (encodingName == NULL) {
	encodingName = "utf-8";
    }
    if (Tcl_SetChannelOption(interp, chan, "-encoding", encodingName)
	    != TCL_OK) {
	Tcl_CloseEx(interp, chan, 0);
	return TCL_ERROR;

    }

    TclNewObj(objPtr);
    Tcl_IncrRefCount(objPtr);

    /*
     * Read first character of stream to check for utf-8 BOM

Changes to library/auto.tcl.

261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
	    cd $oldDir
	    return -options $opts $msg
	}
    }
    auto_mkindex_parser::cleanup

    set fid [open "tclIndex" w]
    fconfigure $fid -translation lf
    puts -nonewline $fid $index
    close $fid
    cd $oldDir
}

# Original version of auto_mkindex that just searches the source code for
# "proc" at the beginning of the line.







|







261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
	    cd $oldDir
	    return -options $opts $msg
	}
    }
    auto_mkindex_parser::cleanup

    set fid [open "tclIndex" w]
    fconfigure $fid -encoding utf-8 -translation lf
    puts -nonewline $fid $index
    close $fid
    cd $oldDir
}

# Original version of auto_mkindex that just searches the source code for
# "proc" at the beginning of the line.
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
    if {![llength $args]} {
	set args *.tcl
    }
    foreach file [lsort [glob -- {*}$args]] {
	set f ""
	set error [catch {
	    set f [open $file]
	    fconfigure $f -eofchar \032
	    while {[gets $f line] >= 0} {
		if {[regexp {^proc[ 	]+([^ 	]*)} $line match procName]} {
		    set procName [lindex [auto_qualify $procName "::"] 0]
		    append index "set [list auto_index($procName)]"
		    append index " \[list source \[file join \$dir [list $file]\]\]\n"
		}
	    }
	    close $f
	} msg opts]
	if {$error} {
	    catch {close $f}
	    cd $oldDir
	    return -options $opts $msg
	}
    }
    set f ""
    set error [catch {
	set f [open tclIndex w]
	fconfigure $f -translation lf
	puts -nonewline $f $index
	close $f
	cd $oldDir
    } msg opts]
    if {$error} {
	catch {close $f}
	cd $oldDir







|


















|







288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
    if {![llength $args]} {
	set args *.tcl
    }
    foreach file [lsort [glob -- {*}$args]] {
	set f ""
	set error [catch {
	    set f [open $file]
	    fconfigure $f -encoding utf-8 -eofchar \032
	    while {[gets $f line] >= 0} {
		if {[regexp {^proc[ 	]+([^ 	]*)} $line match procName]} {
		    set procName [lindex [auto_qualify $procName "::"] 0]
		    append index "set [list auto_index($procName)]"
		    append index " \[list source \[file join \$dir [list $file]\]\]\n"
		}
	    }
	    close $f
	} msg opts]
	if {$error} {
	    catch {close $f}
	    cd $oldDir
	    return -options $opts $msg
	}
    }
    set f ""
    set error [catch {
	set f [open tclIndex w]
	fconfigure $f -encoding utf-8 -translation lf
	puts -nonewline $f $index
	close $f
	cd $oldDir
    } msg opts]
    if {$error} {
	catch {close $f}
	cd $oldDir
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
    variable scriptFile
    variable contextStack
    variable imports

    set scriptFile $file

    set fid [open $file]
    fconfigure $fid -eofchar \032
    set contents [read $fid]
    close $fid

    # There is one problem with sourcing files into the safe interpreter:
    # references like "$x" will fail since code is not really being executed
    # and variables do not really exist.  To avoid this, we replace all $ with
    # \0 (literally, the null char) later, when getting proc names we will







|







400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
    variable scriptFile
    variable contextStack
    variable imports

    set scriptFile $file

    set fid [open $file]
    fconfigure $fid -encoding utf-8 -eofchar \032
    set contents [read $fid]
    close $fid

    # There is one problem with sourcing files into the safe interpreter:
    # references like "$x" will fail since code is not really being executed
    # and variables do not really exist.  To avoid this, we replace all $ with
    # \0 (literally, the null char) later, when getting proc names we will

Changes to library/clock.tcl.

3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324

    if { ![regexp {^[[.-.][:alpha:]_]+(?:/[[.-.][:alpha:]_]+)*$} $fileName] } {
	return -code error \
	    -errorcode [list CLOCK badTimeZone $:fileName] \
	    "time zone \":$fileName\" not valid"
    }
    try {
	source -encoding utf-8 [file join $DataDir $fileName]
    } on error {} {
	return -code error \
	    -errorcode [list CLOCK badTimeZone :$fileName] \
	    "time zone \":$fileName\" not found"
    }
    return
}







|







3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324

    if { ![regexp {^[[.-.][:alpha:]_]+(?:/[[.-.][:alpha:]_]+)*$} $fileName] } {
	return -code error \
	    -errorcode [list CLOCK badTimeZone $:fileName] \
	    "time zone \":$fileName\" not valid"
    }
    try {
	source [file join $DataDir $fileName]
    } on error {} {
	return -code error \
	    -errorcode [list CLOCK badTimeZone :$fileName] \
	    "time zone \":$fileName\" not found"
    }
    return
}

Changes to library/init.tcl.

108
109
110
111
112
113
114
115
116
117
118
119
120
121
122

    proc ::tcl::initClock {} {
	# Auto-loading stubs for 'clock.tcl'

	foreach cmd {add format scan} {
	    proc ::tcl::clock::$cmd args {
		variable TclLibDir
		source -encoding utf-8 [file join $TclLibDir clock.tcl]
		return [uplevel 1 [info level 0]]
	    }
	}

	rename ::tcl::initClock {}
    }
    ::tcl::initClock







|







108
109
110
111
112
113
114
115
116
117
118
119
120
121
122

    proc ::tcl::initClock {} {
	# Auto-loading stubs for 'clock.tcl'

	foreach cmd {add format scan} {
	    proc ::tcl::clock::$cmd args {
		variable TclLibDir
		source [file join $TclLibDir clock.tcl]
		return [uplevel 1 [info level 0]]
	    }
	}

	rename ::tcl::initClock {}
    }
    ::tcl::initClock

Changes to library/install.tcl.

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
      ###
      set package [lindex [split $fname -] 0]
      set version [lindex [split $fname -] 1]
      ###
      # Read the file, and override assumptions as needed
      ###
      set fin [open $file r]
      fconfigure $fin -eofchar \032
      set dat [read $fin]
      close $fin
      # Look for a teapot style Package statement
      foreach line [split $dat \n] {
        set line [string trim $line]
        if { [string range $line 0 9] != "# Package " } continue
        set package [lindex $line 2]







|







31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
      ###
      set package [lindex [split $fname -] 0]
      set version [lindex [split $fname -] 1]
      ###
      # Read the file, and override assumptions as needed
      ###
      set fin [open $file r]
      fconfigure $fin -encoding utf-8 -eofchar \032
      set dat [read $fin]
      close $fin
      # Look for a teapot style Package statement
      foreach line [split $dat \n] {
        set line [string trim $line]
        if { [string range $line 0 9] != "# Package " } continue
        set package [lindex $line 2]
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
        break
      }
      append buffer "package ifneeded $package $version \[list source \[file join \$dir [file tail $file]\]\]" \n
    }
    foreach file [glob -nocomplain $path/*.tcl] {
      if { [file tail $file] == "version_info.tcl" } continue
      set fin [open $file r]
      fconfigure $fin -eofchar \032
      set dat [read $fin]
      close $fin
      if {![regexp "package provide" $dat]} continue
      set fname [file rootname [file tail $file]]
      # Look for a package provide statement
      foreach line [split $dat \n] {
        set line [string trim $line]
        if { [string range $line 0 14] != "package provide" } continue
        set package [lindex $line 2]
        set version [lindex $line 3]
        if {[string index $package 0] in "\$ \[ @"} continue
        if {[string index $version 0] in "\$ \[ @"} continue
        append buffer "package ifneeded $package $version \[list source \[file join \$dir [file tail $file]\]\]" \n
        break
      }
    }
    return $buffer
  }
  set fin [open $pkgidxfile r]
  fconfigure $fin -eofchar \032
  set dat [read $fin]
  close $fin
  set trace 0
  #if {[file tail $path] eq "tool"} {
  #  set trace 1
  #}
  set thisline {}







|



















|







55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
        break
      }
      append buffer "package ifneeded $package $version \[list source \[file join \$dir [file tail $file]\]\]" \n
    }
    foreach file [glob -nocomplain $path/*.tcl] {
      if { [file tail $file] == "version_info.tcl" } continue
      set fin [open $file r]
      fconfigure $fin -encoding utf-8 -eofchar \032
      set dat [read $fin]
      close $fin
      if {![regexp "package provide" $dat]} continue
      set fname [file rootname [file tail $file]]
      # Look for a package provide statement
      foreach line [split $dat \n] {
        set line [string trim $line]
        if { [string range $line 0 14] != "package provide" } continue
        set package [lindex $line 2]
        set version [lindex $line 3]
        if {[string index $package 0] in "\$ \[ @"} continue
        if {[string index $version 0] in "\$ \[ @"} continue
        append buffer "package ifneeded $package $version \[list source \[file join \$dir [file tail $file]\]\]" \n
        break
      }
    }
    return $buffer
  }
  set fin [open $pkgidxfile r]
  fconfigure $fin -encoding utf-8 -eofchar \032
  set dat [read $fin]
  close $fin
  set trace 0
  #if {[file tail $path] eq "tool"} {
  #  set trace 1
  #}
  set thisline {}

Changes to library/package.tcl.

405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
		lappend cmd "-$type" [list $file $procs]
	    }
	}
	append index "\n[eval $cmd]"
    }

    set f [open [file join $dir pkgIndex.tcl] w]
    fconfigure $f -translation lf
    puts $f $index
    close $f
}

# tclPkgSetup --
# This is a utility procedure use by pkgIndex.tcl files.  It is invoked as
# part of a "package ifneeded" script.  It calls "package provide" to indicate







|







405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
		lappend cmd "-$type" [list $file $procs]
	    }
	}
	append index "\n[eval $cmd]"
    }

    set f [open [file join $dir pkgIndex.tcl] w]
    fconfigure $f -encoding utf-8 -translation lf
    puts $f $index
    close $f
}

# tclPkgSetup --
# This is a utility procedure use by pkgIndex.tcl files.  It is invoked as
# part of a "package ifneeded" script.  It calls "package provide" to indicate

Changes to library/safe.tcl.

933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
	set at 2
	if {$encoding eq "identity"} {
	    Log $child "attempt to use the identity encoding"
	    return -code error "permission denied"
	}
    } else {
	set at 0
	set encoding {}
    }
    if {$argc != 1} {
	set msg "wrong # args: should be \"source ?-encoding E? fileName\""
	Log $child "$msg ($args)"
	return -code error $msg
    }
    set file [lindex $args $at]







|







933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
	set at 2
	if {$encoding eq "identity"} {
	    Log $child "attempt to use the identity encoding"
	    return -code error "permission denied"
	}
    } else {
	set at 0
	set encoding utf-8
    }
    if {$argc != 1} {
	set msg "wrong # args: should be \"source ?-encoding E? fileName\""
	Log $child "$msg ($args)"
	return -code error $msg
    }
    set file [lindex $args $at]
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
    # Passed all the tests, lets source it. Note that we do this all manually
    # because we want to control [info script] in the child so information
    # doesn't leak so much. [Bug 2913625]
    set old [::interp eval $child {info script}]
    set replacementMsg "script error"
    set code [catch {
	set f [open $realfile]
	fconfigure $f -eofchar \032
	if {$encoding ne ""} {
	    fconfigure $f -encoding $encoding
	}
	set contents [read $f]
	close $f
	::interp eval $child [list info script $file]
    } msg opt]
    if {$code == 0} {
	set code [catch {::interp eval $child $contents} msg opt]
	set replacementMsg $msg







<
<
|
<







976
977
978
979
980
981
982


983

984
985
986
987
988
989
990
    # Passed all the tests, lets source it. Note that we do this all manually
    # because we want to control [info script] in the child so information
    # doesn't leak so much. [Bug 2913625]
    set old [::interp eval $child {info script}]
    set replacementMsg "script error"
    set code [catch {
	set f [open $realfile]


	fconfigure $f -encoding $encoding -eofchar \032

	set contents [read $f]
	close $f
	::interp eval $child [list info script $file]
    } msg opt]
    if {$code == 0} {
	set code [catch {::interp eval $child $contents} msg opt]
	set replacementMsg $msg

Changes to library/tm.tcl.

263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
		    # understands it, or, if not, update its implementation
		    # appropriately.
		    #
		    # Right now LOCATE's implementation assumes that the path
		    # of the package file is the last element in the list.

		    package ifneeded $pkgname $pkgversion \
			"[::list package provide $pkgname $pkgversion];[::list source -encoding utf-8 $file]"

		    # We abort in this unknown handler only if we got a
		    # satisfying candidate for the requested package.
		    # Otherwise we still have to fallback to the regular
		    # package search to complete the processing.

		    if {($pkgname eq $name)







|







263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
		    # understands it, or, if not, update its implementation
		    # appropriately.
		    #
		    # Right now LOCATE's implementation assumes that the path
		    # of the package file is the last element in the list.

		    package ifneeded $pkgname $pkgversion \
			"[::list package provide $pkgname $pkgversion];[::list source $file]"

		    # We abort in this unknown handler only if we got a
		    # satisfying candidate for the requested package.
		    # Otherwise we still have to fallback to the regular
		    # package search to complete the processing.

		    if {($pkgname eq $name)

Changes to tests/source.test.

110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
    set sourcefile [makeFile {} source.file]
} -body {
    set out [open $sourcefile w]
    fconfigure $out -encoding utf-8
    puts $out "\ufeffset y new-y"
    close $out
    set y old-y
    source -encoding utf-8 $sourcefile
    return $y
} -cleanup {
    removeFile $sourcefile
} -result {new-y}

test source-3.1 {return in middle of source file} -setup {
    set sourcefile [makeFile {







|







110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
    set sourcefile [makeFile {} source.file]
} -body {
    set out [open $sourcefile w]
    fconfigure $out -encoding utf-8
    puts $out "\ufeffset y new-y"
    close $out
    set y old-y
    source $sourcefile
    return $y
} -cleanup {
    removeFile $sourcefile
} -result {new-y}

test source-3.1 {return in middle of source file} -setup {
    set sourcefile [makeFile {
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
    file delete $sourcefile
    set f [open $sourcefile w]
    fconfigure $f -encoding utf-8
    puts $f "set symbol(square-root) \u221A; set x correct"
    close $f
} -body {
    set x unset
    source -encoding utf-8 $sourcefile
    set x
} -cleanup {
    removeFile source.file
} -result correct
test source-7.2 {source -encoding test} -setup {
    # This tests for bad interactions between [source -encoding]
    # and use of the Control-Z character (\u001A) as a cross-platform







|







222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
    file delete $sourcefile
    set f [open $sourcefile w]
    fconfigure $f -encoding utf-8
    puts $f "set symbol(square-root) \u221A; set x correct"
    close $f
} -body {
    set x unset
    source $sourcefile
    set x
} -cleanup {
    removeFile source.file
} -result correct
test source-7.2 {source -encoding test} -setup {
    # This tests for bad interactions between [source -encoding]
    # and use of the Control-Z character (\u001A) as a cross-platform
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
    set sourcefile [makeFile {} source.file]
    file delete $sourcefile
    set f [open $sourcefile w]
    fconfigure $f -encoding utf-8
    puts $f "proc \u20ac {} {return foo}"
    close $f
} -body {
    source -encoding utf-8 $sourcefile
    \u20ac
} -cleanup {
    removeFile source.file
    rename \u20ac {}
} -result foo
test source-7.6 {source -encoding: mismatch encoding error} -setup {
    set sourcefile [makeFile {} source.file]







|







265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
    set sourcefile [makeFile {} source.file]
    file delete $sourcefile
    set f [open $sourcefile w]
    fconfigure $f -encoding utf-8
    puts $f "proc \u20ac {} {return foo}"
    close $f
} -body {
    source $sourcefile
    \u20ac
} -cleanup {
    removeFile source.file
    rename \u20ac {}
} -result foo
test source-7.6 {source -encoding: mismatch encoding error} -setup {
    set sourcefile [makeFile {} source.file]

Changes to unix/README.

87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
	--enable-man-compression=PROG
				Compress the manpages using PROG.
	--enable-dtrace		Enable tcl DTrace provider (if DTrace is
				available on the platform), c.f. tclDTrace.d
				for descriptions of the probes made available,
				see http://wiki.tcl.tk/DTrace for more details
	--with-encoding=ENCODING Specifies the encoding for compile-time
				configuration values. Defaults to iso8859-1,
				which is also sufficient for ASCII.
	--with-tzdata=FLAG	Specifies whether to install timezone data. By
				default, the configure script tries to detect
				whether a usable timezone database is present
				on the system already.

    Mac OS X only (i.e. completely unsupported on other platforms):







|







87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
	--enable-man-compression=PROG
				Compress the manpages using PROG.
	--enable-dtrace		Enable tcl DTrace provider (if DTrace is
				available on the platform), c.f. tclDTrace.d
				for descriptions of the probes made available,
				see http://wiki.tcl.tk/DTrace for more details
	--with-encoding=ENCODING Specifies the encoding for compile-time
				configuration values. Defaults to utf-8,
				which is also sufficient for ASCII.
	--with-tzdata=FLAG	Specifies whether to install timezone data. By
				default, the configure script tries to detect
				whether a usable timezone database is present
				on the system already.

    Mac OS X only (i.e. completely unsupported on other platforms):

Changes to unix/configure.

1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
  --enable-framework      package shared libraries in MacOSX frameworks
                          (default: off)

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-encoding         encoding for configuration values (default:
                          iso8859-1)
  --with-system-libtommath
                          use external libtommath (default: true if available,
                          false otherwise)
  --with-tzdata           install timezone data (default: autodetect)

Some influential environment variables:
  CC          C compiler command







|







1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
  --enable-framework      package shared libraries in MacOSX frameworks
                          (default: off)

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-encoding         encoding for configuration values (default:
                          utf-8)
  --with-system-libtommath
                          use external libtommath (default: true if available,
                          false otherwise)
  --with-tzdata           install timezone data (default: autodetect)

Some influential environment variables:
  CC          C compiler command
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992

cat >>confdefs.h <<_ACEOF
#define TCL_CFGVAL_ENCODING "${with_tcencoding}"
_ACEOF

    else

$as_echo "#define TCL_CFGVAL_ENCODING \"iso8859-1\"" >>confdefs.h

    fi


#--------------------------------------------------------------------
# Look for libraries that we will need when compiling the Tcl shell
#--------------------------------------------------------------------







|







3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992

cat >>confdefs.h <<_ACEOF
#define TCL_CFGVAL_ENCODING "${with_tcencoding}"
_ACEOF

    else

$as_echo "#define TCL_CFGVAL_ENCODING \"utf-8\"" >>confdefs.h

    fi


#--------------------------------------------------------------------
# Look for libraries that we will need when compiling the Tcl shell
#--------------------------------------------------------------------
9734
9735
9736
9737
9738
9739
9740
9741
9742
9743
9744
9745
9746
9747
9748
9749
9750
fi
done

    fi

$as_echo "#define USE_VFORK 1" >>confdefs.h


$as_echo "#define TCL_DEFAULT_ENCODING \"utf-8\"" >>confdefs.h


$as_echo "#define TCL_LOAD_FROM_MEMORY 1" >>confdefs.h


$as_echo "#define TCL_WIDE_CLICKS 1" >>confdefs.h

    for ac_header in AvailabilityMacros.h







<
<
<







9734
9735
9736
9737
9738
9739
9740



9741
9742
9743
9744
9745
9746
9747
fi
done

    fi

$as_echo "#define USE_VFORK 1" >>confdefs.h





$as_echo "#define TCL_LOAD_FROM_MEMORY 1" >>confdefs.h


$as_echo "#define TCL_WIDE_CLICKS 1" >>confdefs.h

    for ac_header in AvailabilityMacros.h

Changes to unix/configure.ac.

602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
    AC_CHECK_HEADERS(copyfile.h)
    AC_CHECK_FUNCS(copyfile)
    if test $tcl_corefoundation = yes; then
	AC_CHECK_HEADERS(libkern/OSAtomic.h)
	AC_CHECK_FUNCS(OSSpinLockLock)
    fi
    AC_DEFINE(USE_VFORK, 1, [Should we use vfork() instead of fork()?])
    AC_DEFINE(TCL_DEFAULT_ENCODING, "utf-8",
	[Are we to override what our default encoding is?])
    AC_DEFINE(TCL_LOAD_FROM_MEMORY, 1,
	[Can this platform load code from memory?])
    AC_DEFINE(TCL_WIDE_CLICKS, 1,
	[Does this platform have wide high-resolution clicks?])
    AC_CHECK_HEADERS(AvailabilityMacros.h)
    if test "$ac_cv_header_AvailabilityMacros_h" = yes; then
	AC_CACHE_CHECK([if weak import is available], tcl_cv_cc_weak_import, [







<
<







602
603
604
605
606
607
608


609
610
611
612
613
614
615
    AC_CHECK_HEADERS(copyfile.h)
    AC_CHECK_FUNCS(copyfile)
    if test $tcl_corefoundation = yes; then
	AC_CHECK_HEADERS(libkern/OSAtomic.h)
	AC_CHECK_FUNCS(OSSpinLockLock)
    fi
    AC_DEFINE(USE_VFORK, 1, [Should we use vfork() instead of fork()?])


    AC_DEFINE(TCL_LOAD_FROM_MEMORY, 1,
	[Can this platform load code from memory?])
    AC_DEFINE(TCL_WIDE_CLICKS, 1,
	[Does this platform have wide high-resolution clicks?])
    AC_CHECK_HEADERS(AvailabilityMacros.h)
    if test "$ac_cv_header_AvailabilityMacros_h" = yes; then
	AC_CACHE_CHECK([if weak import is available], tcl_cv_cc_weak_import, [

Changes to unix/tcl.m4.

2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
#		TCL_CFGVAL_ENCODING
#
#--------------------------------------------------------------------

AC_DEFUN([SC_TCL_CFG_ENCODING], [
    AC_ARG_WITH(encoding,
	AC_HELP_STRING([--with-encoding],
	    [encoding for configuration values (default: iso8859-1)]),
	with_tcencoding=${withval})

    if test x"${with_tcencoding}" != x ; then
	AC_DEFINE_UNQUOTED(TCL_CFGVAL_ENCODING,"${with_tcencoding}",
	    [What encoding should be used for embedded configuration info?])
    else
	AC_DEFINE(TCL_CFGVAL_ENCODING,"iso8859-1",
	    [What encoding should be used for embedded configuration info?])
    fi
])

#--------------------------------------------------------------------
# SC_TCL_CHECK_BROKEN_FUNC
#







|






|







2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
#		TCL_CFGVAL_ENCODING
#
#--------------------------------------------------------------------

AC_DEFUN([SC_TCL_CFG_ENCODING], [
    AC_ARG_WITH(encoding,
	AC_HELP_STRING([--with-encoding],
	    [encoding for configuration values (default: utf-8)]),
	with_tcencoding=${withval})

    if test x"${with_tcencoding}" != x ; then
	AC_DEFINE_UNQUOTED(TCL_CFGVAL_ENCODING,"${with_tcencoding}",
	    [What encoding should be used for embedded configuration info?])
    else
	AC_DEFINE(TCL_CFGVAL_ENCODING,"utf-8",
	    [What encoding should be used for embedded configuration info?])
    fi
])

#--------------------------------------------------------------------
# SC_TCL_CHECK_BROKEN_FUNC
#

Changes to unix/tclConfig.h.in.

411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427

/* Is bytecode debugging enabled? */
#undef TCL_COMPILE_DEBUG

/* Are bytecode statistics enabled? */
#undef TCL_COMPILE_STATS

/* Are we to override what our default encoding is? */
#undef TCL_DEFAULT_ENCODING

/* Is Tcl built as a framework? */
#undef TCL_FRAMEWORK

/* Can this platform load code from memory? */
#undef TCL_LOAD_FROM_MEMORY

/* Is memory debugging enabled? */







<
<
<







411
412
413
414
415
416
417



418
419
420
421
422
423
424

/* Is bytecode debugging enabled? */
#undef TCL_COMPILE_DEBUG

/* Are bytecode statistics enabled? */
#undef TCL_COMPILE_STATS




/* Is Tcl built as a framework? */
#undef TCL_FRAMEWORK

/* Can this platform load code from memory? */
#undef TCL_LOAD_FROM_MEMORY

/* Is memory debugging enabled? */

Changes to unix/tclUnixInit.c.

88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
/*
 * Tcl tries to use standard and homebrew methods to guess the right encoding
 * on the platform. However, there is always a final fallback, and this value
 * is it. Make sure it is a real Tcl encoding.
 */

#ifndef TCL_DEFAULT_ENCODING
#define TCL_DEFAULT_ENCODING "iso8859-1"
#endif

/*
 * Default directory in which to look for Tcl library scripts. The symbol is
 * defined by Makefile.
 */








|







88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
/*
 * Tcl tries to use standard and homebrew methods to guess the right encoding
 * on the platform. However, there is always a final fallback, and this value
 * is it. Make sure it is a real Tcl encoding.
 */

#ifndef TCL_DEFAULT_ENCODING
#define TCL_DEFAULT_ENCODING "utf-8"
#endif

/*
 * Default directory in which to look for Tcl library scripts. The symbol is
 * defined by Makefile.
 */

Changes to win/configure.

3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758

    if test x"${with_tcencoding}" != x ; then
	cat >>confdefs.h <<_ACEOF
#define TCL_CFGVAL_ENCODING "${with_tcencoding}"
_ACEOF

    else
	# Default encoding on windows is not "iso8859-1"
	$as_echo "#define TCL_CFGVAL_ENCODING \"cp1252\"" >>confdefs.h

    fi


#--------------------------------------------------------------------
# The statements below define a collection of symbols related to
# building libtcl as a shared library instead of a static library.







<
|







3743
3744
3745
3746
3747
3748
3749

3750
3751
3752
3753
3754
3755
3756
3757

    if test x"${with_tcencoding}" != x ; then
	cat >>confdefs.h <<_ACEOF
#define TCL_CFGVAL_ENCODING "${with_tcencoding}"
_ACEOF

    else

	$as_echo "#define TCL_CFGVAL_ENCODING \"utf-8\"" >>confdefs.h

    fi


#--------------------------------------------------------------------
# The statements below define a collection of symbols related to
# building libtcl as a shared library instead of a static library.

Changes to win/tcl.m4.

1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096

AC_DEFUN([SC_TCL_CFG_ENCODING], [
    AC_ARG_WITH(encoding, [  --with-encoding         encoding for configuration values], with_tcencoding=${withval})

    if test x"${with_tcencoding}" != x ; then
	AC_DEFINE_UNQUOTED(TCL_CFGVAL_ENCODING,"${with_tcencoding}")
    else
	# Default encoding on windows is not "iso8859-1"
	AC_DEFINE(TCL_CFGVAL_ENCODING,"cp1252")
    fi
])

#--------------------------------------------------------------------
# SC_EMBED_MANIFEST
#
#	Figure out if we can embed the manifest where necessary







<
|







1081
1082
1083
1084
1085
1086
1087

1088
1089
1090
1091
1092
1093
1094
1095

AC_DEFUN([SC_TCL_CFG_ENCODING], [
    AC_ARG_WITH(encoding, [  --with-encoding         encoding for configuration values], with_tcencoding=${withval})

    if test x"${with_tcencoding}" != x ; then
	AC_DEFINE_UNQUOTED(TCL_CFGVAL_ENCODING,"${with_tcencoding}")
    else

	AC_DEFINE(TCL_CFGVAL_ENCODING,"utf-8")
    fi
])

#--------------------------------------------------------------------
# SC_EMBED_MANIFEST
#
#	Figure out if we can embed the manifest where necessary