tls(n) 1.8 tls "Tcl TLS extension"

Name

tls - binding to the OpenSSL library for encrypted socket and I/O channel communications

Table Of Contents

Synopsis

Description

This extension provides TCL script access to secure socket communications using the Transport Layer Security (TLS) protocol. It provides a generic binding to OpenSSL, utilizing the Tcl_StackChannel API in TCL 8.4 and higher. These sockets behave exactly the same as channels created using the built-in socket command, along with additional options for controlling the SSL/TLS session.

Commands

Typically one would use the tls::socket command to create a new encrypted TCP socket. It is compatible with the native TCL ::socket command. Alternatively for an existing TCP socket, the tls::import command can be used to start TLS on the connection.

tls::init ?-option? ?value? ?-option value ...?

Optional function to set the default options used by tls::socket. If you call tls::import directly, this command has no effect. This command supports all of the same options as the tls::socket command, though you should limit your options to only TLS related ones.

tls::socket ?-option? ?value? ?-option value ...? host port

This is a helper function that utilizes the underlying commands socket and tls::import to create the connection. It behaves the same as the native TCL socket command, but also supports the tls:import command options with one additional option. It returns the channel handle id for the new socket.

-autoservername bool

If true, automatically set the -servername argument to the host argument. Default is false.

tls::socket -server command ?-option? ?value? ?-option value ...? port

Same as previous, but instead creates a server socket for clients to connect to just like the Tcl socket -server command. It returns the channel handle id for the new socket.

tls::import channel ?-option? ?value? ?-option value ...?

Start TLS encryption on TCL channel channel via a stacked channel. It need not be a socket, but must provide bi-directional flow. Also sets session parameters for SSL handshake. Valid options are:

-alpn list

List of protocols to offer during Application-Layer Protocol Negotiation (ALPN). For example: h2 and http/1.1, but not h3 or quic.

-cadir directory

Specifies the directory where the Certificate Authority (CA) certificates are stored. The default is platform specific and can be set at compile time. The default location can be overridden by the SSL_CERT_DIR environment variable. See Certificate Validation for more details.

-cafile filename

Specifies the file with the Certificate Authority (CA) certificates to use. The default is "cert.pem", in the OpenSSL directory. The default file can be overridden by the SSL_CERT_FILE environment variable. See Certificate Validation for more details.

-castore URI

Specifies the Uniform Resource Identifier (URI) for the Certificate Authority (CA) store, which may be a single container or a catalog of containers. Starting with OpenSSL 3.2 on Windows, set to "org.openssl.winstore://" to use the built-in Windows Certificate Store. This store only supports root certificate stores. See Certificate Validation for more details.

-certfile filename

Specifies the name of the file with the certificate in PEM format to use as the local (client or server) certificate. It also contains the public key.

-cert string

Specifies the certificate to use as a DER encoded string (X.509 DER).

-cipher string

Specifies the list of ciphers to use for TLS 1.2 and earlier connections. String is a colon ":" separated list of ciphers. Ciphers can be combined using the "+" character. Prefixes can be used to permanently remove "!", delete "-", or move to the end "+" a specified cipher. Keywords @STRENGTH (sort by algorithm key length), @SECLEVEL=n (set security level to n), and DEFAULT (use default cipher list, at start only) can also be specified. See the OpenSSL documentation for the full list of valid values.

-ciphersuites string

Specifies the list of cipher suites to use for TLS 1.3 as a colon ":" separated list of cipher suite names. See the OpenSSL documentation for the full list of valid values.

-command callback

Specifies the callback command to be invoked at several points during the handshake to pass errors, tracing information, and protocol messages. See Callback Options for more info.

-dhparams filename

Specifies the Diffie-Hellman (DH) parameters file.

-keyfile filename

Specifies the private key file. The default value is to use the file specified by the -certfile option.

-key string

Specifies the private key to use as a DER encoded string (PKCS#1 DER).

-model channel

Force this channel to share the same SSL_CTX structure as the specified channel, and therefore share config, callbacks, etc.

-password callback

Specifies the callback command to invoke when OpenSSL needs to obtain a password. This is typically used to unlock the private key of a certificate. The callback should return a password string. See Callback Options for more info.

-post_handshake bool

Allow post-handshake session ticket updates.

-request bool

Request a certificate from peer during the SSL handshake. This is needed to do Certificate Validation. Default is true. See Certificate Validation for more details.

-require bool

Require a valid certificate from peer during the SSL handshake. If this is set to true, then -request must also be set to true and a either -cadir, -cafile, -castore, or a platform default must be provided in order to validate against. The default is false since not all platforms have certificates to validate against in a form compatible with OpenSSL. See Certificate Validation for more details.

-security_level integer

Specifies the security level (value from 0 to 5). The security level affects the allowed cipher suite encryption algorithms, supported ECC curves, supported signature algorithms, DH parameter sizes, certificate key sizes and signature algorithms. The default is 1 prior to OpenSSL 3.2 and 2 thereafter. Level 3 and higher disable support for session tickets and only accept cipher suites that provide forward secrecy.

-server bool

Specifies whether to act as a server and respond with a server handshake when a client connects and provides a client handshake. The default is false.

-servername hostname

Specify the peer's hostname. This is used to set the TLS Server Name Indication (SNI) extension. Set this to the expected servername in the server's certificate or one of the Subject Alternate Names (SAN).

-session_id binary_string

Specifies the session id to resume a session. Not supported yet.

-ssl2 bool

Enable use of SSL v2. The default is false. Note: Recent versions of OpenSSL no longer support SSLv2, so this may not have any effect. See the tls::protocols command for supported protocols.

-ssl3 bool

Enable use of SSL v3. The default is false. Note: Recent versions of OpenSSL may have this disabled at compile time, so this may not have any effect. See the tls::protocols command for supported protocols.

-tls1 bool

Enable use of TLS v1. The default is true. Note: TLS 1.0 needs SHA1 to operate, which is only available in security level 0 for Open SSL 3.0+. See the -security_level option.

-tls1.1 bool

Enable use of TLS v1.1. The default is true. Note: TLS 1.1 needs SHA1 to operate, which is only available in security level 0 for Open SSL 3.0+. See the -security_level option.

-tls1.2 bool

Enable use of TLS v1.2. The default is true.

-tls1.3 bool

Enable use of TLS v1.3. The default is true.

-validatecommand callback

Specifies the callback command to invoke to validate the peer certificates and other config info during the protocol negotiation phase. This can be used by TCL scripts to perform their own Certificate Validation to supplement the default validation provided by OpenSSL. The script must return a boolean true to continue the negotiation. See Callback Options for more info.

tls::unimport channel

Compliment to tls::import. Used to remove the top level stacked channel from channel. This unstacks the encryption of a regular TCL channel. An error is thrown if TLS is not the top stacked channel type.

tls::handshake channel

Forces the TLS negotiation handshake to take place immediately, and returns 0 if handshake is still in progress (non-blocking), or 1 if the handshake was successful. If the handshake failed, an error will be returned.

tls::status ?-local? channel

Returns the current status of an SSL channel. The result is a list of key-value pairs describing the SSL, certificate, and certificate verification status. If the SSL handshake has not yet completed, an empty list is returned. If the -local option is specified, then the local certificate is used. Returned values include:

SSL Status

alpn protocol

The protocol selected after Application-Layer Protocol Negotiation (ALPN).

cipher cipher

The current cipher in use for the session.

peername name

The peername from the certificate.

protocol version

The protocol version used for the connection: SSL2, SSL3, TLS1, TLS1.1, TLS1.2, TLS1.3, or unknown.

sbits n

The number of bits used for the session key.

signatureHashAlgorithm algorithm

The signature hash algorithm.

signatureType type

The signature type value.

verifyDepth n

Maximum depth for the certificate chain verification. Default is -1, to check all.

verifyMode list

List of certificate verification modes.

verifyResult result

Certificate verification result.

ca_names list

List of the Certificate Authorities used to create the certificate.

Certificate Status

all string

Dump of all certificate info.

version value

The certificate version.

serialNumber string

The serial number of the certificate as a hex string.

signature algorithm

Cipher algorithm used for certificate signature.

issuer string

The distinguished name (DN) of the certificate issuer.

notBefore date

The beginning date of the certificate validity.

notAfter date

The expiration date of the certificate validity.

subject string

The distinguished name (DN) of the certificate subject. Fields include: Common Name (CN), Organization (O), Locality or City (L), State or Province (S), and Country Name (C).

issuerUniqueID string

The issuer unique id.

subjectUniqueID string

The subject unique id.

num_extensions n

Number of certificate extensions.

extensions list

List of certificate extension names.

authorityKeyIdentifier string

Authority Key Identifier (AKI) of the Issuing CA certificate that signed the SSL certificate as a hex string. This value matches the SKI value of the Intermediate CA certificate.

subjectKeyIdentifier string

Subject Key Identifier (SKI) hash of the public key inside the certificate as a hex string. Used to identify certificates that contain a particular public key.

subjectAltName list

List of all of the Subject Alternative Names (SAN) including domain names, sub domains, and IP addresses that are secured by the certificate.

ocsp list

List of all Online Certificate Status Protocol (OCSP) URLs that can be used to check the validity of this certificate.

certificate cert

The PEM encoded certificate.

signatureAlgorithm algorithm

Cipher algorithm used for the certificate signature.

signatureValue string

Certificate signature as a hex string.

signatureDigest version

Certificate signing digest as a hex string.

publicKeyAlgorithm algorithm

Certificate signature public key algorithm.

publicKey string

Certificate signature public key as a hex string.

bits n

Number of bits used for certificate signature key.

self_signed boolean

Whether the certificate signature is self signed.

sha1_hash hash

The SHA1 hash of the certificate as a hex string.

sha256_hash hash

The SHA256 hash of the certificate as a hex string.

tls::connection channel

Returns the current connection status of an SSL channel. The result is a list of key-value pairs describing the connection. Returned values include:

SSL Status

state state

State of the connection.

servername name

The name of the connected to server.

protocol version

The protocol version used for the connection: SSL2, SSL3, TLS1, TLS1.1, TLS1.2, TLS1.3, or unknown.

renegotiation_allowed boolean

Whether protocol renegotiation is supported or not.

security_level level

The security level used for selection of ciphers, key size, etc.

session_reused boolean

Whether the session has been reused or not.

is_server boolean

Whether the connection is configured as a server (1) or client (0).

compression mode

Compression method.

expansion mode

Expansion method.

caList list

List of Certificate Authorities (CA) for X.509 certificate.

Cipher Info

cipher cipher

The current cipher in use for the connection.

standard_name name

The standard RFC name of cipher.

algorithm_bits n

The number of processed bits used for cipher.

secret_bits n

The number of secret bits used for cipher.

min_version version

The minimum protocol version for cipher.

cipher_is_aead boolean

Whether the cipher is Authenticated Encryption with Associated Data (AEAD).

cipher_id id

The OpenSSL cipher id.

description string

A text description of the cipher.

handshake_digest boolean

Digest used during handshake.

Session Info

alpn protocol

The protocol selected after Application-Layer Protocol Negotiation (ALPN).

resumable boolean

Whether the session can be resumed or not.

start_time seconds

Time since session started in seconds since epoch.

timeout seconds

Max duration of session in seconds before time-out.

lifetime seconds

Session ticket lifetime hint in seconds.

session_id binary_string

Unique session id for use in resuming the session.

session_ticket binary_string

Unique session ticket for use in resuming the session.

ticket_app_data binary_string

Unique session ticket application data.

master_key binary_string

Unique session master key.

session_cache_mode mode

Server cache mode (client, server, or both).

tls::ciphers ?protocol? ?verbose? ?supported?

Without any args, returns a list of all symmetric ciphers for use with the -cipher option. With protocol, only the ciphers supported for that protocol are returned. See the tls::protocols command for the supported protocols. If verbose is specified as true then a verbose, human readable list is returned with additional information on the cipher. If supported is specified as true, then only the ciphers supported for protocol will be listed.

tls::protocols

Returns a list of the supported SSL/TLS protocols. Valid values are: ssl2, ssl3, tls1, tls1.1, tls1.2, and tls1.3. Exact list depends on OpenSSL version and compile time flags.

tls::version

Returns the OpenSSL version string.

Certificate Validation

Summary of command line options

The following options are used for peer Certificate Validation:

-cadir directory

Specifies the directory where the Certificate Authority (CA) certificates are stored. The default is platform specific, but is usually "/etc/ssl/certs" on Linux/Unix systems. The default location can be overridden by the SSL_CERT_DIR environment variable.

-cafile filename

Specifies the file with the Certificate Authority (CA) certificates to use in PEM file format. The default is "cert.pem", in the OpenSSL directory. On Linux/Unix systems, this is usually "/etc/ssl/ca-bundle.pem". The default file can be overridden by the SSL_CERT_FILE environment variable.

-castore URI

Specifies the Uniform Resource Identifier (URI) for the Certificate Authority (CA) store, which may be a single container or a catalog of containers. Starting with OpenSSL 3.2 on Windows, set to "org.openssl.winstore://" to use the built-in Windows Certificate Store. This store only supports root certificate stores.

-request bool

Request a certificate from peer during the SSL handshake. This is needed to do Certificate Validation. Default is true. In addition, the client can manually inspect and accept or reject each certificate using the -validatecommand option.

-require bool

Require a valid certificate from peer during the SSL handshake. If this is set to true, then -request must also be set to true and either -cadir, -cafile, -castore, or a platform default must be provided in order to validate against. The default is false since not all platforms have certificates to validate against in a form compatible with OpenSSL. See Certificate Validation for more details.

When are command line options needed?

By default, a client TLS connection does NOT validate the server certificate chain. This limitation is due to the lack of a common cross platform database of Certificate Authority (CA) provided certificates to validate against. Many Linux systems natively support OpenSSL and thus have these certificates installed as part of the OS, but MacOS and Windows do not. In order to use the -require option, one of the following must be true:

  • On Linux and Unix systems with OpenSSL already installed, if the CA certificates are stored in the standard locations, or if the SSL_CERT_DIR or SSL_CERT_FILE environment variables are set, then -cadir, -cadir, and -castore aren't needed.

  • If OpenSSL is not installed in the default location, or when using Mac OS or Windows and OpenSSL is installed, the SSL_CERT_DIR and/or SSL_CERT_FILE environment variables or the one of the -cadir, -cadir, or -castore options must be defined.

  • On Windows, starting in OpenSSL 3.2, it is now possible to access the built-in Windows Certificate Store from OpenSSL. This can be achieved by setting the -castore option to "org.openssl.winstore://".

  • If OpenSSL is not installed, the CA certificates must be downloaded and installed with the user software. The CURL team makes them available at CA certificates extracted from Mozilla in the "cacert.pem" file. You must then either set the SSL_CERT_DIR and/or SSL_CERT_FILE environment variables or the -cadir or -cafile options to the CA cert file's install location. It is your responsibility to keep this file up to date.

Callback Options

As previously described, each channel can be given their own callbacks to handle intermediate processing by the OpenSSL library, using the -command, -password, and -validate_command options passed to either of tls::socket or tls::import. Unlike previous versions of TclTLS, only if the callback generates an error, will the bgerror command be invoked with the error information.

Values for Command Callback

The callback for the -command option is invoked at several points during the OpenSSL handshake and during routine operations. See below for the possible arguments passed to the callback script. Values returned from the callback are ignored.

error channelId message

This form of callback is invoked whenever an error occurs during the initial connection, handshake, or I/O operations. The message argument can be from the Tcl_ErrnoMsg, OpenSSL function ERR_reason_error_string(), or a custom message. This callback is new for TclTLS 1.8.

info channelId major minor message type

This form of callback is invoked by the OpenSSL function SSL_set_info_callback() during the initial connection and handshake operations. The arguments are:

major

Major category for error. Valid enums are: handshake, alert, connect, accept.

minor

Minor category for error. Valid enums are: start, done, read, write, loop, exit.

message

Descriptive message string which may be generated either by SSL_state_string_long() or SSL_alert_desc_string_long(), depending on the context.

type

For alerts, the possible values are: warning, fatal, and unknown. For others, info is used. This argument is new for TclTLS 1.8.

message channelId direction version content_type message

This form of callback is invoked by the OpenSSL function SSL_set_msg_callback() whenever a message is sent or received during the initial connection, handshake, or I/O operations. It is only available when OpenSSL is complied with the enable-ssl-trace option. This callback is new for TclTLS 1.8. The arguments are:

direction

Direction is either Sent or Received.

version

Version is the protocol version.

content_type

Content type is the message content type.

message

Message is more info from the SSL_trace API. This argument is new for TclTLS 1.8.

session channelId session_id session_ticket lifetime

This form of callback is invoked by the OpenSSL function SSL_CTX_sess_set_new_cb() whenever a new session id is sent by the server during the initial connection and handshake and also during the session if the -post_handshake option is set to true. This callback is new for TclTLS 1.8. The arguments are:

session_id

Session Id is the current session identifier

session_ticket

Ticket is the session ticket info

lifetime

Lifetime is the ticket lifetime in seconds.

verify channelId depth cert status error

This callback was moved to the -verify_callback in TclTLS 1.8.

Values for Password Callback

The callback for the -password option is invoked by TclTLS whenever OpenSSL needs to obtain a password. See below for the possible arguments passed to the callback script. The user provided password is expected to be returned by the callback.

password rwflag size

Invoked when loading or storing an encrypted PEM certificate. The arguments are:

rwflag

The read/write flag is 0 for reading/decryption or 1 for writing/encryption. The latter can be used to determine when to prompt the user to confirm. This argument is new for TclTLS 1.8.

size

The size is the maximum length of the password in bytes. This argument is new for TclTLS 1.8.

Values for Validate Command Callback

The callback for the -validatecommand option is invoked during the handshake process in order for the application to validate the provided value(s). See below for the possible arguments passed to the callback script. If not specified, OpenSSL will accept all valid certificates and extensions. To reject the value and abort the connection, the callback should return 0. To accept the value and continue the connection, it should return 1. To reject the value, but continue the connection, it should return 2. This callback is new for TclTLS 1.8.

alpn channelId protocol match

For servers, this form of callback is invoked when the client ALPN extension is received. If match is true, then protocol is the first -alpn protocol option in common to both the client and server. If not, the first client specified protocol is used. This callback is called after the Hello and ALPN callbacks.

hello channelId servername

For servers, this form of callback is invoked during client hello message processing. The purpose is so the server can select the appropriate certificate to present to the client, and to make other configuration adjustments relevant to that server name and its configuration. It is called before the SNI and ALPN callbacks.

sni channelId servername

For servers, this form of callback is invoked when the Server Name Indication (SNI) extension is received. The servername argument is the client provided server name specified in the -servername</b> option. The purpose is so when a server supports multiple names, the right certificate can be used. It is called after the hello callback but before the ALPN callback.

verify channelId depth cert status error

This form of callback is invoked by OpenSSL when a new certificate is received from the peer. It allows the client to check the certificate verification results and choose whether to continue or not. It is called for each certificate in the certificate chain. This callback was moved from -command in TclTLS 1.8. The arguments are:

depth

The depth is the integer depth of the certificate in the certificate chain, where 0 is the peer certificate and higher values going up to the Certificate Authority (CA).

cert

The cert argument is a list of key-value pairs similar to those returned by tls::status.

status

The status argument is the boolean validity of the current certificate where 0 is invalid and 1 is valid.

error

The error argument is the error message, if any, generated by X509_STORE_CTX_get_error().

Reference implementations of these callbacks are provided in "tls.tcl" as tls::callback, tls::password, and tls::validate_command respectively. Note that these are only sample implementations. In a more realistic deployment you would specify your own callback scripts on each TLS channel using the -command, -password, and -validate_command options.

The default behavior when the -command and -validate_command options are not specified, is for TclTLS to process the associated library callbacks internally. The default behavior when the -password option is not specified is for TclTLS to process the associated library callbacks by attempting to call tls::password. The difference between these two behaviors is a consequence of maintaining compatibility with earlier implementations.

The use of the reference callbacks tls::callback, tls::password, and tls::validate_command is not recommended. They may be removed from future releases.

Debug

For most debugging needs, the -callback option can be used to provide sufficient insight and information on the TLS handshake and progress. If further troubleshooting insight is needed, the compile time option --enable-debug can be used to get detailed execution flow status.

TLS key logging can be enabled by setting the environment variable SSLKEYLOGFILE to the name of the file to log to. Then whenever TLS key material is generated or received it will be logged to the file. This is useful for logging key data for network logging tools to use to decrypt the data.

The tls::debug variable provides some additional control over these reference callbacks. Its value is zero by default. Higher values produce more diagnostic output, and will also force the verify method in tls::callback to accept the certificate, even when it is invalid if the -validatecommand option is set to tls::validate_command.

The use of the variable tls::debug is not recommended. It may be removed from future releases.

Debug Examples

These examples use the default Unix platform SSL certificates. For standard installations, -cadir and -cafile should not be needed. If your certificates are in non-standard locations, update -cadir or use -cafile as needed.

Example #1: Use HTTP package

package require http
package require tls
set url "https://www.tcl.tk/"
http::register https 443 [list ::tls::socket -autoservername true -require true -cadir /etc/ssl/certs  -command ::tls::callback -password ::tls::password -validatecommand ::tls::validate_command]
# Check for error
set token [http::geturl $url]
if {[http::status $token] ne "ok"} {
    puts [format "Error %s" [http::status $token]]
}
# Get web page
set data [http::data $token]
puts [string length $data]
# Cleanup
::http::cleanup $token

Example #2: Use raw socket

package require tls
set url "www.tcl-lang.org"
set port 443
set ch [tls::socket -autoservername 1 -servername $url -request 1 -require 1  -alpn {http/1.1} -cadir /etc/ssl/certs -command ::tls::callback  -password ::tls::password -validatecommand ::tls::validate_command $url $port]
chan configure $ch -buffersize 65536
tls::handshake $ch
puts $ch "GET / HTTP/1.1"
flush $ch
after 500
set data [read $ch]
array set status [tls::status $ch]
array set conn [tls::connection $ch]
array set chan [chan configure $ch]
close $ch
parray status
parray conn
parray chan

HTTP Package Examples

These examples use the default Unix platform SSL certificates. For standard installations, -cadir and -cafile should not be needed. If your certificates are in non-standard locations, set -cadir or use -cafile as needed.

Example #3: Get web page

package require http
package require tls
set url "https://www.tcl.tk/"
http::register https 443 [list ::tls::socket -autoservername true -require true -cadir /etc/ssl/certs]
# Check for error
set token [http::geturl $url]
if {[http::status $token] ne "ok"} {
    puts [format "Error %s" [http::status $token]]
}
# Get web page
set data [http::data $token]
puts $data
# Cleanup
::http::cleanup $token

Example #4: Download file

package require http
package require tls
set url "https://wiki.tcl-lang.org/sitemap.xml"
set filename [file tail $url]
http::register https 443 [list ::tls::socket -autoservername true -require true -cadir /etc/ssl/certs]
# Get file
set ch [open $filename wb]
set token [::http::geturl $url -blocksize 65536 -channel $ch]
# Cleanup
close $ch
::http::cleanup $token

Special Considerations

The capabilities of this package can vary enormously based upon how the linked to OpenSSL library was configured and built. New versions may obsolete older protocol versions, add or remove ciphers, change default values, etc. Use the tls::protocols commands to obtain the supported protocol versions.

See Also

OpenSSL, http, socket

Keywords

I/O, IP Address, OpenSSL, SSL, TCP, TLS, TclTLS, asynchronous I/O, bind, certificate, channel, connection, domain name, host, https, network, network address, socket, tls

Category

tls