View Ticket
Bounty program for improvements to Tcl and certain Tcl packages.
Ticket Hash: 034c8d2587c3810d268e84a15d2a3e187e54b452
Title: Issue with OpenSSL1.1 and dh_param access
Status: Open Type: Build Problem
Severity: Important Priority: Immediate
Subsystem: Resolution: Open
Last Modified: 2020-08-15 21:47:25
Version Found In: 1.7.21
User Comments:
betsalel added on 2020-05-14 21:30:24: (text/x-fossil-plain)
Issue compiling with openSSL1.1 on Mac:

In file included from ./tls.c:84:
./dh_params.h:33:4: error: incomplete definition of type 'struct dh_st'
        dh->p = BN_bin2bn(dh2048_p, sizeof(dh2048_p), NULL);
        ~~^
/usr/local/opt/[email protected]/include/openssl/ossl_typ.h:104:16: note: forward declaration of 'struct dh_st'
typedef struct dh_st DH;
               ^
In file included from ./tls.c:84:
./dh_params.h:34:4: error: incomplete definition of type 'struct dh_st'
        dh->g = BN_bin2bn(dh2048_g, sizeof(dh2048_g), NULL);
        ~~^
/usr/local/opt/[email protected]/include/openssl/ossl_typ.h:104:16: note: forward declaration of 'struct dh_st'
typedef struct dh_st DH;
               ^
In file included from ./tls.c:84:
./dh_params.h:35:9: error: incomplete definition of type 'struct dh_st'
        if ((dh->p == NULL) || (dh->g == NULL))
             ~~^
/usr/local/opt/[email protected]/include/openssl/ossl_typ.h:104:16: note: forward declaration of 'struct dh_st'
typedef struct dh_st DH;
               ^
In file included from ./tls.c:84:
./dh_params.h:35:28: error: incomplete definition of type 'struct dh_st'
        if ((dh->p == NULL) || (dh->g == NULL))
                                ~~^
/usr/local/opt/[email protected]/include/openssl/ossl_typ.h:104:16: note: forward declaration of 'struct dh_st'
typedef struct dh_st DH;


Fixed by updating the dh_params.h to use the getter/setter methods (used DH_set0_pqg instead of dh->p and dh->g):

	if ((dh = DH_new()) == NULL) return(NULL);
	DH_set0_pqg(dh, BN_bin2bn(dh2048_p, sizeof(dh2048_p), NULL), NULL, BN_bin2bn(dh2048_g, sizeof(dh2048_g), NULL));
	if ((DH_get0_p(dh) == NULL) || (DH_get0_g(dh) == NULL))
		{ DH_free(dh); return(NULL); }

anonymous (claiming to be medranocalvo) added on 2020-05-29 07:47:10: (text/x-fossil-plain)
The actual issue here is that the `openssl` program used for generating dh_param.h is the one found in the search path (PATH), and not one found via pkg-config.

That is, by default macOS ships an old OpenSSL 0.9.8 (actually LibreSSL, IIRC).  When one provides a newer one (e.g. installed through brew) with PKG_CONFIG_PATH, the gen_dh_params script will use the first `openssl` executable it finds on PATH, which generates code incompatible with the new API.

A possible solution would be to search an openssl executable in the exec_prefix found by pkg-config, and use it in gen_dh_params.

anonymous (claiming to be lars_h) added on 2020-08-15 21:47:25:
The comment by medranocalvo saved me here!

Considering the very high degree of redundancy in the gen_dh_params script:
 1. call openssl executable,
 2. download parameters from the internet,
 3. use precomputed parameter values embedded in the script;
it seems ridiculous that it does not check whether the openssl executable 
option 1 picks has a suitable version. Option 3 *does* account for the API 
differences (has more bitsizes, and is likely way faster, so why is that 
not preferred?).