Author:         Jan Nijtmans <[email protected]>
Author:	        Christian Werner <[email protected]>
State:          Draft
Type:           Project
Tcl-Version:    8.6, 9.0
Tcl-Branch:     tip-709

Abstract

This TIP proposes to accept MemoryModule.[ch] in the Tcl Repository for the Windows build. These files are under the Mozilla Public License as prominently stated at the very top of these files.

In the default Windows build, MemoryModule.[ch] is not used. In Tcl 8.6 and 9.0 it is only enabled when the build is configured with --enable-memorymodule.

Rationale

This TIP is meant to solve a longstanding bug in the implementation of the VFS loader for Windows. On Windows a dynamically-linked library (DLL) residing in a VFS is copied to a temporary directory before being loaded. This directory is never cleaned up, resulting in left-over temporary directories with every tclsh/wish run. The bug is present in all Tcl releases (8.6 too), but it became more visible in 9.0 because of the added support for a zipfs, which is the reason this TIP targets 8.6 and 9.0.

A TIP is needed since MemoryModule.[ch] is distributed under a different license that is compatible with the Tcl license.

The Mozilla Public License doesn't impose any restrictions on software linked to it: the whole combined work can still be distributed under any license. But there are some restrictions on the two mentioned files:

Any modification of those 2 files must be done under the MPL license too.
The source code of such a modification must be published.

Tcl is compliant with those restrictions.

The current implementation of MemoryModule.[ch] is derived from MemoryModule master branch, with 4 additional pull-requests added. And it contains a trivial fix, making it work on ARM64 too.

Specification

This is an opt-in feature which must explicitly enabled by the option --enable-memorymodule when using the Windows Makefile build or OPTS=memorymodule when using the makefile.vc build."

When enabling this, a DLL inside a VFS is no longer copied to a temporary directory in order to load it. The DLL is loaded directly in memory using the functionality of MemoryModule.

Known limitations

There are two known limitations at the moment. If any of those affect you, don't configure Tcl with --enable-memorymodule. The limitations might be lifted in the future.

Loading a DLL which depends on another DLL that is also in a VFS doesn't work, but that doesn't work without MemoryModule either, so it cannot be seen as a limitation of MemoryModule itself.
A DLL that is protected with WinLicense does not work.

If the loading by MemoryModule fails, a fallback to the copy strategy takes place.

TLS (Thread Local Storage)

A DLL using TLS works fine with MemoryModule. When compiling with MSVC nothing special needs to be done. When compiling such a DLL with gcc the resulting DLL depends on libgcc_s_seh-1.dll which in turn depends on libwinpthread-1.dll. Those two DLL's are not normally installed on Windows so limitation 1) above comes into play. If those 2 DLL's are present, loading from memory works. Otherwise the fallback to DLL copying takes place (which doesn't help in this case).

Implementation

See the tip-709 branch.

A new win/dltest directory with testcases for the MemoryModule functionality is added.

Extensive tests are currently done using the LUCK (Lean Undroidwish Construction Kit) environment.

Copyright

This document has been placed in the public domain.