| Ticket UUID: | effa2e2346f8372ad4982d16bad189fab96ceb56 | ||
| Title: | RFE - lazy loading of TclOO for faster interp creation | ||
| Type: | Bug | Created on: | 2025-08-16 05:36:29 |
| Submitter: | apnadkarni | Assigned to: | nobody |
| Subsystem: | - New Builtin Commands | Severity: | Minor |
| Priority: | 5 Medium | Last modified: | 2026-03-11 05:59:44 |
| Status: | Closed | Closed by: | apnadkarni |
| Resolution: | Fixed | Closed on: | 2026-03-11 05:59:44 |
| Version: | |||
| Description: | ||||
|
Tcl interp creation time more than doubled in 9.{0,1,2} compared to 8.6 as reported in 62019f8aa9. This difference has mostly been negated for 9.0.3 with zipfs related fixes. However, opportunities for further optimization remain. One of these is loading of TclOO. As measured on my system, TclOO initialization takes up close to 40% of interp creation time. Since a very substantial (majority even?) number of Tcl applications do not use oo, one possibility is to lazy load TclOO. The apn-oo-lazy-init branch implements a proof of concept for this. For comparison: Current main branch:
apn-oo-lazy-init branch:
Test suite passes. The implementation creates stubs commands for the public oo commands that do initialization on first call. The initialization in turn overwrites the stub implementations with the "real" oo implementation. Other implementation strategies may be possible (tclIndex, unknown etc.) but need to keep in mind all code paths need to be covered (e.g. initial invocation through Comments on the proposal, improvements, breakage or even alternative methods appreciated. | ||||
| User Comments: | ||||
apnadkarni added on 2025-08-16 10:39:19:
Note: the branch has not been tested with TclOO's C API. Looking for an extension that uses it ... sebres added on 2025-08-16 13:06:20:
Nice thing. As for objections or rather theoretical breakage, I think the way you did it is one of the best under these circumstances (without having namespace-based lazy-load facility). For instance one can rewrite `package require tcl::oo` to force initialisation, so could help for some theoretical case it is really needed (for example like below). And the possible breakage may be theoretic endless loop in TclOOInitModuleObjCmd, if some command gets injected, mocked-up etc, e. g. by something like this (red is lazy branch):
As for improvements, I'd later provide my attempt. > Looking for an extension that uses it ... I think, Itcl would be good candidate (v4.x is based on tclOO). apnadkarni added on 2025-08-16 13:42:17:
Thanks for taking a look. I will test the C API with Itcl, which triggered my memory that I think TDBC also uses it. I think it should be possible to protect the infinite recursion with the mockup as well. Will check. sebres added on 2025-08-16 14:30:26:
You're welcome. [6a997a7f19eaa6fa] provides several improvements:
As for mz previous mocked-up example, it'd work now, and would not enter endless loop anymore, but the mock-up becomes outdated (replaced by mentioned update after lazy load), see the new diff (red is lazy branch):
So as one can see, the mock-up is quasi removed here.
And the only solution would be to use Alternative would to create all this command properly (to point to correct handler). dkf added on 2025-08-17 00:22:40:
What is the slow part of TclOO's startup? It might be possible to make some parts lazy without having to tackle everything. (The entanglement between apnadkarni added on 2025-08-17 03:01:22:
@sebres, thanks for the patch. Will wait on comments from others before merging. apnadkarni added on 2025-08-17 03:06:49:
@dkf, the tclOOSetupScript takes up a little over 80% of TclOOInit. I was afraid of breaking up the init sequence in any form as I understand only a bit of code and changing the sequence would both add risk and complicate testing. Right now the sequence itself is more or less unchanged but takes place at a different point. Of course, this is not going to be merged without your blessing no matter other opinions :-) dkf added on 2025-08-17 17:07:29:
We can probably make the scripted part quite a bit more efficient. An initial (not very high effort) analysis indicates that making some parts be either implemented as C code or by lazy loading them should help.
dkf added on 2025-08-17 21:29:15:
I've made the Still need to do apnadkarni added on 2025-08-18 05:50:56:
@dkf, even if lazy loaded, it would be worthwhile to make the loading more efficient in any case with C implementation wherever possible. dkf added on 2025-08-18 08:59:30:
And that's Next, I need to re-evaluate what parts of the startup sequence are slow before making another move; this was just some low-hanging fruit. dkf added on 2025-08-18 12:16:25:
After the current changes, it looks like about half the cost is the apnadkarni added on 2025-08-18 13:27:18:
Created a branch apn-oo-lazy-init-sbmod-91 for 9.1. This includes trunk changes from dkf that fix the (bogus?) test failure reported in 8a13caa3 Also, a change such as this is probably best left for 9.1, not a 9.0 patch release so planning to close the original branch. dkf added on 2025-08-19 09:02:22:
I found another cheap win; using full slot syntax in the init script instead of directing things via the slot's unknown method handler. dkf added on 2025-08-19 14:52:35:
Slots are now almost completely done in C; the only bits still in the init script are setting the default operations and hiding the We now need to reevaluate how slow things are. They're probably still not fast enough... but I don't know how much of a loss things are now. dkf added on 2025-08-23 08:34:22:
I've moved a lot more of the cost into C and adjusted the code to avoid some of the slower paths in slotted definitions (getting a nice overall saving; the bits that remain are more awkward). I don't know if the changes make this RFE's code less useful or if we should still go ahead with it (and if they still technically apply); I'll investigate. dkf added on 2025-08-23 12:54:03:
I'm not convinced that it is a good idea to make apnadkarni added on 2025-08-24 03:07:35:
@dkf, the `info object` and `info class` were made lazy loaded not for performance but for compatibility. I wanted the messages for "info class,object" to be exactly the same as 9.0.2 irrespective of whether any oo calls that would lazy load oo had been made or not. Without loading them on demand, error messages were a little different causing couple of test case failures. If that is not important or there is some other way to achieve the same, that would be fine with me. /Ashok dkf added on 2025-08-31 10:55:07:
I'd love to see a rerun of the timings with the current TclOO is now implemented entirely in C except for the apnadkarni added on 2025-09-01 03:01:28:
The current main branch:
The lazy loading branch
The starting point was 1861 so much of the cost of oo init has been eliminated by the move to C which is great news! Not sure whether it is worth lazy loading to save an additional 10% of initialization cost until further investigation of where else the initialization might be optimized. On travel on and off for a few weeks so that might be a while. dkf added on 2025-09-01 08:45:17:
Thanks! (My testbed's using a little faster hardware, when it decides to turn on the speed-stepping.) You'll likely need to rethink how you lazily loading the classes, but I'm not sure how yet. Likely by postponing calling The main thing to beware of here is that there may be cases where a class (or object) is used in user code before calling it as a command, and before any call to apnadkarni added on 2026-03-11 05:59:44:
Closing as the additional gains with lazy init do not seem worthwhile after moving the oo init code to C. | ||||
