Tcl Library Source Code

View Ticket
Login
Ticket UUID: 785d2954d485669f6f9e0f24ffe3bb3e8deabcc0
Title: Improve C-implementation of JSON-parsing
Type: Patch Version: 1.17
Submitter: mi Created on: 2015-05-27 21:15:58
Subsystem: json Assigned To: aku
Priority: 5 Medium Severity: Minor
Status: Open Last Modified: 2015-06-04 12:04:23
Resolution: None Closed By: nobody
    Closed on:
Description:

The proposed improvement will:

  1. Speed-up parsing by using dictionaries, if the underlying Tcl offers them (is above 8.4).
  2. Use less memory by reusing string-objects for "null", "true", and "false -- the only string-literals in JSON.
  3. Speed-up parsing and most usage by treating numeric values as such by creating Int, Long, or WideInt Tcl-objects instead of strings.

The additions to the JSON test-suite are included as well. The complete patch is now part of the FreeBSD ports-collection.

User Comments: mi added on 2015-06-04 12:04:23:

Access and manipulation of O by some thread T' != T can cause crash and burn of the Tcl core.
I do not see, how this is possible -- as long as the general rules (respect of refCount) are obeyed. And where they aren't, the crash is likely with or without threads...

Could you describe a hypothetical scenario, where such a crash might occur because of the change I am proposing?

This is a property of the Tcl core itself.
Could you point at documentation or some other reference about this? I don't see any mention of threads in Tcl_NewObj(3)...

I find it hard to distinguish your above statement from something like "Tcl-objects can not be shared between threads," -- which is, of course, not true. If that's not, what you meant, could you explain the difference?


aku added on 2015-06-04 07:55:20:
Commit [fe05de3454].
- Thread-safe static Tcl_Obj*
- Stricter dict check
- Fixed bad data in the numbers.* test due to bad download of patch from my side.

This passes the json testsuite for me. Please conform for your environs.

> They aren't wedded to any particular Interpreter nor Thread. 

Incorrect. Any Tcl_Obj* O allocated (via Tcl_NewObj() and variants) is _implicitly_ (and automatically) tied to the thread T doing that allocation. Access and manipulation of O by some thread T' != T can cause crash and burn of the Tcl core. This is a property of the Tcl core itself.

aku added on 2015-05-28 05:44:09:
Created branch from the patch, for further review and editing.
See [<a href='timeline?r=tkt-785d2954d4-jsonc&unhide'>tkt-785d2954d4-jsonc</a>].

The condition for USE_DICT looks to be a bit loose as well, triggering for Tcl 7.5+, not just 8.5+. And there will be fools who will try.

mi added on 2015-05-27 22:17:40:

The constant objects are created and maintained in the static array inside the "staticobj" function. They aren't wedded to any particular Interpreter nor Thread.

They begin life with increased refcount and so will never be released. I think...

The only potential source of problem is when multiple threads try to parse JSON-text and all create a constant object of the same name. This will not cause a crash, but will leak some of the objects.

Perhaps, the object-creation inside staticobj-function should protected by a mutex, but I don't see a more dangerous problem than that.

Creating one such array of literals per thread would still be wasteful, though not as wasteful as the bazillion copies resulting from parsing even a moderately-sized JSON-blob. What do you think?


aku added on 2015-05-27 22:02:03:
First glance review ...

The string literal setup is broken with respected to multi-threaded Tcl core.
Tcl_Obj* are wedded to the thread T they are created in. Only T is allowed to 
to release it, *and* manage the refcount. While the former will never happen the latter (refcount changes) will, that is the point in having the literals shared.

While a regular threaded core will not complain, expect by possibly crashing unexpectedly, a mem-debug build of a threaded core will generate panic on access to any Tcl_Obj* from a thread outside of its origin thread.

That all said, this is fixable by making the array[] for the literals a member of a struct which will be allocated per-thread. We have examples of such things in the core itself.

Attachments: