| Ticket UUID: | d6e9b4db409b4800b2b01ba40d86168b8d0636be | |||
| Title: | Image format SVG: memory overflow on big files | |||
| Type: | Bug | Version: | trunk | |
| Submitter: | oehhar | Created on: | 2020-08-25 16:06:11 | |
| Subsystem: | 41. Photo Images | Assigned To: | nobody | |
| Priority: | 5 Medium | Severity: | Minor | |
| Status: | Closed | Last Modified: | 2020-12-09 12:18:52 | |
| Resolution: | Fixed | Closed By: | oehhar | |
| Closed on: | 2020-12-09 12:18:52 | |||
| Description: |
Thanks to Paul Obermeier, the following bugs were observed and fixed:
Checkin [79583944] within branch "tip-529-image-metadata" solves both issues. The following line was at the initial version of the ticket but is a false alarm
| |||
| User Comments: |
oehhar added on 2020-12-09 12:18:52:
Thanks, Francois, for the work. A question to Paul Obermeier if this alternate solution is ok was not answered within 2 weeks. The alternate Androwish solution is now merged with commit [44d8b4d9]. Thank you all, Harald fvogel added on 2020-11-21 13:10:17: I have digged into the code and have understood what's happening and why the error message is unexpected (at first sight) in imgSVGnano-5.2. In fact this error message is correct. Since the format of the data was not given, each stringMatchProc is tried in turn in MatchStringFormat(), but none of them returns without error. No known image format is recognized. The default format is then tried, and returns the error from its parser. All this is in fact what is expected. To return a correct and meaningful error message in the test, I have specified -format svg in the test and everything is fine then. See [b625f93d|test imgSVGnano-5.2]. fvogel added on 2020-11-21 09:53:31: I have added a non-regression test imgSVGnano-5.2. The important thing this test is ckecking is that image data not containing <svg and > now return an error upon reading as an svg image. Note that the expected result of the test matches the current output but is not adequate. The returned error text should be improved. oehhar added on 2020-11-16 13:31:15: The alternate solution from Androwish is now available for review in the branch [bug-d6e9b4db_svg-match-big-file-from-androwish] starting with commit [f4d783e0]. oehhar added on 2020-11-16 09:09:14: Francois, thank you for your contribution. So, my next steps are:
I see little possibility, that the upstream project accepts this patch. The project is end of life and only security patches are included. I am personally very happy with tksvg. Thank you, Harald fvogel added on 2020-11-14 21:53:36: I was prompted on the Tcl Core list as follows: may I ask your opinion on the patch by Paul for bug "Image format SVG: memory overflow on big files" ? Therefore here is my brain dump about this (not sure I understand everything! please bear with me...): The bottom line is that writing a second svg parser looks like an unexpected idea when there is already one in the codebase. The fact the existing one loads the entire file before basic analysis regarding svg-correct format could (should?) be considered a bug of the existing parser rather than a justification for writing a new parser aside of the existing one. The fact nanosvg is coming from upstream then would imply that:
So my opinion is that Christian Werner's suggestion should go in our Tk code, and the parser bug be fixed upstream. Paul Obermeier's proposal could be posted as a pull request at nanoSVG after some reformatting and factorization with existing code. oehhar added on 2020-09-14 12:17:00: Thank you, Francois, for the post. The issue is as follows: image create photo i1 -file big.zip will fail with big performance impact and a tcl memory error on bin zip files. The use case is the image browser application created by Paul, where whole directory trees are presented to the match function. The issue is:
The proposed solution in branch [bug-d6e9b4db_svg-match-big-file] authored by Paul is:
I hope, it is clrearer now. Thank you, Harald fvogel added on 2020-09-14 11:58:38: To be completely honest, I'm lost about what is the problem, what is the false alarm, what is the fix and so on. Deferring to those who understand the situation is my best bet, sorry. oehhar added on 2020-09-14 10:13:25: It is probably bad style to change the ticket title and text. But the false alarm about "circles1.svg" was removed. The main point is to not examine big (binary) files until the end. The cherrypicked version of the patch is now in branch [bug-d6e9b4db_svg-match-big-file] starting with checkin [9afe0b7d] (based on trunk). I am in contact with Paul to find an eventual test for the patch. Otherwise, I would just merge it, as it is clearly a regession and a solution. Any comments ? Thank you, Harald oehhar added on 2020-09-14 06:54:58: Added a test case in trunk [a40ea6ef] for an image without any of the attributes "width", "height" or "viewbox". This test does not fail in tk trunk but fails in tksvg 0.3. So, tk trunk is not vulnerable to the given issue. It already passed in release tk 8.3a3. Thank you, Harald oehhar added on 2020-09-13 11:06:48: I want to summarize some private mails with Paul and Christian. Why is "circles1.svg" not recognized by nanosvg ? Paul: As far as I remeber, the reason is that none of the attributes "width", "height" or "viewbox" is defined. His new size parser handles this situation. ---- Is there an advantage of the solution by Christian or Paul: Christian: I was affraid of the fact that there are now two XML parseres in play. So I just limmitted the file size. It would be ideal to have a pre-scanmode in nanosvg which only outputs the svg yes/no, height and width. But this would imply a big change. Paul: I did not want to touch the nansvg code. So I wrote an own parser. A no-go of the nanosvg parser is, that there is no stop in case of binary data. It was not created for the task to detect svg format. ---- My former posting was wrong, that the patch should go to trunk, not core-8-6-branch. oehhar added on 2020-09-10 16:39:56: Here is a test with the image "circle1.svg" with tcl8.6.10: % image create photo i1 -file circles1.svg couldn't recognize data in image file "circles1.svg" The same test with current tk branch [tip-529-image-metadata] can successfully load and display the image. % image create photo i1 -file circles1.svg i1 % pack [label .l -image i1] Note that Christian Werner has another solution for the big data issue and eventually also for circles1.svg: http://www.androwish.org/index.html/vinfo/a9be9dd0259e47f0?diff=1 I will try to backport the work of Paul to core-8.6-branch. I think, this is sensible work. Maybe, Christian has also an advice on this ? Thank you, Harald | |||
Home
Timeline
Branches
Tags
Forum
Tickets
Download
Wiki