Tk Library Source Code

View Ticket
Login
Ticket UUID: 1254934
Title: mime - Extra line feed removed from output
Type: Patch Version: None
Submitter: erl Created on: 2005-08-09 12:23:30
Subsystem: mime Assigned To: patthoyts
Priority: 7 High Severity:
Status: Closed Last Modified: 2006-10-03 23:30:27
Resolution: Fixed Closed By: andreas_kupries
    Closed on: 2006-10-03 16:30:27
Description:
The following patch fixed a problem with extra line
feeds in mime attachments in code to upload binary
files to a web page. I imagine that the extra line feed
is not noticed with text files.

A line feed is output before the border, making this
line-feed unneccessary. 

This problem is mentioned on http://wiki.tcl.tk/13675

I have only tested the fix with my code - it should be
tested to verify that it does not cause problems in
other situations.

The patch file is generated with cvs diff -u, I hope
that's ok (I wasn't sure what the standard way of doing
this was.
User Comments: andreas_kupries added on 2006-10-03 23:30:27:
Logged In: YES 
user_id=75003

Committed my patch.

andreas_kupries added on 2006-10-03 04:36:52:
Logged In: YES 
user_id=75003

Please try the updated patch.

andreas_kupries added on 2006-10-03 04:36:24:

File Added - 196265: mime.patch.v2

andreas_kupries added on 2006-10-03 04:19:51:
Logged In: YES 
user_id=75003


Ah, slogging through the relevant RFC's showed the solution.
According to RFC 2046 the \r\n before the boundary itself
belongs to the boundary, not the body, so changing the
'parts' branch is out. It has to be 'strings' and 'file'.

I will make a new patch and upload it here.

andreas_kupries added on 2006-10-03 03:54:44:
Logged In: YES 
user_id=75003

The command is 'buildmessageaux'. It has a 3-way switch for
file, strings, and parts. The patch modified the 'file'
branch. Modifying the 'string' branch provides the desired
result. Given the structural similarty the fix to the file
branch is likely right as well, just not used in the example.

An alternate fix might be to change only the 'parts' branch,
as that is where the \r\n in fron of the separator line is
added.

This depends however on what a plain part would look like,
outside of a composition. Gut feeling tells me that making
the change in parts is wrong.

andreas_kupries added on 2006-10-03 03:47:21:
Logged In: YES 
user_id=75003

Note: Whatever we do to buildmessage and helpers we will
likely have to replicate for copymessage and helpers as
well. 'build...' is for getting a string, 'copy...
' writes directly to a channel. Structurally they should
work identically.

andreas_kupries added on 2006-10-03 03:45:00:
Logged In: YES 
user_id=75003

Tried out the patch and have to report that the provided
patch does NOT solve the problem. The path in question in
question is actually not used at all by the demo ...

Now trying to find the code paths actually used.

andreas_kupries added on 2006-10-03 03:36:15:
Logged In: YES 
user_id=75003

Cross-reference: See bug report
http://sourceforge.net/tracker/index.php?func=detail&aid=1213527&group_id=12883&atid=112883

andreas_kupries added on 2006-10-03 03:26:33:
Logged In: YES 
user_id=75003

Running the tet script I see the webserver reporting an
11-byte 'hello world' as 13 byte. Adding more logging of
values to the script I find:

The part generated in 'form-data-add_binary' seems to be ok.
It has \r\n at the end, this however might be very necessary
to put the separator line later closing the part on its own
line.

However the message then composed from all the parts, in
'form-data-compose' has an additional \r\n before the
separator line.

There seems to be a disagreement between parts and their
umbrella which of them has to add the \r\n for the separator
line to the whole message. Now apparently both do it.

It is unclear if this happens only for encoding binary.
This has to be tested.

I wonder if the problem is that the binary part is not
generated with some 'size' information in the headers.

erl added on 2006-01-30 23:46:02:
Logged In: YES 
user_id=116933

Does anyone have the time to look at the testcase (if you
have a PHP5 web server, then it shouldn't be too hard)?

As I believe this is a bug in the Tcl MIME handling, that I
believe I fixed in August, if would be nice if someone could
take the time to hopefully approve it and get it checked in
(or comment on any objections).

erl added on 2005-10-21 17:22:15:

File Added - 153318: testmd5.php

erl added on 2005-10-21 17:18:00:

File Added - 153317: testMIME.tcl

erl added on 2005-10-21 17:16:50:
Logged In: YES 
user_id=116933

Ok, here is a demonstration of the problem.

This Tcl program (testMIME.tcl) uploads a file
(transferFile.txt) to a web server (currently connects to
one of my servers, I will also post the receiving PHP script
so you can put it on the server of your choice).

The program will print the size of the file, and then show
the response from the web server. In my case I created a
text file with the word 'Hello'; the Tcl script reported a
file length of 5, and the HTTP server a file length of 7.

You will have to create the transferFile.txt.

I believe my patch fixes this discrepancy.

FWIW, I wish Tcl's http class had a method for doing this.
The code in testMIME is taken from examples on the Tcl Wiki.

erl added on 2005-09-29 14:28:10:
Logged In: YES 
user_id=116933

The test case is a bit complicated - it involves setting up
a web server script (I use PHP) to receive an uploaded file,
and calculate a checksum which will differ from the sent file.

The Tcl code is fairly cumbersome to handle HTTP headers and
what not.

I need some time to extract the relevant functions from my
code and construct a nice, minimal test case, maybe with a
PHP script as the client. I will be revisiting the upload
part of my project soon, I'll get back to you with the test
case. 

Thank you for your attention.

andreas_kupries added on 2005-09-27 07:24:53:
Logged In: YES 
user_id=75003

Erland, please provide a test case or example of the bug,
and how it is fixed by you patch.

patthoyts added on 2005-09-05 18:18:35:
Logged In: YES 
user_id=202636

It doesn't seem to me to make any difference to the output.
I tried to create a test for this condition and plumped for
test mime-9.0 {bug 1254934} {
  list [catch {
     set toka [mime::initialize -canonical text/plain
-string testing]
     set tokb [mime::initialize -canonical
application/octet-stream \
           -string [string repeat \0 160]]
     set tok [mime::initialize -canonical multipart/related \
          -parts [list $tok1 $tokb]]
     set r [mime::buildmessage $tok]
     mime::finalize $tok -subordinates -all
     set r
  } res] $res
} {0 {}}

Checking the output with and withouth these "\r\n" appended
makes no difference to the buildmessage output.

Can we get some sort of test case that shows the problem?

erl added on 2005-08-09 19:23:31:

File Added - 144966: mime.patch

Attachments: