Debugging multipart/form-data form requests with Ruby on Rails

I have a rails app which can accept POST API requests using multipart/form-data. We had some trouble however with one client trying to connect, and the POST arguments simply not appearing rails’ params hash.

Here are my steps to debug this issue:

In the application controller, I created a before_filter to print the following debug info:

    puts "***"
    puts params.inspect
    puts request.body.string
    puts "***"

This will print out both all the params ruby parses, and also the request body (to help see what data the app received, and compare it to what was parsed).

I can test the API with the following CURL command, and it works (by works I mean that params[:example] contains the string test.

curl -X POST -F example=test http://localhost:3000/test

My debug text above prints:

***
{"example"=>"test", "controller"=>"users", "action"=>"syndication_tokens", "format"=>"json"}
------------------------------50c34e88eb11
Content-Disposition: form-data; name="example"

test
------------------------------50c34e88eb11--
***

However, the clients don’t use curl. They have to generate these requests themselves. While I’ve proved that the API works – how can I test the client request using curl (thus removing bugs in the client from the equation)? I’ve looked at RFC2046, and the body above is a decent example, and have an idea of how these messages were formatted. My first attempt was to create a text file named test.txt with the following data:

------------------------------4ebf00fbcf09
Content-Disposition: form-data; name="example"

test
------------------------------4ebf00fbcf09--

And call CURL like so:

curl -X POST -H "Content-Type: multipart/form-data; boundary=----------------------------4ebf00fbcf09" -d @test.txt http://localhost:3000/test

However, this does not work in Rails.

Here is what is printed:

***
{"controller"=>"users", "action"=>"syndication_tokens", "format"=>"json"}
------------------------------4ebf00fbcf09Content-Disposition: form-data; name="example"test------------------------------4ebf00fbcf09--
***

It looks like the line breaks are being lost. Considering I’m quite sure that the format of the multipart/form-data message is correct, I wonder if that’s why the data is not being parsed correctly.

There is a useful debug method from the CURL end --trace-ascii output.txt which lets you see what CURL is sending. Lets take a look. I’m just going to show the Send data output. Firstly with the -F approach:

0097: Host: localhost:3000
00ad: Accept: */*
00ba: Content-Length: 146
00cf: Expect: 100-continue
00e5: Content-Type: multipart/form-data; boundary=--------------------
0125: --------2752933cd6dd
013b: 
== Info: Done waiting for 100-continue
=> Send data, 146 bytes (0x92)
0000: ------------------------------2752933cd6dd
002c: Content-Disposition: form-data; name="example"
005c: 
005e: test
0064: ------------------------------2752933cd6dd--

And then the -d @test.txt approach:

0097: Host: localhost:3000
00ad: Accept: */*
00ba: Content-Type: multipart/form-data; boundary=--------------------
00fa: --------4ebf00fbcf09
0110: Content-Length: 136
0125: 
=> Send data, 136 bytes (0x88)
0000: ------------------------------4ebf00fbcf09Content-Disposition: f
0040: orm-data; name="example"test------------------------------4ebf00
0080: fbcf09--
== Info: upload completely sent off: 136 out of 136 bytes

The line breaks are missing from this data as well. Definitely looks like the issue.

From the RFC, I’ve noted that they use CRLF as the line breaks. So I open my test.txt file in an editor which I know and trust for line-break issues (jEdit, instructions here), and convert it to \r\n. However, the issue is the same.

It turns out, that curl will actually munge your file when using -d @test.txt notation. So the solution to this issue is to parse in the text as binary data using --data-binary

Thus, the final answer is: for this to work the HTTP body MUST use CRLF line endings. And, to test this with curl, you MUST use --data-binary instead of plain -d. The final command (where the contents of test.txt are listed above, and the line endings are converted to CRLF) is:

curl -X POST -H "Content-Type: multipart/form-data; boundary=----------------------------4ebf00fbcf09" --data-binary @test.txt http://localhost:3000/test

If you want a one-liner that avoids temporary files, then you can do something like:

curl -X POST -H "Content-Type: multipart/form-data; boundary=----------------------------4ebf00fbcf09" -d $'------------------------------4ebf00fbcf09\r\nContent-Disposition: form-data; name="example"\r\n\r\ntest\r\n------------------------------4ebf00fbcf09--\r\n' http://localhost:3000/test

Regarding the other clients – the important thing when generating the multi-part requests is to use \r\n line endings. I tried variants of test.txt with other line endings, and none of them were parsed correctly by rails (which is correct, as the RFC does call for CRLF).


One comment on “Debugging multipart/form-data form requests with Ruby on Rails

  1. Finally i quit my regular job, now i earn decent money online you should try too, just search in google – bluehand
    roulette system

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>