CollectionHelper#normalize_params is choking on ActionController::UploadedStringIO objects.

When trying to upload images, certain image types result in a parameter like this: #<actioncontroller::uploadedstringio:0x34c62d1622e0>

When RPM tries to process this action, it thinks that this object is of type "Enumerable" and tries to call ".first(20)" on it, which causes rails to throw this error:

undefined method `first' for #<actioncontroller::uploadedstringio:0x34c62d1622e0>

The stack trace is:

undefined method `first' for #<actioncontroller::uploadedstringio:0x34c62d1622e0>
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:19:in `normalize_params'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:15:in `normalize_params'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:14:in `each'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:14:in `normalize_params'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:15:in `normalize_params'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:14:in `each'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/collection_helper.rb:14:in `normalize_params'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/transaction_sampler.rb:260:in `set_transaction_info'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/transaction_sampler.rb:123:in `notice_transaction'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/rails/action_controller.rb:52:in `perform_action'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:41:in `trace_method_execution_with_scope'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/rails/action_controller.rb:47:in `perform_action'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:18:in `trace_method_execution_no_scope'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/rails/action_controller.rb:41:in `perform_action'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/base.rb:529:in `send'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/base.rb:529:in `process_without_filters'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/filters.rb:569:in `process_without_session_management_support'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/session_management.rb:130:in `sass_old_process'
/usr/local/lib/ruby/gems/1.8/gems/haml-2.0.2/lib/sass/plugin/rails.rb:19:in `process'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/base.rb:389:in `process'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:149:in `handle_request'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:107:in `dispatch'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:104:in `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:104:in `dispatch'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi'
/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.1.0/lib/action_controller/dispatcher.rb:35:in `dispatch_without_newrelic'
/home/gcafe/rails/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/dispatcher_instrumentation.rb:47:in `dispatch'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/rails.rb:76:in `process'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/rails.rb:74:in `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/rails.rb:74:in `process'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:159:in `process_client'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:158:in `each'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:158:in `process_client'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:285:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:285:in `initialize'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:285:in `new'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:285:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:268:in `initialize'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:268:in `new'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel.rb:268:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:282:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:281:in `each'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:281:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:128:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
/usr/local/bin/mongrel_rails:19:in `load'
/usr/local/bin/mongrel_rails:19</actioncontroller::uploadedstringio:0x34c62d1622e0></actioncontroller::uploadedstringio:0x34c62d1622e0></actioncontroller::uploadedstringio:0x34c62d1622e0>

I’m frustrated

Inappropriate?

Reply to this problem

3 people have this problem

I have this problem, too! Tell me when someone solves it.
The more people who report this problem, the more it gets noticed.

The company marked this problem solved.

Create a customer community for your own organization

Plans starting at $19/month

Remove Inappropriate?
Delete or Edit

onemorehour replied on December 04, 2008 19:50

Fixed it with this patch, for anyone who is interested:
diff -u collection_helper.rb.old collection_helper.rb.new --- collection_helper.rb.old 2008-12-04 14:42:03.000000000 -0500 +++ collection_helper.rb.new 2008-12-04 14:41:47.000000000 -0500 @@ -16,7 +16,11 @@ end new_params when Enumerable: - params.first(20).collect { | v | normalize_params(v)} + if params.respond_to? :first + params.first(20).collect { | v | normalize_params(v)} + else + params + end else truncate(flatten(params), 256) end

this solves the problem

Add a comment This solves the problem
EMPLOYEE
Remove Inappropriate?
Delete or Edit

Bill Kayser, Official Rep, replied on December 04, 2008 21:41
Thanks for the report. We will fix this in the next agent update.

My fix would be different. My only concern with your fix is that if params is a local enumerable type not part of Rails, then it will cause an error trying to serialize it and send it to the server.

Here's what I am thinking as a fix:
```
Index: collection_helper.rb

===================================================================

--- collection_helper.rb	(revision 5327)

+++ collection_helper.rb	(working copy)

@@ -16,7 +16,7 @@

       end

       new_params

       when Enumerable:

-      params.first(20).collect { | v | normalize_params(v)}

+      params.to_a.first(20).collect { | v | normalize_params(v)}

     else

       truncate(flatten(params), 256)

     end
```
What do you think?
this solves the problem

Add a comment This solves the problem Mark as a company solution
Remove Inappropriate?
Delete or Edit

onemorehour replied on December 06, 2008 16:37

Hi Bill,

Correct me if I'm wrong, but what you're proposing seems like a dangerous solution. In the case we're talking about, params is a single object of type ActionController::UploadedStringIO. I don't know what calling to_a on it will return, but there are only two possibilities that come to mind:

One is that it will just return a one-object array containing the object itself. If that's the case, this will cause an infinite loop in your code, since you're recursively calling normalize_params on the contents of the array!

The other possibility is that it somehow tries to split the object (which really represents a stream of data) into an array of pieces (bytes?). This seems like it would be unnecessarily slow and/or memory intensive.

this solves the problem

Add a comment This solves the problem
EMPLOYEE

Remove Inappropriate?
Delete or Edit

Bill Kayser, Official Rep, replied on December 07, 2008 00:50

That's a good point. UploadedStringIO is pretty much the same thing as StringIO in terms of it's Enumerable implementation. It appears this implementation basically returns an array of strings, one per line.

While I still think it would return an acceptable result which is the first 20 lines of the uploaded stringio object, I'm worried about converting the entire object to an array in case it's a very large string.

I'm inclined now to just iterate over the first 20 elements and capture them one at a time, instead of converting it to an array. It's much more verbose but seems more efficient memory wise.

My main concern is getting everything in that hash converted into objects in the Ruby Core library so they will get deserialized safely by our server. It doesn't matter much if we leave out information... this is only reference information stored with errors and transaction traces.

I may also treat StringIO as a special case, and only pass over the first 'line' (truncated if necessary).

Your thoughts?

this solves the problem

Add a comment This solves the problem Mark as a company solution
EMPLOYEE

Remove Inappropriate?
Delete or Edit

Bill Kayser, Official Rep, replied on December 10, 2008 01:40

We've released a new agent with a fix. The new agent is version 2.7.4 and has the version of the patch I described above which calls .to_a. The next release will have the version which does not call to_a.

I’m confident

this solves the problem

1 Comment Add a comment This solves the problem Mark as a company solution
Remove
René Vangsgaard commented on February 26, 2009 13:33

For reference: the mentioned fix also fixes another bug I have experienced:
Calling to_a on UploadedStringIO makes subsequent calls to UploadedStringIO.read return empty string.

The new implementation using UploadedStringIO.each do not empty the UploadedStringIO instance.