Loading Profile...
Why is WebFrontend/Mongrel/Average Queue Time the single biggest consumer of time in my app?
Just installed and deployed and waited a couple minutes to get interesting data. In fact, the most interesting data I found is that Mongrel seems to be the most significant time consumer in my application?
Am I mis-analyzing the data?
Am I mis-analyzing the data?
I’m puzzled
1
person has this question
I have this question, too!
Tell me when someone answers.
The more people who ask this question, the more it gets noticed.
The more people who ask this question, the more it gets noticed.
Create a customer community for your own organization
Plans starting at $19/month
-
Inappropriate?Hi François,
What you're seeing is the amount of time HTTP requests are stuck in Mongrel before they enter your application. A little background on Mongrel - it's multithreaded and will create a new thread for each incoming HTTP request. Request threads then compete to enter your Rails app to be processed (Rails is single threaded by default). Mongrel happens to track when a request thread is created. RPM picks that up and creates a metric stream for the average amount of time all requests are stuck in Mongrel before your application gets them.
This brings up a great point - even if your Rails application is responding quickly, users may be seeing slow performance. The Mongrel queue is an important "upstream" system between the user and your application.
Back to the problem at hand...requests are stuck in the Mongrel queue. This can be cross checked with the Mongrel Queue Length metric that's available on the RPM "Controller" view. If that metric is consistently above 2 or 3 then it confirms what the Mongrel Average Queue Time metric shows. If this is the case, you need more Mongrels for your application. Based on the CPU and RAM metrics available in RPM, you either need to simply run more Mongrels on the same hardware, or get new hardware.
Hope this helps!
- Jim
I’m confident
2 Comments
Add a comment
This answers the question
-
While the time spent waiting in Mongrel queues is an important piece of data to report, putting it in the Time Consumption report this way seems a poor way to report it. The time thread A spends processing a request is doubly reported as the time thread B spends waiting on A to complete. Since the time waiting in the Mongrel queue is not part of the application code, it doesn't make sense to include it in this report. It throws off all the other numbers and makes it hard to see the smaller numbers.
Certainly, from the perspective of a request, the portion of time spent waiting in a Mongrel queue is useful to know. I'd like to see that broken out orthogonal to the report of where in the application thread time is spent. -
Hi Josh,
I completely agree. We started collecting this metric a little while ago and haven't found a good home for it in the UI yet. Until we do, if Mongrel queuing is a problem for your application then at least you have somewhere in RPM where this is visible.
If this number is so large that it dwarfs other metrics, then I would strongly encourage you to resolve this quickly. While Mongrel isn't part of your application code, it is part of your application.
- Jim
VP Engineering -
Inappropriate?I don't agree with the recommendation about adding more mongrels to resolve this WebFrontEnd problem. What we saw on our servers was low CPU (<5%) and response time for most of the day. And then at exactly 10pm the response time slow down over 20x and only a few mongrels had their queue length went up to 18 to 24 while other mongrels queue were at 0 and 1. This happened for only ten minutes and the site was back to normal after that. The time consumption chart shows WebFrontEnd/Mongrel/Average taking 800% of the time during this 10 minute window. I suspect something else is going on and not because lack of mongrels running.
I’m confused
-
Inappropriate?Hi HD,
It sounds like you had an "event" which caused a slow down in your application and a consequent increase in the time requests sat in the mongrel queue. I would agree that adding more mongrels in this case is not warranted. Any mongrel capacity review should only be done with long time windows. However, if requests spend a large % of their time in the mongrel queue consistently over several hours, then your mongrels are too busy. You either need more of them or make the existing ones run faster.
EMPLOYEE
