Loading Profile...
Track bot hits API Limit for Direct Messages
I am running a bot that provides tracking functionality by using the Search API and sending direct messages. Now that there are about 100 people using it and getting messages, Ive noticed over the last few days that several messages haven't been getting sent. I understand the need for limiting casual users DMs to curb spammers, but my bot is legitamate.
I reported this issue on the development list (it exposes the contents of a random DM when using the json interface) and have filed a request through help.twitter.com (issue 320434) to have the limit removed or drastically increased. The rate_limit_status API call doesn't indicate DM limit status.
Any help is appreciated!
I reported this issue on the development list (it exposes the contents of a random DM when using the json interface) and have filed a request through help.twitter.com (issue 320434) to have the limit removed or drastically increased. The rate_limit_status API call doesn't indicate DM limit status.
Any help is appreciated!
I’m anxious
5
people have this problem
I have this problem, too!
Tell me when someone solves it.
The more people who report this problem, the more it gets noticed.
The more people who report this problem, the more it gets noticed.
The best solution from everyone
-
In case you haven't realized... Tracking is now working 100%
New things in the works... I've received my laptop and I'm going to be setting it up over the weekend (wiping Windows (ewww) and installing Foresight Linux) so that I can do some development work on it again.
Much thanks to Crystal and Alex for their help (and patience with my pestering them constantly!) in getting this working.
I’m thankful
4 people say
this solves the problem
-
Inappropriate?I hope this gets fixed soon, since I'm one of the ~100 people who use TweetTrak.
I’m glad you have this service and hope it is whitelisted.
-
Inappropriate?Yeah, Twitter, if you're listening, please make an exception for Ryan's service. He built an excellent product to fill a need a lot of Twitter users have.
I’m hopeful.
-
Inappropriate?Twitter, please do all you can to support RyanK in keeping TweetTrak running smoothly.
UntilTwitter is able to restore its own track function, TweetTrak will remain essential to many Twitter users - including you, crystal!
TweetTrak even helps fill some of the gap left by Twitter's non-delivery of SMS to countries outside Canada, India and the USA, by sending DMs which we can at least receive by email if we choose.
Thanks, RyanK, for all your work.
I’m hopeful too.
-
Inappropriate?Hi Ryan,
Try requesting white list status for your application here:
http://twitter.com/help/request_white...
If this doesn't work, please let us know.
Thanks,
Crystal
2 people say
this solves the problem
1 Comment Add a comment
This solves the problem
-
Thanks! Made the request and holding tight to hear back. -
Inappropriate?Appears as if its been running now for about 3+ hours... I didn't get any confirmation back that I'm whitelisted now, but I'm going to assume all is well. If it appears as if I'm hitting a limit again, I'll be back! =)
Thanks all for your help... Twitter users everywhere (that have heard of TweetTrak that is) are rejoicing!
I’m thankful
2 people say
this solves the problem
-
Inappropriate?TweetTrak is working again!
An e-mail notification of the first TweetTrak DM for days arrived just 10 min. ago (at 01:33 CEST = 23:33 UTC).
BTW (just checked): This time there was only 7min. delay between the tweet I was following and the TweetTrak DM alert. In the past it's sometimes been hours.
Thanks, crystal and RyanK!
I’m delighted
1 Comment Add a comment
This solves the problem
-
that delay is most likely from search.twitter.com being a little bit behind. I've seen it as far as 3-4 hour behind.. and I'm sure its been worse. Its normally fairly up to the moment.
Also, TweetTrak is gentle on twitter's services.. when it doesn't see a term for a while it checks less often (still at least once every 5 minutes with the first day... scales out to 6 hours after a few weeks... those values are subject to change) -
Inappropriate?OK, its been working in spurts of about 4 hours, then stops for about 2.
A little over 24 hours ago I received a notice that the lifted whitelisting should take effect within 24 hours, but DMs are no longer going through.
On the plus side, I used to get a robot page when I would try to send a DM, however now I get a nice error message telling me I've sent too many DMs:
Am I being too anxious and just need some more patience? or has something gone wrong? Are there different whitelists for DMs and for other API calls? Hopefully this takes care of itself, but if someone needs to do something, then hopefully this helps notify someone.
Sorry if i'm being annoying.. lol
Ryan
I’m anxious
1 Comment Add a comment
This solves the problem
-
worked fine for a few more hours but just hit the limit again. The email I received said that the limits no longer applied, so I presume that it somehow isn't in place yet. HALP! =)
-
Inappropriate?Hey Ryan, I'm looking into it today.2 Comments Add a comment
This solves the problem
-
Thanks, crystal! We need TweetTrak back!
-
Just a quick update... worked for about 3-4 hours today, and has been blocking DMs for the last 32 minutes.. so whitelisting seems to have still not taken effect.
-
-
Inappropriate?Still not seeing any change in direct messages going through... it works for a few hours... is down for 2-8 hours... then works again for a little bit.
Is there any update on this yet?
I’m frustrated
-
Inappropriate?Just a quick update... 21 hours ago the second whitelist request with the server IP address was approved... haven't been able to send any DMs for the last 13 hours.
Supposedly, within the next 3 hours this new request should take effect, lifting all limits. I'm going to be out for much of the day, but will check how things are going later.
I’m frustrated
-
Inappropriate?Sadly, I was told that a problem was identified in the whitelisting code, and it was expected to be fixed on Monday. Apparently, we are just waiting for the code to be moved to production.
Wednesday, I was told that the change has not yet made it to the production servers, but it should be moved there yesterday.
Yet, today (Thursday), I am still hitting these limits. Over two weeks since I made the initial request, and this is is 13 days old now here.
I'm really disappointed, and feel like I've been stringing along my users by promising services that are unavailable more often then not... And for something like tracking, if it doesn't work 100%, you can't really rely on it, and then it becomes pointless.
I'm not blaming Crystal or Alex... I just want this issue to be transparent so that people know what's going on and why they are receiving notices intermittently.
I’m frustrated
-
Inappropriate?RyanK, any word from the Twitter team? It's a great service (since their tracking isn't working) and I hope to see it back soon.
I’m anxious
1 Comment Add a comment
This solves the problem
-
Nope, my reply above yours sums it up. Got an email Monday saying it was a bug in the whitelisting code and it would be going live Monday. Tuesday evening I sent another report saying I'm still seeing it blocked. Wednesday around noon pacific time, I got a reply saying that the fix has not yet gone live, but it should go in that same day.
About an hour ago, I sent another message reporting that it still being blocked. I have no other information as to what the holdup may be (perhaps a large change is being made which prevents smaller changes from going in... perhaps their deployment is complicated... i really don't know)
As soon as I hear anything, I'll send a tweet from @TweetTrak and keep this page updated.
I'm just hoping this gets resolved soon! -
Inappropriate?Just got an email from Alex saying that there were some other deployments that delayed this one, but the current plan is that this change will go in tomorrow (Friday) so hopefully the weekend will be 'normal'.
I'll be keeping an eye on the tracking situation starting tomorrow afternoon and will tweet from @TweetTrak when its working... and hopefully won't have to tweet again about it being broken again.
I’m hopeful, but not optimistic
-
Inappropriate?OK, this morning, I see that twitter released many improvements to their interface. I have to presume that this is what the hold up was. That there wasn't an easy way to release the update to the whitelisting code without releasing the interface, and that wasn't ready.
At least I'm going with that.. easy enough to believe and makes me feel better about how everything went. After a 10 hour outage, about an hour ago it appears as if DMs started going out.
The test will be if they continue to go out all day. I have not yet heard back from anyone at twitter to confirm that the change is in and that they believe it is fixed. We shall see.
I’m hopeful, and beginning to become optimistic!
-
Inappropriate?That only lasted about 2 hours... Direct Messages from @TweetTrak are being blocked again. So, I guess the interface update did not bring along the updates to allow the Direct Message Whitelisting to actually work.
There is still hope that the changes necessary for this are still going to be made today. With any luck it will go in before the weekend is here and this won't have to wait until next week.
Still no word yet from the twitter-folk... but still giving them the benefit of the doubt that they are still planning on getting this working today.
I’m frustrated, and on the brink of losing hope and just giving up
-
Inappropriate?Don't give up hope, RyanK. 8-)
Fwiw, I didn't expect the UI update to include any fixes to the DM limit because the Twitter blogpost re: the new UI said as much. Here's an excerpt:
The scope of this project was limited to light-weight front-end work. We have whole other teams working on back-end changes and more fundamental functionality changes
I think we need a back-end change for the DM limit to be lifted
I’m optimistic
4 Comments Add a comment
This solves the problem
-
Thanks for the support!
Alex confirmed to me that the change should be rolling out and that it was delayed due to some larger high priority updates (the interface, i presume).
Also, the new interface includes a lot more AJAX calls, which means that the back end would need to be updated to support those calls, which means that the change to the code which checks the limits could also be updated.
That's why I was hopeful that the interface change included the backend changes. Seeing as how this was supposed to go in on Monday, and wasn't completed and that I was confirmed that it would go into today lead to my optimism.
The bad thing about being optimistic is that it gets your hopes up. When those hopes are continually dashed, it makes it hard to continue to be optimistic. Add in that when people are complaining about missing notifications, I have to blame it on Twitter and keep saying things like "it should be working tomorrow". It really makes me and my project look bad when I have to continue be proven wrong.
All that said, Twitter is a free service. I'm thankful for their efforts, I just wish that things actually went according to plan. I can't really work to enhance my project until this one issue is resolved, and its going on its third week now. =/
So yeah.. thanks for your support! I'm trying to not check in on it since I know it most likely isn't going to even have a chance at working until next week now... -
re: Ajax calls
For some reason, I was assuming that the backend was still the same REST-ful API interface, and that the AJAX calls were just using the existing "since_id" parameter to get newer updates. Don't know why I have this impression, though. Hm.
re: higher priority fixes
I'm assuming it has something to do with the fail whales that are mentioned in the Twitter Status blog.
re: users not getting notifications and complaining
I agree that it's very difficult to manage the expectations of your users when you are dependent on someone else doing something. It might be safer to not set any timing expectations at all...?
re: not checking on it until next week
This sounds like a good idea to me. Less stress! 8-) -
Well, the problem isn't specifically with the API, I get the errors on the website as well when I'm at the limit. So it is conceivable that it may have been in this update.
And yeah, I heard from Alex on the API team... they were busy with some performance issues.. I saw the whales today too for the first time in a long time. I don't blame them at all.. like I said, they are managing a large app... My issue is not pressing in the larger scheme of things.
As for expectations, I just pass along what I can and try to give a grain of salt withi it.. hopefully people take it that way.
Good news is.. Alex said he personally will see that its done on Monday.. So no checking in on that this weekend... and I'm passing that info along.
Thanks again for your support and your positive comments... =) Things should be much smoother after this.. -
re: Monday
Hurray! 8-) And thank *you* for pouring energy, talent and emotions into TweetTrak. -
Inappropriate?Ok, it appears as if things have been working for about 4-5 hours... I've been hesitant to report the good news.
You should be getting updates regularly now. My new laptop is in transit and I'll be addressing some of the issues (dupe tweets when they match multiple terms, ignoring friends that you follow already, and some more robustness/scalability/statistics gathering)
Please reply here, at http://www.getsatisfaction.com/TweetTrak or just send a tweet with @RyanK or @TweetTrak.. I'll see them all.. =)
I’m cautiously optimistic
-
Inappropriate?In case you haven't realized... Tracking is now working 100%
New things in the works... I've received my laptop and I'm going to be setting it up over the weekend (wiping Windows (ewww) and installing Foresight Linux) so that I can do some development work on it again.
Much thanks to Crystal and Alex for their help (and patience with my pestering them constantly!) in getting this working.
I’m thankful
4 people say
this solves the problem
2 Comments Add a comment
This solves the problem
-
I definitely noticed! And am thrilled! Thank you!
-
Oh, btw... is it correct to assume you're in the Twitter Dev Google Group and have seen Alex's announcement re: an impending change for JSON responses? It will take effect on Oct 25, 2008
EMPLOYEE
