Unread counts incorrect on dev.newsblur.com

  • 6
  • Problem
  • Updated 7 years ago
I had not noticed it until now, but looks like even though there are unread articles, the feed shows unread count of 0.

Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes

Posted 7 years ago

  • 6
Photo of dalmet

dalmet

  • 16 Posts
  • 0 Reply Likes
I'm still having these issues atm on www and iOS.
Photo of onkelpuck

onkelpuck

  • 34 Posts
  • 6 Reply Likes
Things look better over here today.

- I have had the same problems since signing up a couple of days ago

- as of today it looks better. NewsBlur and NetNewsWire (set-up for testing) appear to be in sync

It looks to me like you found the bug :-) I hope I am right....
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes
All of my feeds seem fine today. There was some leftover strangeness from yesterday, but all of the feeds / articles from last night forward are correct.

Thank you again!
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
Status quo here, no improvement at all. Pretty much every feed I click on is off (my first login since last night).
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
What's your username?
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Eureka! Found the bug. Turns out all of my meddling above was nice and all, but the real problem is slave lag.

My task servers are reading from MongoDB slaves, but they are writing to the master. Problem is, as an optimization, when those stories get written, I immediately read all stories from the past 2 weeks (from slave) and use them to count unread counts. I was missing all of the new stories, because the lag from writing to master and then waiting for it to propagate down to the slaves was too long. That also explains why this bug is so random and is now getting worse. My slave lag is going up!

The fix is to either fix the slave lag, or swap to reading from master for this one call. I could also perform a safe write the ensures things are written to a majority of slaves, but that won't fix the bug entirely. But if I swap to read from master, that's a huge hit to the same db that serves users, meaning the average load time will spike.

Not sure what I'll decide to do, but I'm going to investigate and perform some load tests to figure out the right solution. Good news is that this is without a doubt the cause of the bug.
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
That's great news, what a relief!

Please let us know when you think you've squashed it, hopefully soon.

Thanks for your work sorting this out.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
After a pause, the issue is back :-(
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
If you refresh the page, is the count correct? Are you on https?
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Note that I just made some major db upgrades last night, so I am fully expecting to see the fallout from that today.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Yes, refresh worked, but before it worked, it emptied by entire stream (no feeds listed at all). Had to close the browser and restart it and now the counts seem correct.

Not on https.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
The distinction of whether it counts correctly once you refresh the page is an important one.

If it does correct itself, that means it knew it wasn't correct and corrected itself as soon as it could. The bug here is that it didn't refresh on its own quickly enough. It updates itself when new stories come in and when you read stories. I made a change last week that allows that unread count to reflect stories that have come in since you started reading the feed. I think I'm going to move that back, since it makes the likelihood of there being new stories offsetting a count "incorrectly" is higher. Perhaps let you know that new stories have come in so you should refresh the feed would be best.

If it does not correct itself, then the problem, which I fixed last week, is that it thinks it is correct. There are no bugs left (that I know of) around forcing it to count itself and still coming up incorrect. But the more insidious bug is when it thinks it is correct but isn't. (Again, forcing a recount at this point would correct it). This correctness can be referred to as the dirty bit. It's actually `needs_unread_count` in the database. Point is, this kind of bug should immediately be fixed and is more of a real bug than an interface improvement.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Yes, it did correct the count upon refresh. Will see how deep the problem is. I have many feeds, so if it continues to be a problem, I should see it fairly quickly.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
An update - this incorrect count issue persists, although tab refresh resolves it.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
After all of the bugs that I've fixed, I'm no longer sure what this looks like. Are these stories in feeds that you've read in blurblogs as well? The unread count being lazily updated after reading a site's story in a blurblog is something I'm working on this week. Are the stories that are offsetting the count newer than 5 minutes? On your dashboard, in the "Welcome" module, do you see a small icon with a radar scanning back and forth? When you hover over it, there should be a tooltip that says "Real-time". If it's in red, it will say "Polling".
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
No, these are not stories in blurblogs. These are my feeds.

As for the stories, I usually go top-to-bottom, across feeds (with N keyboard shortcut). So I read the first 4-5 stories of this feed when I realized the unread count went to 0 despite me seeing older stories unread. So I am not sure I can answer that question about stories being new in the past 5 minutes.

I don't see a small radar icon in the Welcome module (dev.newsblur, FYI).
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Bad:



Good:

Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
I don't have the red bar, so I guess I have the "good" picture.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
What browser is this on? Those unread counts are largely maintained by the new real-time socket that you are connected to. I'm just wondering if this is Chrome or IE.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Oh ... IE9. :-P
Photo of volongoto

volongoto

  • 6 Posts
  • 0 Reply Likes
It looks like this is an old issue, but I started seeing this problem n dev.newsblur.com yesterday. I have changed the timezone sometime in the last 2-3 days. But I'm not sure f it's related or not.