Unread counts incorrect on dev.newsblur.com

  • 6
  • Problem
  • Updated 7 years ago
I had not noticed it until now, but looks like even though there are unread articles, the feed shows unread count of 0.

Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes

Posted 8 years ago

  • 6
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Working on the unread counts issue today. Did you share an article from this feed, or alternatively, did you read a story shared from TechCrunch from somebody else?
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
I did not share anything from this feed (or read anything from somebody else's shares) today, but I did share back on the 13th. Of course, the problem was only noticed today, could have existed then too.
Photo of Greg Lindahl

Greg Lindahl

  • 4 Posts
  • 0 Reply Likes
I never share or read shared stuff. What I'm seeing is that the unread count is always too small, and when I 'n' my way through the articles, I go to the next feed before I've read everything. Then, a few seconds to minutes later, the correct count for the feed I was formerly reading appears.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Not sure if you'll be able to answer this, but did the feed that had its count updated a few seconds later get new stories?

As for the unread counts issues that I'm working on, they all have to do with shared story feeds and reading the same story that occurs in multiple feeds. A bit edge case for right now. So if there are issues around incorrect unread counts for feeds, tell me more. As you may know, I never experience this, and I think it occurs rarely enough that it's not pervasive, but I want to figure out anything I can about those edge cases.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
The biggest issue is that by the time I have a chance to look into a buggy unread count, the moment has passed and the feed probably has a new story which makes the unread count update itself. Counting stories isn't the problem, it's knowing when to count stories. If you Insta-fetch a feed, it will update your unread count.

What's happening here is that a story probably did come in while you were reading the feed, but because you were on the feed, I don't update your unread count, since I'm not sliding the story in. (Believe it or not, maybe 2 years ago, I did slide in the story as it came in. There's vestigial code laying around that in all likelihood still works fine.)

When you finish reading the feed and click to another, after the next refresh (every 60 seconds if you're a premium, but less often if you are on real-time updates, which is 99.5% of folks), the feed you were reading gets updated with the right count. And opening the site doesn't correct the count either, but it could potentially fix these problems. Unread count wrong? Just open the site. That would at least ease the pain a little bit, since the count goes away immediately. The problem with that is that it would increase load times from 120ms to 240ms on average for maybe 10% of all feed openings. Wouldn't be terrible. I'll see about getting that in today.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Tricky, eh? I know you probably don't want to hear this question, but how doees Google Reader handle such a thing? They have supported Pubsubhubhub for a while too. :-(
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
I think I am having the same problem and it's been happening roughly over the past week. I was hoping it would go away, but it's getting worse to the point where the site is practically unusable.

Basically I started to wonder why some feeds which update frequently were not displaying any new articles recently. Then, one of two things happens when I click on the suspicious feed.

1) I discover that there are new articles which are already marked as read, which I had never been aware of. By that I mean I never saw or skipped them. They were essentially received but ignored by Newsblur.

2) I'll click on the feed and see several new unread articles in bold, despite the side panel not listing anything new to read. So I only discovered them when I checked to see if Newsblur was missing something.

This is a pretty serious issue, because if I have to hunt thru all of my various feeds and folders to find articles to read (which may or may not be marked as read or unread, no less) it kind of makes the site useless. It's frustrating.

Regarding sharing articles, I do this occasionally but I am pretty sure it is happening across the board — if not every feed it is certainly having problems with feeds I never share articles from. Also, 'insta-fetch' doesn't make a difference. If Newsblur missed ever marking something as unread and I insta-fetch to see what might be new, nothing happens.

I have a premium account, too, so naturally I expect not to have to insta-fetch too often (although it frequently seems necessary with less popular sites).

These problems seem to be occurring regardless of whether I am actively using the website or away from the computer for a while. For example, there's a site which updates every morning before I wake up. The past few days, it never reports anything unread. So using the site seems to have nothing to do with the articles being skipped.

I hope this information helps you squash whatever bugs is causing this chaos. :)

Edit: Oh one other thing, I use www.newsblur.com rather than dev.newsblur.com, but it sounds like the same problem so this seemed like the right place to post.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
If you're premium, sites should be updating every hour at the very minimum, unless the site is real-time, in which case it is roughly 10-20 times less often than normal. So if a popular feed is real-time but would've been fetched every 4 minutes, it's now being fetched every 40-80 minutes.

Can you confirm that most of the feeds giving issues are in real-time? Right-click the feed and it'll tell you if it is in real-time.

As for looking to see what Google Reader does, I'm all for it. I never really used it, but I'm not opposed to seeing how they solve these issues. To be honest, I keep my influence focused on products that are not direct competitors. I think keyboard shortcuts are (mostly) the same as Reader.

So I have a frequently updated site open in Reader and I'll be watching it to see how the counts are updated. I can easily change that behavior.

Although it seems like the unread counts are just not being counted correctly, which is a big problem. I can wire up a button that forces a recount of all of your feeds. Perhaps I should have that run automatically every hour you're on the site. But that's just a big band-aid to the underlying problem. These unread counts *should* be accurate, and in the case of a real-time feed, instantaneously correct.
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
I am not sure I understand how to tell if the site is real time or not?

However, when they update it seems that either the unread count is ignored (thus I don't know where to find unread articles), inaccurate (it might say 3 unread but there are 7, for example), or the articles themselves are marked as read before I ever see them (less common, but the most frustrating).

It seems to be happening across the board, and I have 52 sites right now—some popular, some not. If it's not every feed, it's most of them.

Also FWIW, I've been using Newsblur for several months now (since Google Reader redesigned) and this is certainly a brand new problem for me... so I'd imagine something new/different in the past week must have caused the problem?
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
The only reason I mentioned GReader is because I had not noticed such an issue there. Once I progress through reading articles in a feed, I would not see mistakenly marked read/unread status. I would see unread count go up for "real time"/pubsubhubhub feeds, but very rarely (if at all) would I see an article marked as read, become unread. Or worse, have a feed show 0 unread but in fact have some article actually not read.

Perhaps they have the capacity/scale/inclination to count in real time vs how newsblur does it?
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
Is there any update on fixing this problem?

I can report that it continues to happen. It seems like it's not as bad, but still quite enough to drive me crazy.

It's still both problems, too. Some feeds do not register new posts and more often the post count is too low (this is less annoying since at least I know there is something new to read).
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
I just discovered this happens to me on www, so something will eventually be fixed. But I'm not sure why it's happening. I have an idea and it would explain all the irregularities. The task servers are suffering from some severe clock drift. They are usually behind anywhere from a few seconds to a couple minutes. It's highly likely that incorrect times are resulting in incorrect counts.

Are your unread counts off by only 1? This issue is specific to normal feeds. Social feeds (blurblogs) do not suffer from this problem, so if those unread counts are off, it's 99% likely that there is a story being shared in multiple blurblogs and you read it in one and the others just haven't been updated with the correct count. This is a known bug and the fix isn't so easy.
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes
For me, the feed counts are off by 1 to several.

I've been having a few issues with the feeds that seem to be related to one another.

Sometimes I will get new articles in a feed that come in marked as read when I haven't read them yet.

Sometimes I get articles re-marked as unread after i have read them.

I have a few feeds that don't seem to tell me when there are new unread articles, but if i check the feed there will be several unread.

I have a couple of feeds where the unread count is just off.

Let me know if you want me to expand on any of those. I haven't quite picked up a pattern yet, but i think sometimes when the articles come in as read, they eventually wind up getting marked as unread. That doesn't happen all the time though.
Photo of kris

kris

  • 2 Posts
  • 0 Reply Likes
I'm experiencing the same behavior as Ryan, pop race, and the others on 'www'. At first I thought it was a result of alternately checking my feeds from the iOS app and the site, but I haven't used the iOS app all day, and I'm still seeing the behavior.

Some of the feeds where I notice this behavior most often are:

daringfireball.net
prollyisnotprobably.com
urbanvelo.org

I don't know how to tell if those sites are real-time.

I've never used sharing on NewsBlur.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Today dev.newsblur has been massively off. So many feed have unread stories, but feed count on the feed and the parent folder is unavailable :-(
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Yes, today it has been unavailable as opposed to showing a number which is incorrect.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
I just pushed out a fix for what I hope is the issue. It seems that some feeds were timing out during the "mark-feed-as-dirty" process, which is immediately before the count-unreads process. I separated the two, forcing the subscriptions to at least be marked dirty so each user can force an update (which should happen immediately).

I noticed a rather substantial uptick in incorrect counts as well. Note that NewsBlur is starting to deal with a much larger workload, so some issues that you only see at scale are starting to crop up.

Here's the commit: https://github.com/samuelclay/NewsBlu...
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
Yes, understand. Also, note that I am only referring to dev.newsblur. fwiw.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
The backend is the same. Are you saying it works on www but not on dev? It's nearly guaranteed that it's the same problem. Anyway, I put out a fix and have a background job running through every single subscribed feed to update it to the correct counts, so if you see any incorrect unread counts, shout here.
Photo of Romit Mehta

Romit Mehta

  • 67 Posts
  • 0 Reply Likes
No, I only use dev.newsblur ... just wanted to clarify, in case it helped.
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes
It looks like i'm still having the same trouble.

One example i have is for deadspin.com. I had everything marked as read about 1.5 - 2 hours ago, there are now 4 new articles that are actually new which are marked unread, but it also marked the prior 9 articles as unread. The count on the main feed still show there are no new articles.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Yeah, that sounds exactly like a date bug. Do you remember where you left off with deadspin? How many unread stories did you have? I'm hoping 0 (including red, yellow, and green), because that means the unread count was moved to a story. 9 stories... would you say that covers about 5-8 hours? Way less? Way more? It might be a timezone bug, but there's no way that could manifest itself just now after all this time.
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes


Here's a screenshot. It's showing 6 unread now in the total for the feed.

Everything below the "read" one (5th from the top) was marked as read earlier in the day and the feed showed no unreads. The "how to make a pipe" one i marked read at about 8 PM or so.
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes
I dunno, i'm not too worried about it. Maybe there was some leftover weirdness after you applied that fix earlier. I'll check on it tomorrow and let you know.
Photo of Samuel Clay

Samuel Clay, Official Rep

  • 6514 Posts
  • 1474 Reply Likes
Bingo, I think I found it. I have 4 feed fetchers, one of them I added about 2-3 weeks ago (uhh, maybe 4 weeks now?). The newest one I added is using a newer version of Django, which has this weird timezone support. It's highly likely that it is setting the timezone incorrectly, and since you read all of the stories, the mark_as_read bar has been moved to a position about 5-8 hours off from where it should be.

I just fixed this, so please let me know if this happens again. Also, to make matters worse, it's hard to tell if this when this happens again it's fallout from this bug or the same bug. To address that, I'm going to run a counting script, which takes 2-3 hours to run.
Photo of Ryan

Ryan

  • 6 Posts
  • 0 Reply Likes
That timeframe fits in pretty well with about when first noticed the unread counts were going slightly wonky.

I'll keep you updated. Thanks for all of your help!
Photo of poprace

poprace

  • 9 Posts
  • 0 Reply Likes
The problem is still happening here. I just signed on to Newsblur and I clicked a feed with '0 new stories' because I /knew/ something must have been published tonight, sure enough there were 2 unreads.

The unread counts are off by 1 sometimes, but usually more, sometimes even 10+ with an active feed.

Ryan seems to be describing the exact problems I am having. Furthermore, I neglected to mention but I have also suspected that stories I previously read were being marked as unread at some point. It seems to happen in particular in one feed which only posts around 4-8 articles a day. I don't think it's related to marking as read or even time zones, however, because sometimes I'll see posts from the previous day as unread. I've used this RSS since I signed up for Newsblur and it never had any such problem before.

FWIW, I almost always clear this entire folder down to 0 unreads before bed and the problems still happen first time I sign in the next day.

I always use WWW.

I never use Newsblur on my phone -- OT: I wish there was an Android app as good as gReader for Newsblur, I bought Blar but it's Blah... wish I had my dollar back :P.

Anyway, I will try to get my unread count down to close to 0 as I can tonight and I'll report back tomorrow.
Photo of jzsimon

jzsimon

  • 38 Posts
  • 2 Reply Likes
This sums up the problems I'm having also. My username is jzsimon.