New scrobbles database

  • 63
  • Announcement
  • Updated 2 weeks ago
  • (Edited)
Hi everyone,

We are beginning to roll out a new scrobbles database. This changes how we store your scrobbles - a big change for us, while to you most things will look the same :)

You will get the following improvements from this:

  • Support for corrections: You can enable and disable corrections properly, with your corrected or uncorrected scrobbles matching in all places

  • Album artists are correctly shown in album views (e.g. the album charts, instead of track artists)

  • Imported plays (e.g. from iTunes, when you first joined) are shown and counted correctly in all views

Please note, as we're gradually rolling out the update, not all of you can see these changes yet.

What you will see:

Your overall scrobble counts should always remain the same.

As a result of the above mentioned improvements, you might see changes to your artist, album and track counts, as well as to scrobble counts of artists, albums and tracks:

  • For example, if you've disabled corrections, your artist, album and track counts usually will go up, and otherwise corrected items will be split up in your charts.

  • Your Loved Tracks now properly support corrections, too. You might see that your Loved Tracks count has decreased when corrections are enabled, as duplicate versions of the same track will have been merged.

  • Where you've previously scrobbled the same album with the same album artist a number of times, you probably would have had these plays split up in your album charts, with the track artists being shown instead. This will now show one album entry, with the album artist as artist. When this happens, you will see a decreased albums count in your library.

  • This also fixes a problem where you could have seen the same album twice in your album charts, with different play counts.

What to do if you notice anything odd:

  • If artists (albums, tracks) have suddenly been split up in your library, check if you have corrections disabled - and if necessary, enable them.

  • If the overall scrobbles count in your library has changed - check if you have imported plays from your media player (when you joined; feature available before 2015), these are counted correctly now. The overall scrobbles counts on your profile and in your library should now match.

  • If you notice any slowness in your library, charts or scrobblegraphs not loading - do let us know.

  • If you notice any scrobbles missing or too many - check if any of the above applies, and if not, do let us know.

  • If you have any trouble deleting an artist, album or track from your library - do let us know.

Did you know scrobbling is already turning 16 years old next week? It's been a big journey and our new scrobbles database sets us up for many more years to come. Thanks for scrobbling.

Lots of love,

Photo of Laura

Laura, Official Rep

  • 448 Posts
  • 365 Reply Likes

Posted 10 months ago

  • 63
Photo of Jon

Jon, Community & Customer Services

  • 4871 Posts
  • 3623 Reply Likes
Official Response
Thanks for your feedback everyone, we've rolled this out to all users now.  With that in mind, I want to try and clarify a change we've made that's come up a few times, but isn't mentioned in the original post, concerning artist corrections and newer releases:  

  • Where there were previously corrections that only existed in the library, but not in the catalogue - these have been undone, so that the corrections seen in the library match those seen in the catalogue. This fixes numerous bugs people have been experiencing (e.g. not being able to delete scrobbles, not being able to unlove tracks, certain artists / albums appearing twice in the charts, other corrections weirdness, etc).
We didn't mention this before, because it's somewhat complicated to explain, and appears as a regression even though we have valid reasons for doing it. 

So what does this actually mean?

If you scrobble music from an artist that's corrected to something else (e.g. Utada Hikaru which corrects to 宇多田ヒカル ), you will find that their newer tracks and albums (mid 2015 onwards) are still displayed in your library under the 'noredirect' spelling. 

If you're used to how corrections work on the site (especially in the past), this might seem counter-intuitive, so I'm going to use an example: 

If I scrobble  by Utada Hikaru on Spotify, it will scrobble on my profile as 宇多田ヒカル.  This is because we already have a catalogue correction for that track, which you can view here (notice that it redirected you to the correct spelling).

Now, if I scrobble  by Utada Hikaru on Spotify (from the 2016 album Fantôme), it scrobbles on my profile as Utada Hikaru — 道.  This is because we don't yet have a catalogue correction for that track yet,as you can see here (notice that it doesn't correct you to the right page, even though the artist page is corrected).

Shouldn't artist corrections just automatically apply to all tracks?

Conceptually yes, but in practise no. As the Catalogue corrections dataset is static and can't be edited, we can't map existing artist corrections to their new tracks yet -- as soon as we're able to do this, this problem will fix itself. 

Note that we will still count those scrobbles under the corrected spelling (e.g. 宇多田ヒカル) in your artist chart. So the main place you're going to notice this issue is under your recent tracks, top tracks, and top albums.  Top artists shouldn't be affected, and anything that was already in the catalogue before we relaunched the website in 2015 should be displayed under the correct name. 

Again this will only affect new music from artists where you're scrobbling the uncorrected / noredirect spelling (e.g. k-pop and j-pop artists played in Spotify, but any new release scrobbled with the noredirect spelling).  We will fix this, but sorry for any inconvenience caused in the meantime.

Why is this happening at all? 

For clarity, I'm going to grossly simplify the actual technical reasons, but in essence: to display your scrobbled data correctly and efficiently, your user library and the catalogue each maintain their own separate version of the corrections mapping dataset.  When we relaunched the website in August 2015, a problem resulted in these two sets of corrections to fall out of sync with each other, i.e. the library had corrections that the catalogue didn't know about, and vice versa.  

This caused the vast majority of the problems people have been reporting over the last 3 years, such as not being able to unlove or delete tracks; artists and albums appearing twice in your library; tracks and albums not being clickable, etc.  It also prevented us from finishing the catalogue project, which would allow us add new corrections to the site, and remove bad ones.  So even though the library was doing the right thing in some cases, it was doing it in the wrong way, and causing serious problems with user libraries and the catalogue as a whole.  

To fix this, we first need to ensure that the library and catalogue corrections datasets are properly in sync again, and that's what this change does, even though it's going to look wrong for some releases.  The next step to fully fixing this is to complete work on the Catalogue so that we can start applying artist corrections to their new tracks.  Once that's that done, these problems should be fixed for good, however I don't have a timeline for when exactly we're going to address that part of the work yet (we're still monitoring the rollout of the new database, and the new catalogue page design).  Thanks for your understanding and patience in the meantime.
Photo of Jon

Jon, Community & Customer Services

  • 4871 Posts
  • 3623 Reply Likes
Official Response

We've become aware of a a display issue with charts falling out of sync and not showing the correct scrobble count on some library pages.  Your actual scrobbles have not been affected, but you may see some inconsistencies in playcounts between different library pages.  For example, you might see that a track has 2 plays in your charts, but when you open the track page you only see one scrobble for the track.

Our engineers are aware of the problem and are working on a fix, which we expect will go out early next week.  Sorry for any inconvenience or concern caused by this in the meantime.

For more information, this problem is being tracked and discussed here.