I've got some pretty old albums, including re-recs, that have been lingering in my featured page for a while, and I've tried clicking on the "+" button, but it only gives me the option to add the album to a collaboration.
How can I take the albums out of the featured page?
Select "albums" in your profile and then select your "featured records" album. Click revise and then you can reorganize and add or delete records. It will only remove the records from your featured album not your profile.