Bookmark Recovery Tips

Edit Subject

@Deborah, I suggest you read what I wrote more carefully. This isn't personal at all. Cheerleaders offer nothing other than salve for Larry's bruised ego. While that is nice, what I'm offering Larry is a way forward for his business. If he's smart, he'll listen. If not, he'll continue what he was doing before. The message might sound harsh but sugar-coating the truth isn't going to help anyone, least of all Larry. This is an opportunity for him to learn from his mistakes and salvage his business.

By the way, I did not "imply" there was "neglect, ignorance and hubris" and Larry's part. I outright said it.

Neglect - not to attend with due care or attention. There is no question that Larry did not pay due care or attention to the issue of disaster recovery.

Ignorance - the state of being uneducated or uninformed. Larry clearly was ignorant of proper backup procedures. He portrays having to backup a 500GB database as something extraordinary. It isn't.

Hubris - overbearing pride or presumption. Larry Halff still seems to think this was bad luck. It wasn't. This is his pride talking. This situation was entirely predictable and unavoidable. If he doesn't understand and accept that now, I'll say it again, Magnolia does not deserve to stay in business because he will only cause more grief for users in the future.

The truth hurts sometimes.

@Deborah,

I, for one, commend onionsformagnolia for saying things the way they are. Larry made an irresponsible decision and he failed all of us. I, actually, think Onions is being too generous saying that he would continue with Ma.gnolia if this situations clears out and we recover the links. If this happens I would only stay in Ma.gnolia enough time to gather my links and take them elsewhere.

No one in this forum has the right to police what people say and condemn our attitude. This is a forum to express our concerns and Larry has done a lousy job in taking care of this situation so far, so it's understandable.

If you deal with this differently, well, kudos to you, but we are all different.

@Heidi, I agree with you that these "solutions" are misleading and unfair to anyone who has more than a few dozen bookmarks. If you have thousands of bookmarks with a few hundred tags, forget it. None of these "solutions" will work for you. The only hope is that the drive recovery people manage to recover the database.

@Gerry Quach, Larry should be encouraged all right. He should be encouraged to leave system administration to professionals in the future! It would have cost under $400/month to have a proper backup in place, and that's with having a full daily, weekly, monthly backup of the 500GB database. Larry could have paid for three generations of backups for four years for less than what this episode is going to cost him, assuming this isn't going to cost him "the business", such as it was. It could have been done even cheaper than that if he had seeded the original backup using "sneakernet", stored the backups on a computer at home, and just transferred the deltas from the original on a daily, weekly, monthly basis. This disaster wasn't caused by Larry being unlucky. It was caused due to his neglect, lack of clue in disaster recovery procedures, and hubris.

I'm sure that most of the Magnolia users have no idea how to backup data on that scale. If they're like the average user, they've probably lost data before, at least several times, due to poor or non-existent backups. Of course they would be sympathetic to Larry when he portrays this as a "Sorry but shit happens." event. He'll get no sympathy from me. He doesn't deserve to be let off the hook that easily. If the data recovery folks manage to recover the database, we can all breathe a collective sigh of relief but it would be a huge mistake for Larry or anyone else to continue believing that this disaster was unavoidable.

@Larry, I know I said that I would never entrust anything to you again. I retract that. If and only if the data recovery folks manage to recover the database, you have a chance of resurrecting Magnolia but it will require that you implement very different communications and disaster recovery policies than you've had. If you still think that this disaster was unavoidable, you haven't learned anything and don't deserve to remain in business. If you continue to pursue a CYA (Cover Your Ass) communications strategy, again, you haven't learned anything and again, you don't deserve to remain in business. Demonstrate through your words and deeds that you deserve the trust of users and you'll have it. Continue on your current course and you don't deserve it and Magnolia will die a well-deserved death, It's all up to you (and the drive recovery people).

You learn something new every day. I had never heard of the term "Sybil attack". Rest assured, I registered last night and posted whatever I posted under my own nick, nothing more, nothing less. That other people may choose to agree or disagree with what I have written is not under my control nor do I really care one way or another.

As for criticisms of how this is personal, no, it isn't. I have no idea who Larry is and I had no idea that Magnolia was/is a one man band. That he may "feel bad" isn't my concern right now. I'd rather that he wasn't going through this but he is. I didn't bring it upon him. He did.

To the person above who is said that bad stuff happens and that backups aren't infallible, though that's true, best practices call for a few generations of backups. We do monthly, weekly, and daily scripted backups of all our data and more importantly, we TEST recoveries to make sure that we can restore from the backups. The backups are highly-automated and require very little by way of babysitting on a day-to-day basis. Moreover, as I've already explained, if Magnolia were so easy to back up for its users, then we wouldn't need Magnolia, would we? The whole point of this service for me was that it provided a tool that I didn't have locally.

The concern I have is that through neglect, ignorance and hubris, data that may have otherwise been recoverable could now be irretrievably lost. There are some things one can do to preserve as much data as possible in disaster scenarios such as this and there is no evidence that these measures have been taken. Perhaps all the right things have been done but how can we know? All we have are platitudes, deflections and expressions of regret with no substantive information.

As for drive recovery being expensive, considering the number of Magnolia users who are affected by this, I would think it would be a no-brainer to spend whatever it takes to do it. If it costs any more than an exorbitant $5000, chances are there would be nothing to recover anyway. I'd happily contribute to a fund directed at this but again, we don't even know if Larry has control of the server(s) any more.

Those of you who seem to think that who I am and whether I'm male or female matters, and who think I'm being mean to Larry, ought to concentrate on the message, not the messenger. My message is simple. Larry has a moral obligation to Magnolia users to provide a complete and detailed explanation without obfuscation, deflection or evasion.

Onionsformagnolia - have you considered:

1. Sharing your real name with the rest of us, instead of taking personal shots at Larry from out of the dark?

2. The possibility that maybe Ma.gnolia DID have a backup, and it failed at just the moment it was needed? That maybe Larry wasn't negligent, but merely incredibly unlucky? That backups aren't about risk elimination, because that's impossible, but merely risk reduction?

3. That maybe you ought to be angry with yourself, because every newbie on the Internet knows that one is supposed to back up one's pages on one's own hard disk? Ma.gnolia is close to being an all text environment, 1 Meg worth of text is about 66 pages worth (I think), and even a relatively old disk can hold over 10GB or 10,000 megs of information, so unless you've written over 660,000 pages worth of material, it's hard to picture you running out of space to store it all, even on that first disk, unless you filled it up with rich media like photos, etc.

Even if you did - you couldn't spare the tiny fraction of your memory space needed to store as much material as a man is likely to be able to write, even in 100 years? So, if you didn't have your bookmarks - and any comments you had attached to them - backed up on your own computer, whose fault is that?

4. The possibility of just calming way the H*** down? That rant of yours went way over the top. I understand the frustration, and I don't blame you for feeling frustrated, but at this point you seem to be looking for excuses to be personally abusive.

You haven't lost your loved ones, your home, or even your job. You lost a few bookmarks that you should have had the sense to backup, yourself - so please, get a little perspective. You haven't suffered a loss so great that you should be expected to be incapable of reason.

Bad things can happen in life, even without anybody doing anything wrong. That's not a comforting thought, but it is reality. Ma.gnolia is a low cost service, albeit a very nice low cost service, one with relatively few users and a tasteful minimum of advertising - meaning that its revenue stream is limited, and there was a limit to just how much hardware they could afford to buy, and how much space they could purchase on remote servers for backup purposes, before things went wrong. Security costs money. The less you buy, the less you get.

If you buy a honda, do you complain about the fact that you don't get the same level of performance that your neighbor is seeing out of his volvo? That's basically what you're doing, right now. That's not to say that we shouldn't expect anything out of Larry. We have a right to expect him to make a good faith effort to make use of the resources at his disposal in a conscientious manner - just like we'd expect honda to sell us a car without a leaking fuel line - but he has the right to expect us to understand that those resources aren't unlimited, just like the guy at the dealership would have the right to look at you a little strangely if you complained that your civic couldn't go from 0 to 60 in 3.7 seconds, like your other neighbor's porsche. You should get what you pay for, which is not nothing, even in the case of the free users - note that Ma.gnolia runs Google AdSense, so the free users pay for the service with the donation of the content that makes Ma.gnolia's ad revenue possible - but you don't get everything, either.

But I get the feeling that you're here to scream, not listen.

It is true that Gmail has had outages but I'm not aware of any instances of data loss. Users have an expectation that these services are run by competent people and as such, don't consider the possibility that they might not be.

The appeal of Magnolia for me wasn't that I didn't have the ability to manage my own bookmarks locally. It was that I could tag the bookmarks in a cross-browser, cross-machine way and share my bookmarks with others. If I could have backed up those bookmarks and the semantic data around them easily, I wouldn't have needed Magnolia. A collection of bookmarks without semantic data about those bookmarks is useless to me. I already have about 10,000 of those in my Firefox bookmarks that I've been dragging around since the mid '90s. The real value is in the many hours that countless others on Magnolia and I spent categorizing the web.

I really appreciate the tips that people other than Larry have been posting here. I know you're trying to be helpful and that we're in the same boat. However, the only real solution lies with Larry and so far, he has been less than forthcoming.

The Google cache scheme where you search for tags is only helpful if you have only one page of bookmarks per tag. I have many tags for which I have multiple pages of links and it's not obvious how to get them. Moreover, it is a monumental task even if I could get all them that I would not have to undertake if Larry and Co. managed to get their act together.

Larry, your silence is deafening. I repeat, where is the backup? Why have you not been able to recover? What is the real problem? The people who entrusted their bookmarks and semantic tags to Magnolia deserve straight answers.

By the way, none of the approaches to getting bookmarks from alternate sources should work for bookmarks marked as "private" if Magnolia worked as advertised. I know I have a significant number of those. In fact, I think that was one of the things that attracted me to Magnolia. I don't recall such an option with the alternatives I had evaluated.

Larry, on your home page, you wrote:

"So far, my efforts to recover Ma.gnolia's data store have been unsuccessful. While I'm continuing to work at it, both from the data store..."

How is that possible? What is this "data store" you're talking about? Was there no back up?

"In this past year, many of us have seen much loss around us."

What does this have to do with the current problem, other than a lame attempt at deflection?

"While bookmarks seem small on the national or global scale, I know that many of you had built intellectual and social capital through the bookmarks, groups, and connections you made here."

A blinding flash of the obvious. So far, you've said many words without saying anything.

"Ma.gnolia was approaching the third anniversary of its public launch; for me, it was the project and people to which I'd devoted most of my time, energy, and love for nearly four years. It's still a little too soon to give word about the return of Ma.gnolia the service and the future of the M2 project, but I will keep this site and our Twitter account updated as those decisions are made."

You have nerve to even talk about the return of Magnolia and M2, whatever that is/was. Forget it. This "business", if it ever was viable, is now dead all but in name. I wouldn't entrust anything to you any more because of the way you've bungled this. You've been quite evasive in answering direct questions posed to you about what happened. If you owe at least one thing to Magnolia users, it's an honest explanation.

At the moment, it looks like I've lost thousands of bookmarks that I've added over the last couple of years and I did not have them anywhere else because I was foolish enough to assume that Magnolia was being run by competent people! To say that I'm pissed off is an understatement. The irony is that I had evaluated and rejected Delicious and SimpPy for reasons that I don't even recall any more. I've lost something of tremendous value and by the looks of it, I will not be able to recover anything since none of the "solutions" that you have posted so far work for me. I'm working on the assumption that any competent system administrator would have been able to get a server running by now and restored from a backup, even if *some* data was lost.

What happened? Your server got seized by creditors? You had a nervous breakdown? Ex-wife got the server? Your dog ate the server? Your server got abducted by aliens? You had a disk crash and hadn't done a back up since, well, ever?

It's hard to imagine how you could look any worse so I suggest you just come clean. If it's a matter of money, I'd contribute toward a solution but not without knowing the whole, unvarnished truth. So, how about it? Are you going to explain exactly what happened or are you going to continue your "keep 'em in the dark and feed them horse !@#$" communications strategy?

another hint can be retrieving your bookmarks via your blog.

I mean, on my english blog I had a weekly post published automatically from Ma.gnolia tools as in:

http://koolinus.wordpress.com/2009/01...

As you can see I have the link, my rating and my tags ... the only missing info is the ma.gnolia group to which I've sent it but, without ma.gnolia group info doesn't make sense, or not ?

Here's another Google Cache approach:

http://www.google.com/search?safe=off...

Someone should build a Python script to grab all the cache pages for that, extract the data, then move on to the next page of results (using start=100, start=200, etc. in the parameters.) Or maybe this will be how I learn Python. :-P

In any event, this only retrieves about 200 bookmarks. I have about 1100, most of which are safely backed up.

Here's how I'm rummaging through my tag cloud via Google cache. Google shows your name in the tag title, so if you have a fairly unique name use this query:

"Your Name" inurl:tags site:ma.gnolia.com

(substitute your name where it shows "Your Name" above) If your userid is unique (mine 'camera' is not) this might provide even better results

inurl:tags,youruserid site:ma.gnolia.com

(substitute your ma.gnolia userid where it shows youruserid above)

Note that after a while (about 10 pages) Google will think you're a virus (I'm not kidding - here take a look ) and will stop issuing results. I had to wait some time (I tried again a couple hours later) to continue my recovery.

I've written a quick and dirty little Python script to save ma.gnolia.com links from Google cache.

The links are saved as a boolmark.html file accepted by most browsers,
delicious etc in the directory from which it is run.

This may or may not be allowed by Google's terms of service. I assume no responsibility for any use or harm caused from use of this script. Use at
your own risk.

Don't forget to enter your ma.gnolia.com username into the script.

Also note that it only pulls the first three pages (last 30 bookmarks added) as that's all Google seems to keep. You may change the number at your own risk. The script should continue without error.

The script can be found at: http://dpaste.com/116737/

It will be removed in 30 days. Only tested with Python 2.5