Possible to speed up GMail archives?

  • Question
  • Updated 3 years ago
  • Answered
GMail backups take a LONG time. I'm currently backing up 300,000+ emails from GMail. It seems like it has to run through and display every message with (skipped) before it finally catches up to the recent stuff. Is there a way to help MailStore only backup (or even pay attention to) things that are more recent? Is there a setting to have it only look at things new since the last backup run? I would like to schedule it daily but not if it's going to look at every message every time.

rlindner

  • 2 Posts
  • 0 Likes
  • confused, frustrated

Posted 3 years ago

  • 1

sneader

  • 1 Post
  • 0 Likes
I have the same problem. My All Mail folder indicates I have 375,000 messages. Seems that if I lose the connection to GMail, MailStore craps out, and there is no way to have it "start where it left off" -- it has to start from the beginning.

I think a partial solution would be for MailStore to not give up so easy... if it loses the connection... well, keep trying to restore the connection!?

I am not sure I've ever had a successful backup of GMail, although I think it has been close at times.

Will Wisp

  • 9 Posts
  • 0 Likes
If you make a new Gmail Label and only apply it to recent mails, you can then tell MailStore to only archive the contents of that folder/label. Then you could run archives for each other label in turn.

You can do date selective searches in Gmail by using the normal text search box. Enter a search string of - before:2010/1/1 after:2009/1/1 - you will get results of every mail in 2009. I just tested this and it works ok.
(Note that the 'after:2009/1/1' bit will also include all the mails of 1st Jan.)

(You can also set up a filter to do the same thing. Put the date string into the 'has the words' selection box and the filter will find the relevant mails. Don't forget to check the 'also apply filter to...' check-box before updating.)

You can perform that search for each year or six month or whatever, and then apply a label to all the mails in results. Then do the backups selectively.

BTW, when applying the tick to select ALL search results, there's a little blue text link that comes up to 'Select all conversations that match this search'. Use that to apply selection to ALL the results, even if there's lots more than one page full. Just mentioning it as it's easy to miss.

Disclaimer: I haven't actually TRIED this yet, but it seems ok in theory. :)

rlindner

  • 2 Posts
  • 0 Likes
An interesting idea but I think if I select a label it will remove stuff that is already backed up. I tried this label filtering technique when the initial backup was taking a long time. I thought since it was already past sent-mail I'd filter that out so it could jump right to the others but then it looked like it was removing those from my archive.

Will Wisp

  • 9 Posts
  • 0 Likes
I don't think you can 'filter out' in that way. I think the logic is that it will process a label that is included, and ignore a label excluded, but if a mail has both labels, the exclusion doesn't count because the included label will be processed anyway.

Will Wisp

  • 9 Posts
  • 0 Likes
According to Daniel on the 'some items missing..' topic, he said:

"Duplicates might also occur when an email was COPIED to other folder, as MailStore only handles one unique email per archiving source. If an email is found in more than one folder, it can be found in the archive at the last location MailStore found it while processing the profile."

So could it be that the removed items you saw were ones that were duplicated by other labels? From what he says, there can be only one occurrence of any one mail and the most recently found one would supercede (and delete) any already archived. Perhaps if you year-dated the mails and backed up ONLY the date labels, there would be no duplication? (Except where email conversations started in one year and finished in the next. In this case they'd have labels for two date ranges. I just tested it and that's what happens.)

Will Wisp

  • 9 Posts
  • 0 Likes
Ok, I just ran a test on one of my Gmail accounts. I created a label for 2008 and one for 2009. There was only one conversation of two mails, that crossed over the year end, one sent at the end of 2008 and the reply at start of 2009. That conversation had picked up both labels from the filtering and labeling.

Mailstore backed up 2008 without any problem and without any duplications.
Then I backed up 2009 and it ran fine, but reported that it had moved two messages in the archive, which sounds correct if, as expected, it had re-assigned that 2-message conversation to 2009.

As a test I then re-ran 2008. It ran in just a few seconds as there were no changes. Except that it reported having moved two messages. Again, this seems to be correct. As Daniel said, Mailstore will re-assign a duplication to the most recently run archive.

So it would seem that it works ok to use date labels. A good method might be to create a profile for the first year/period and run it. Then edit the same profile adding the next period and run again etc. That way, you're only using one profile and so keeping things tidy. Each time, the periods already archived would just skip through in seconds before adding the mails for the newly added period. Finally, create a second profile for the current period which you want to run daily. This can be added/incorporated into to the previous archive when the period is complete and a new one started.

Just for tidiness, it might be worth always running 'Sent Mail' last so as to be complete and up to date and not have lost duplicates to others.

Will Wisp

  • 9 Posts
  • 0 Likes
A further note about labels: If you restore the mails from backup created using date ranges, only the date labels will now exist. But for the old labels that are conditional on filters, then the old label information can be restored by re-running the filter and checking the box to 'Also Apply Filter to' the resulting filtered mails. Now delete the labels created by MailStore and everything should be back as it was. Probably :) ....

....Apart from the Sent Mail which will be in a new folder/label. I think it could be moved into Sent Mail using a different IMAP client but maybe someone could advise?

Oh, and BTW, this backup strategy is based just on guesswork and a few small experiments. I haven't actually tried a full backup and restore as I've only been messing with this MailStore prog for a short while so don't assume I know what I'm on about, but good luck.