Help get this topic noticed by sharing it on Twitter, Facebook, or email.
I’m awaiting this feature

Configuring your Mozy backup set is really slow when you have lots of files. It should let you edit your backup set by editing a text file.

The Mozy client was not design to scale to huge numbers of files, and when you try to edit the configuration, the client slows to a crawl. Unfortunately, many computer users have huge numbers of files to back up.

The Mozy client should let you specify a backup set by editing text file instead, so you don't have to go through the slow graphical interface. If it were set up so you edit a text file and fill it with directives like this, it would be much better:

# Backs up everything in documents and settings
starts:c:/documents and settings/

# Backs up files ending in .gif
ends:.gif

# Don't back up files ending in .tmp
-ends: .tmp

# Backs up any file that contains 'photos' in the filename or directory
contains: photos

# Don't back up any file starting with a dot
-regexp: \.[^/]*$

# Don't back up any file in an .svn directory
-regexp: ^.*/\.svn/

This is just an example -- I'll leave it up the Mozy engineers to come up with a better and more flexible syntax. You should be able to apply rules to full paths, directories, filenames, or extensions.
7 people like
this idea
+1
Reply
  • I’m frustrated
    No kidding...it takes forever. Also if you make too many changes there's no way to save them without closing out of the interface. But if you don't save them it's likely to CRASH and you start over.

    There's also not good way to exclude a specific file or two (at least that works). I gave up on trying different exclude rules because it took forever each time and it still failed. There are a couple of files in a folder that are always in use exclusively and I want it to ignore those two files only because otherwise it claims the backup failed when I expect those files to be in use.

    I had to do an exclude of the file types and only backup that folder, then create another job to grab the sub-folders. Unfortunately if someone creates a file-type that is of the same type of the files that are in use in that same folder, it will fail to back it up.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • 1
    Lee, RoHo-

    We've been working (behind the scenes, obviously) to improve support for extremely large file sets. There are a few tricks you can use to seriously boost performance:

    1. Don't create backup sets with root drives as locations (especially not the system drive). When you do so, it must scan every folder in the entire drive to find what you need it to look for. Keep it as restrictive as possible and it will do far better.

    2. Don't check the root of any drive in the File System tab. This presents the same kind of problem as #1- Mozy is forced to scan for changes when you open the configuration application. That, as you've seen, can take quite a bit of time.

    3. If you're a Pro user, try using the MozyPro Admin console. There is a "Client Configuration" utility that offers much the same functionality as the actual client application. The only thing missing is the "File System" tab, but with a bit of creativity, you can turn this to your advantage. Just remember that if you choose a location and don't create any rules, it will default to backing up *everything* in the location, just as if you had selected it in the File System tab in the client.

    Again, we're working hard on giving an actual solution to the problem. I hope the workarounds will give you a hand- let me know if there's anything else I can do to help.

    Tom
    Mozy, Inc.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • I’m frustrated
    Frankly, I can't see ANY reason why it should take 30-45 minutes to get in so I can change my configuration. During this time, one of my 2 cores is 100% dedicated to mozyproconf.exe. There is no hard drive access, and seems to be no network access. Requiring this much time simply indicates a poor design. Lee's idea would provide a workaround, but how about simply fixing the code instead of adding a new feature.

    Tom, don't "improve" performance. You would need at least a 99% reduction in load time across the board for this to load in 30 seconds. (Yes, I did the math, that is the correct number. You would probably need remove several orders of magnitude from the big O. I don't think this is likely to happen.) 30 seconds is still awful, but it would be heavenly compared to 45 minutes.

    Why is blazes would you want to crawl every file in every folder that has anything to do with any backup? That is a ridiculous thing to do in response to a user initiated request. Why not keep a small cache to remember the files in the backup. All you need to know is the files involved in the last backup attempt, and how big they were. I can configure things pretty well just starting from that information. Use this to provide an estimate for the space used/available feature. If I navigate to a folder, check the contents of that folder, but not sub folders, and update the estimate and cache. That's all the user really wants or needs anyway. If you think anybody ever REALLY needs an exact number, you could provide a button to get it. At 45+ minutes per click, I don't see anybody ever bothering to click the button.

    Also, based on your response, I'm guessing that Mozy does similarly stupid things when it does a backup? (Walking every file on the computer?) That would explain some things. I had to move my backups to once daily at 4AM to prevent it from hogging my computer at random times. This is where your "improvements" should go.

    Well, it's been an hour now since I tried to get into the configuration, and it's still chewing. I have better things to do today, so I'm going to kill it and reboot. (I can't use the cancel button, it will complete the operation anyway, with the added kick in the jaw of NOT letting me into the configuration afterwards.) Maybe I'll leave it overnight and hope to be able to reconfigure the next morning...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • John-

    I'll be a bit more clear- the configuration utility is completely different from the backup agent. One of the problems is that we assume that it is safe to load the entire fileset into memory when using hte configuration application. It leads to problems like the one you described. You see this more clearly than others as you have a programming background. It was an oversight- we hadn't intended on Mozy being used for backing up more than 200,000 files. We're taking great pains right now to rewrite this. As it stands, the suggestions you've made are quite good- and we've incorporated them already. We do keep a cache of known files- and we take it one step farther.

    The client installs what is called a file system filter that is designed to watch for changes *as they take place*. Given the Win32 IO Manager architecture, it does not introduce any real degree of overhead and allows Mozy to be aware of not only which files that change, but the exact blocks that change. Still, by necessity, if you have the C: drive selected as a backup set source, that filter will be watching the entire drive. Again, this isn't a real performance killer, but it's not good either, especially since it can be avoided.

    We've done as much as possible to reduce the load on the system while backing up. As I mentioned, the backup agent itself isn't the piece impacted by the number of files- it's just the configuration application. And yes, your math is correct (I've run the numbers on various tests I've run myself). It needs a complete rewrite taking into consideration the lessons we've learned (from you and others) over the past years.

    It's in the works. We're working through a complete refresh of the client right now and I'm sure you'll be pleased with the results. If possible, try the steps I outlined earlier. Let me know if you have any other suggestions or thoughts- now is the perfect time to let them be known.

    Thanks,
    Tom
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • Lee-

    I would love to give this functionality to Home users- I'll raise this in our next product management meeting and see if we can't move with it.

    Thanks,
    Tom
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m happy
    Tom,

    Thanks for the reply. I'm cool with a rewrite, even if it takes a little longer. I believe in rewrites. I'll look forward to the update.

    As for your recommendations, I've done as much as I can. My "My Documents" folder is over 10Gb, mostly because I have a gazillion software development workspaces underneath it, and those can get awfully big, awfully fast. I have stuff in the root "My Documents" folder too though, so that is the cutoff point for my backup sets. This means that there isn't a whole lot I can do to reduce the impact of this problem. I did trim the set a bit and it now only looks at 75% of the stuff it used to, but I really can't cull it anymore.

    The file system filter sounds great. I didn't mention it earlier because I wasn't sure about portability, and I think it still leaves a sticky situation if the content is modified while Windows isn't up (dual boot systems, external storage, or if I slave my drive to another machine.) If you want to take that mess on though more power to you. I'll be thrilled to get the performance boost.

    Thanks!
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • I’m disappointed
    Months have passed, and this is still an issue.

    Dear Mozy - any plans on getting this to actually work with large sets?

    You suggest being able to backup whole machines, yet Mozy on mine sits for 30 minutes before it even starts to backup a single file.

    Apologies for venting, I just find it hard to get Mozy to work as advertised.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Roman, this thread is real old and those issues should be resolved. I need more info. How many files are you selecting for backup? Do you have a ticket open with support that I could look at the log file? What's the ticket number?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m frustrated
    Hey guys, has this been fixed yet? I still have the exact same problem now. I have Mozy configured to back up my entire C drive and it's basically a huge disaster. The configuration utility fails when I open it (perhaps you are still trying to load the entire fileset into memory when the user opens the configuration settings), and backups take like a half an hour just to start. I don't think that I could recommend Mozy to anybody based on this performance track record. In fact, I am probably going to have to uninstall it because it's bringing down my entire computer.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Try selecting specific folders instead of the entire C drive. If you backup the entire C drive, it will get the system files and temp files that are constantly changing which will slow down the computer. You won't be able to restore those files anyway, so it kind of makes it pointless to back them up.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I select the C drive partly because not all of my files are neatly organized into Program Files / Documents and Settings / My Documents / Desktop all the time, and I don't want to miss anything. Also, more to the point, while it's nice to get your documents back, it's even nicer to get your entire filesystem back. Is there any way to configure Mozy to just have an entire image of your hard disk? Is that something you're working on?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • And actually, it looks like the backup has already excluded the entire Program Files and Windows directories (even though I didn't tell it to do that...), and almost everything else that's left is important. So it seems like Mozy just doesn't scale well if you have a lot of documents in My Documents. I have about 160k files.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Currently no. We just backup the files and we don't do an image backup.

    You will probably see some computer slow down after you start selecting more than 200k files, but if you have a good computer it should work. My home computer is backing up around 130k files just fine but its an older computer with a pentium 4.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • It has more to do with hard drive speed than processor speed. I'm on a few years old mid grade laptop and Mozy just cripples the poor thing when it goes to do a backup. It's probably filtering through several hundred thousand files though, and probably finding over 200k for backup.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • By the way, I changed it to do just My Documents and Desktop and it's still slow. It's not just a problem you get when you add the entire C drive.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m frustrated
    I too am still seeing the slowdowns and I'm on v 2.4.3.0
    I have 155k files and a good number of them are shadow files from SVN. Going back to Lee's original post, I wish there was a way to remove (filter) files by extension or by folder name.

    Just his last two rules would wipe out 30+% of the files my backup set.
    # Don't back up any file starting with a dot
    -regexp: \.[^/]*$

    # Don't back up any file in an .svn directory
    -regexp: ^.*/\.svn/

    I haven't used the backup config app since i set it up because it makes my computer unusable for 20-30mins. I'm running a Core2Due 2.4ghz with 4gb ram, windows 7 64bit, and 500gb HD (200gb used) backup set of ~40gb. I run the backups at night, because if the backup happens while I'm using my computer everything just drags.

    I've also found that the memory usage while doing backups can sometimes be in the 1.5gb+ and I can only assume that's leading to swap files which would totally lead to the slowdown we're all seeing.

    Maybe you need to spend some time on memory usage optimization first? My my 4 cents.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • Jeff, you should be able to create backup sets that will exclude file extensions or folders.

    Open up settings, click on options, advanced, and make sure "show advanced backup set features" is checked. Then click on the backup sets tab, right click in the white space in the left window, and add a backup set. Create a rule for that specific file type or location, and check the box that says, "Files matching this set will be EXCLUDED from the final backup set."

    That should prevent those files from being backed up.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m annoyed
    This is still an issue. I cannot even open the program and edit my settings, so now I'm needlessly backing up files that I can't remove solely because I can't remove them (and I can't add new directories to the backup). It's not an issue of an old computer, either.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I'm still seeing this issue again now that I'm up to 200k files on that system.

    Here are my thoughts. It appears only when you click the "settings" button when looking at the mozy status popup. This is also when in calculates the total size of the backup set. I have a feeling that it's doing a HD scan and not using an index file of some sort to determine the size of the backup set.

    I recently picked up a new system with a OCZ Vertex 3 (500mb/s+ solid state HD) and now the settings panel opens nearly instantly on the new system. Massive HD read, but given the solid state drives don't have any latency it's pretty quick.

    I'd suggest the mozy dev team figure out a way to not scan the file size of all the files right when the settings panel is openned. Or better yet only scan once you click a verify button or something.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • 1
    @jeff, no fair getting on here and bragging about hardware that the rest of us have been drooling over.

    Joking aside, I have a much faster hard drive now than I did 3 years ago (lost the last one due to hard drive failure, but the Mozy backup made THAT part of the recovery painless.) Just like jeff, the backup set information loads much faster for me now than before, but like most users I still live in the land of sane read speeds, so it takes about 10 minutes.

    Some ideas for optimizing this:
    1. Don't show me what WILL be backed up, show me what WAS backed up (or would have been backed up on the last backup, if it failed to finish for some reason). This simplifies the problem to a simple directory explorer interface and lightweight lookups to some database of information about the last backup. You can load each level one at a time, rather than having to scan everything to determine whether or not to display a checkmark on a directory. This information is sufficiently accurate for the vast majority of likely use cases (for example, if a backup fails to complete due to being over quota, and a user wants to remove some of the files, or if the backup completes, and a user notices that they are near quota, and want to trim the set down).
    2. Assuming an implementation of #1, you could do the heavy scan in the background to check for anything that was missed, so users with lightweight information will still see the optimal data relatively quickly.

    I'd like to add that this *is* much better than 3 years ago. There is only a moderate delay (30-40 seconds) to get into the options screen, the UI thread only blocks for perhaps 10 seconds, and I can look at and deal with the other options if needed without waiting...all vast improvements. Most of the secondary problems related to this issue are resolved now, but I'd still love to see the file information itself be handled with much less delay.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • I’m frustrated
    I don't think hardware advances are the answer to this problem. After all, hard drives are getting faster but people are also storing more files. This is an obvious design flaw in Mozy that should be fixed algorithmically. It's kind of crazy that they haven't noticed it yet since it's an aspect of routine use. This is one of the reasons I use Carbonite instead, which scales better.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I will admit I was surprised that the first posts in this thread were 3 years old and that this problem still exists. This issue virtually renders Mozy unusable since I cannot change/update any setting (As of now, I have chosen to locate files I want backed up in a directory that is already set for backup). In addition, I'm wasting bandwidth (which doesn't matter much on my end, but I imagine Mozy is interested in reducing traffic, unnecessary backups) because I can't deselect some files (for example, podcasts) that change constantly and that I have no interest in backing up.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Yes, problem was not solved. As Mozy Home customer, I am very unhappy with it.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m undecided
    I seem to recall that last time I needed to open the interface it worked a lot better than I remembered so I think it has improved but I have not worked with it extensively because the backup set has not needed to be changed much since I posted my reply over 4 years ago.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • Wow - I am a MozyPro user with a million+ files and I happened across this thread and realize this is an ongoing issue for 4 years. Doubt I will be renewing with MozyPro.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited

  • I’m frustrated
    How can this still be an issue? Fix it Mozy or we'll simply use a provider which doesn't take hours to select a large amount of files.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited