Options for syncing Maven2 repo between two sites

Hi,

We will be moving data centers and want to setup another Maven2 repo that is managed with Nexus OSS. We want both repositories to be online and read\writeable until we migrate all our environments to the new site. I have a few questions:

1. What is the best method for copying the repo to the new location.
2. What is the best method for keeping the two repos in sync? We want to minimize network bandwidth usage.
1 person has
this question
+1
Reply
  • Jason, I'm sure there are a couple thousand ways to accomplish this task with different combinations of proxy repositories and strategies for the migration.

    First, you are going to want to seed the configuration. This post: http://www.sonatype.com/people/2010/0... deals with Nexus 1.4.1, it may be a bit dated, but it should provide you with a good starting point for which directories under sonatype-work/ really matter. When you create this new repository you are going to want to, at the very least, copy over all of the configuration.

    If you've copied over your configuration, your new Nexus instance will have all the same proxy repositories defined. Nexus will download uncached artifacts from remote repositories on an as-needed basis. If you want to minimize the amount of artifacts that have to be downloaded again, you can always copy the contents of sonatype-work/nexus/storage that correspond to your proxy repositories.

    This leaves hosted repositories. You have an option here. You can decide to just schedule a few hours of downtime and transfer the contents of sonatype-work/nexus/storage over to the new instance and then trigger a re-index of the repository via the Nexus interface. This reduces your total downtime to the amount of time it would take to transfer the contents of your sonatype-work/nexus/storage directory from one data center to the other.

    Also, something to consider... if you have a large number of project that need to migrate you can always set up your old Nexus instance to proxy the new Nexus instance. This would allow you to spread the migration out over a longer period of time without having to move every over at once. If you did this, your new instance would be the "master", and the old instance would act as a Proxy.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • We want all our builds to place artifacts on old hosted repository and then have new nexus hosted repositories sync with old nexus hosted repositories. Our code push scripts would point to the old nexus hosted repository for QA and Dev environments, and code push scripts for Prod environment would point to new hosted repositories. The proxy idea wouldn't really work in this scenario. How can we accomplish this within Nexus? I think we can do this using rsync at the file system but wasn't sure if Nexus had something built in to do this.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited sad, anxious, confused, frustrated

  • There's nothing built into the versions of Nexus currently on the market, but someone else may be able to comment on future product plans.

    For the time being, your best bet might just be to rsync -av (without the --delete) option and then trigger a reindex.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited sad, anxious, confused, frustrated

  • when a nexus instance is a proxy, does it cache the artifacts and then send them to the master? Is there any better documentation other than the book?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Poking around I noticed under scheduled tasks, there was a "sync shadow repository", what is that? Could we use that to accomplish keeping the repos in sync?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Here, instead of trying to describe this, I made a diagram and recorded a video. Yes, you can absolutely use rsync, but you can also configure a few Proxy repositories in both Nexus instances to reference the other:

    https://www.youtube.com/watch?v=T0buDL...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited sad, anxious, confused, frustrated

  • If repository A on nexus1 gets updated, repository A on nexus2 would not get updated until a request for the artifact comes in to Nexus2 and then the artifact is fetched from Nexus1 right? Each subsequent requests would be local, right?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned happy, confident, thankful, excited sad, anxious, confused, frustrated