Help get this topic noticed by sharing it on Twitter, Facebook, or email.

DB_Refresh batch sizes

When calling SF_Refresh without the 'bulkapi' option, we want to make sure we are not clobbering the Web Service call limit for SF.com.

First Question:
For a non-bulkapi call, what is the default number of records in a batch?
Is it 2000 rows like in SF_Replicate?

Second Question:
We are trying to balance performance of SF_REFRESH with limiting Web Service Calls. If I pass 'Yes' for parm SchemaError and it finds an issue it needs to initiate an SF_REPLICATE.
Is the only way to insure an SF_Replicate spawned from an SF_REFRESH gets run with 'bulkapi' to always do a SF_REFRESH with the bulkapi parm?
1 person has
this question
+1
Reply
  • 1. SF_Refresh always asks for 2000 rows / call but salesforce can dynamically reduce that.

    2. Avoid doing sf_refresh with bulkapi. If you are aware that you have a high delta of row changes or deletes, then just run a sf_replicate with bulkapi or pkchunk.

    In some cases for tables where all rows change every day, you would completely avoid sf_refresh and just do a sf_replicate w/bulkapi.

    If you need a more in depth discussion, contact support@forceamp.com and ask for a best practices meeting .
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited indifferent, undecided, unconcerned kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • My concern with doing SF_REFRESH with no bulkapi is if there is a schema change I was not aware of and we want it to automatically do a SF_REPLICATE for us, then we will be doing a SF_Replicate for 3,000,000 rows in 2000 row chunks and eat up 1500 web service calls.

    The other alternative is to always do a SF_REFRESH with bulkapi and waste time each refresh cycle.

    It would be super if we could do a SF_REFRESH with no bulkapi, and if SF_REPLICATE needs to be called for a schema error that we could pass a seperate bulkapi parm for the SF_REPLICATE.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited indifferent, undecided, unconcerned kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • "It would be super..."

    You can. Just setup a sql job with 2 steps:

    Step 1: Sf_refresh with no bulkapi or yes.
    Step 2: SF_Replicate with pkchunk or bulkapi

    Then set the step options to skip step 2 if step 1 does not error.

    If there is a schema error, step 1 will fail and the job will run step 2.
    If there is not a schema error, then the sf_refresh worked and step 2 will be skipped.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited indifferent, undecided, unconcerned kidding, amused, unsure, silly sad, anxious, confused, frustrated