Help get this topic noticed by sharing it on Twitter, Facebook, or email.

sf_tableload with bulkapi (version 1) option

Hi

Is there anyway to force sf_tableloader to use version1 of bulkApi to get away from 150MB limit of bulkAPI2?

Somehow it looks like sf_tableloader always detects which API to use automatically which normally choose v2 for big volume data, and fails due to150MB limit.

bulkApi 1 doesn't have such limit, so why we don't have an option to force this version with sf_tableloader?

DBAmp version: V3.7.3

Thanks

Ming
1 person has
this question
+1
Reply
  • What is the exact command being run? Do you have complete message output from the latest SF_TableLoader where it is running the bulkApi2?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • Also, I highly suggest you upgrade to the latest version of DBAmp as soon as possible. Not only will you get all the bug fixes, but you will also get the enhancement of DBAmp automatically choosing which API to use for you.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • Specifically, the new version will choose bulkapi V1 over bulkapi V2 in most situations with sf_tableloader.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • Thank you Justin and Bill

    Script:

    exec SF_TableLoader 'upsert','salesforce','cloupra__Service__c_mbrAccountload', 'cloupra__External_Id__c'

    Result:

    --- Starting SF_TableLoader for cloupra__Service__c_mbrAccountload V3.5.8
    14:05:50: Run the DBAmpNet2.exe program.
    14:05:50: DBAmpNet2 3.7.3.0 (c) Copyright 2015-2017 forceAmp.com LLC
    14:05:50: Parameters: upsert cloupra__Service__c_mbrAccountload AUSYD16DB23CI1\AUSYD16DB23CI1 SalesForce salesforce
    14:05:50: Using the Salesforce bulkapi2 API.
    14:05:54: Sort column will be used to order input rows.
    14:05:55: Warning: Column Id ignored because it not creatable in the cloupra__service__c object.
    14:05:55: Warning: Column pensionuserid ignored because it does not exist in the cloupra__service__c object.
    14:05:55: Warning: Column Family_Law_Event_Last_Date__c ignored because it does not exist in the cloupra__service__c object.
    14:05:55: Warning: Column Hold_Correspondence_Flag__c ignored because it does not exist in the cloupra__service__c object.
    14:05:55: Warning: Column Supermatch_Online_Search__c ignored because it does not exist in the cloupra__service__c object.
    14:05:55: Warning: Column Sort ignored because it does not exist in the cloupra__service__c object.
    14:06:48: Drop cloupra__Service__c_mbrAccountload_Result if it exists.
    14:06:48: Create cloupra__Service__c_mbrAccountload_Result with new structure.
    14:06:48: Update Error column in cloupra__Service__c_mbrAccountload_Result.
    14:06:48: Drop Id column.
    14:06:48: Add Id column.
    14:06:48: 418824 rows read from SQL Table.
    14:06:48: Job 7500R000002gav5QAA created on salesforce.
    14:07:37: Error: Unable to upload job data.
    14:07:37: System.Net.WebException: The remote server returned an error: (400) Bad Request.
    at System.Net.HttpWebRequest.GetResponse()
    at DBAmpNet2.LoaderBulkAPI2.CreateBatch()
    14:07:37: Error: BULK_API_ERROR: Encountered an exception when processing the request: InvalidJob : Error encountered while processing job: Uploaded batch file size greater than max size. Org max upload file size is 150000000 bytes
    14:07:37: Deleted file: D:\DBAmp\Work Directory\cloupra__Service__c_mbrAccountload.csv
    14:07:37: DBAmpNet2 Operation FAILED.
    14:07:39: Error: DBAmpNet2.exe was unsuccessful.
    14:07:39: Error: Command string is C:\"Program Files"\DBAmp\DBAmpNet2.exe BulkOpsBulk "upsert" "cloupra__Service__c_mbrAccountload" "AUSYD16DB23CI1\AUSYD16DB23CI1" "SalesForce" "salesforce" "cloupra__External_Id__c" " "
    --- Ending SF_TableLoader. Operation FAILED.
    Msg 50000, Level 16, State 1, Procedure SF_TableLoader, Line 314
    SF_TableLoader Error: 14:05:50: DBAmpNet2 3.7.3.0 (c) Copyright 2015-2017 forceAmp.com LLC14:05:50: Parameters: upsert cloupra__Service__c_mbrAccountload AUSYD16DB23CI1\AUSYD16DB23CI1 SalesForce salesforce14:05:50: Using the Salesforce bulkapi2 API.14:05:54: Sort column will be used to order input rows.14:05:55: Warning: Column Id ignored because it not creatable in the cloupra__service__c object.14:05:55: Warning: Column pensionuserid ignored because it does not exist in the cloupra__service__c object.14:05:55: Warning: Column Family_Law_Event_Last_Date__c ignored because it does not exist in the cloupra__service__c object.14:05:55: Warning: Column Hold_Correspondence_Flag__c ignored because it does not exist in the cloupra__service__c object.14:05:55: Warning: Column Supermatch_Online_Search__c ignored because it does not exist in the cloupra__service__c object.14:05:55: Warning: Column Sort ignored because it does not exist in the cloupra__service__c object.14:06:48: Drop cloupra__Service__c_mbrAccountload_Result if it exists.14:06:48: Create cloupra__Service__c_mbrAccountload_Result with new structure.14:06:48: Update Error column in cloupra__Service__c_mbrAccountload_Result.14:06:48: Drop Id column.14:06:48: Add Id column.14:06:48: 418824 rows read from SQL Table.14:06:48: Job 7500R000002gav5QAA created on salesforce.14:07:37: Error: Unable to upload job data.14:07:37: System.Net.WebException: The remote server returned an error: (400) Bad Request. at System.Net.HttpWebRequest.GetResponse() at DBAmpNet2.LoaderBulkAPI2.CreateBatch()14:07:37: Error: BULK_API_ERROR: Encountered an exception when processing the request: InvalidJob : Error encountered while processing job: Uploaded batch file size greater than max size. Org max upload file size is 150000000 bytes14:07:37: Deleted file: D:\DBAmp\Work Directory\cloupra__Service__c_mbrAccountload.csv14:07:37: DBAmpNet2 Operation FAILED.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • I think you might have answered my queston that the new version of DBAmp might fix the issue - if it can automatically choose to use bulkApi vesrion 1 over 2 in such case (size over 150M)

    We will upgrade to new version of DBAmp and test again

    Thanks
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned

  • Do not skip the step to restart the sql instance or to recreate the DBAmp stored procedures. It looks like the stored procs step was skipped the last time you upgraded.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly sad, anxious, confused, frustrated happy, confident, thankful, excited indifferent, undecided, unconcerned