Help get this topic noticed by sharing it on Twitter, Facebook, or email.

DBAmp.exe with pkchunk option failing with batchsize of 25000

Trying to export all the data of Activity_Data__c table that has around 2.5 million rows.
The number of rows is not very high, but one of the fields an data type "Long Text Area(131072)" making the data size pretty big (on SQL Server with data_compression=page the table size is around 60GB).

DBAmp is giving errors even with batchsize of 25000.
On Salesforce side, in Bulk Data Load Jobs, the job reports:
Status: Aborted
Progress: 100%
Records Processed: 2448302
All rows are exported, only indication for a problem is the status.

Below the used command and log.
And suggestion how to resolve this, except using SF_ReplicateLarge?

exec master..xp_cmdshell 'C:\"Program Files"\DBAmp\DBAmp.exe "Export:pkchunk,batchsize(25000)" "Activity_Data__c" "DEHB05SR177\ODSDEV" "DBAmp" "LS_Salesforce_ProjectUAT"'

18:58:30: DBAmp Bulk Operations. V3.4.4 (c) Copyright 2006-2017 forceAmp.com LLC
18:58:30: Populating local table Activity_Data__c , DEHB05SR177\ODSDEV / DBAmp .
18:58:30: Batch size reset to 25000 rows per batch.
18:58:31: DBAmp is using the SQL Native Client.
18:58:31: Opening SQL Server rowset
18:58:31: Using the bulkapi with polling every 60 seconds.
18:58:31: Job 7506E0000006F7dQAE created on salesforce.
18:58:32: Job still running.
18:58:47: Job still running.
18:59:47: Job still running.
19:00:47: Fetching batch 7516E0000006gwMQAQ.
19:05:06: Error: Unable to retrieve batch result.
19:05:06: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
19:05:06:
19:05:06: Retrying to retrieve batch result.
19:11:22: 25000 rows downloaded.
19:11:22: Fetching batch 7516E0000006gwNQAQ.
19:15:44: Error: Unable to retrieve batch result.
19:15:44: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
19:15:44:
19:15:44: Retrying to retrieve batch result.
19:23:18: 50000 rows downloaded.
19:23:18: Fetching batch 7516E0000006gwTQAQ.
19:29:54: 75000 rows downloaded.
19:29:54: Fetching batch 7516E0000006gwWQAQ.
19:37:02: 100000 rows downloaded.
19:37:03: Fetching batch 7516E0000006gwXQAQ.
19:41:00: Error: Unable to retrieve batch result.
19:41:00: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
19:41:00:
19:41:00: Retrying to retrieve batch result.
19:48:28: 125000 rows downloaded.
19:48:28: Fetching batch 7516E0000006gwZQAQ.
19:53:39: Error: Unable to retrieve batch result.
19:53:39: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
19:53:39:
19:53:39: Retrying to retrieve batch result.
20:00:01: 150000 rows downloaded.
20:00:01: Fetching batch 7516E0000006gwaQAA.
20:05:04: Error: Unable to retrieve batch result.
20:05:04: Failed to read from reply.
Win32Error::
The connection with the server was terminated abnormally
NULL
20:05:04:
20:05:04: Retrying to retrieve batch result.
20:12:48: 175000 rows downloaded.
20:12:48: Fetching batch 7516E0000006gweQAA.
20:18:09: Error: Unable to retrieve batch result.
20:18:09: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
20:18:09:
20:18:09: Retrying to retrieve batch result.
20:24:58: 200000 rows downloaded.
20:24:58: Fetching batch 7516E0000006gwfQAA.
20:31:29: 225000 rows downloaded.
20:31:29: Fetching batch 7516E0000006gwhQAA.
20:36:34: 250000 rows downloaded.
20:36:34: Fetching batch 7516E0000006gwiQAA.
20:42:15: Error: Unable to retrieve batch result.
20:42:15: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
20:42:15:
20:42:15: Retrying to retrieve batch result.
20:48:54: 275000 rows downloaded.
20:48:54: Fetching batch 7516E0000006gwkQAA.
20:53:42: Error: Unable to retrieve batch result.
20:53:42: Failed to read from reply.
Win32Error::
The connection with the server was terminated abnormally
NULL
20:53:42:
20:53:42: Retrying to retrieve batch result.
21:00:39: 300000 rows downloaded.
21:00:39: Fetching batch 7516E0000006gwlQAA.
21:07:54: 325000 rows downloaded.
21:07:54: Fetching batch 7516E0000006gwnQAA.
21:13:06: 350000 rows downloaded.
21:13:06: Fetching batch 7516E0000006gwoQAA.
21:18:24: 375000 rows downloaded.
21:18:24: Fetching batch 7516E0000006gwpQAA.
21:23:44: 400000 rows downloaded.
21:23:44: Fetching batch 7516E0000006gwqQAA.
21:30:05: 425000 rows downloaded.
21:30:05: Fetching batch 7516E0000006gwuQAA.
21:35:34: Error: Unable to retrieve batch result.
21:35:34: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
21:35:34:
21:35:34: Retrying to retrieve batch result.
21:41:54: 450000 rows downloaded.
21:41:54: Fetching batch 7516E0000006gwvQAA.
21:47:47: 475000 rows downloaded.
21:47:47: Job still running.
21:48:48: Fetching batch 7516E0000006gwEQAQ.
21:55:59: 500000 rows downloaded.
21:56:00: Fetching batch 7516E0000006gwJQAQ.
22:02:46: 525000 rows downloaded.
22:02:46: Fetching batch 7516E0000006gwKQAQ.
22:10:04: 550000 rows downloaded.
22:10:05: Fetching batch 7516E0000006gwLQAQ.
22:17:13: 575000 rows downloaded.
22:17:13: Fetching batch 7516E0000006gwOQAQ.
22:24:21: 600000 rows downloaded.
22:24:21: Fetching batch 7516E0000006gwPQAQ.
22:28:16: Error: Unable to retrieve batch result.
22:28:16: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
22:28:16:
22:28:16: Retrying to retrieve batch result.
22:34:37: 625000 rows downloaded.
22:34:37: Fetching batch 7516E0000006gwQQAQ.
22:39:22: Error: Unable to retrieve batch result.
22:39:22: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
22:39:22:
22:39:22: Retrying to retrieve batch result.
22:43:24: Error: Unable to retrieve batch result.
22:43:24: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
22:43:24:
22:43:24: Retrying to retrieve batch result.
22:48:40: Error: Unable to retrieve batch result.
22:48:40: Failed to read from reply.
Win32Error::
The server returned an invalid or unrecognized response
NULL
22:48:40:
NULL
1 person has
this problem
+1
Reply
  • The error is occurring because your proxy / firewall is timing out due to a TCP Idle Timeout. The Idle timeout occurs when the firewall sends a message to salesforce and does not hear a reply for (in your case) 5 minutes. The firewall considers the connection to be "dead" and terminates it. DBAmp will retry 3 times to get the result.

    You should ask your network team to increase the TCP Idle Timeout (also called the Dead Connection timeout) to at least 20 minutes for all https traffic from this server to salesforce.

    You could also keep reducing the batchsize until you reach a size where the salesforce server could respond within the 5 minute time.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited