Issue with batch contact creation API (v3)

apoyas · ‎Jul 2, 2020

Hello HubSpot support team!

We are working on HubSpot integration with Microsoft Dynamics 365 Business Central (BC) for a customer.

Customer wants newly created or changed contacts in BC to be synchronized to HubSpot.

For contact creation we use the following API:

POST /crm/v3/objects/contacts/batch/create

A few days ago we did a rather ladge scale test: we wanted to create ~3600 contacts.

System split those contacts into batches of 10 and sent to HubSpot.

All batches were sent sequentially: one at a time, wait for restults an then sent the next one.

Around 30% of requests ended with the following errors:

    "status": "error",
    "message": "DEADLINE_EXCEEDED: deadline exceeded after 4.999935737s. [remote_addr=vocal-bear.iad02.hubspot-networks.net/172.16.0.208:2113]",
    "correlationId": "f1d74329-ec41-44a7-a3c9-05de6d7e2960",
    "category": "INTERNAL_ERROR"

    "status": "error",
    "message": "DEADLINE_EXCEEDED: ClientCall was cancelled at or after deadline. [remote_addr=kind-mask.iad02.hubspot-networks.net/172.16.34.176:2048]",
    "correlationId": "dc98404a-b8c3-4f71-af60-9b22e481b63d",
    "category": "INTERNAL_ERROR"

Another type of error we got was related to some of the contacts having values not acceptable for HubSpot:

    "status": "error",
    "message": "Property values were not valid: [{\"isValid\":false,\"message\":\"253402214400000 is more than 1,000 years ago or 1,000 years from now.\",\"error\":\"INVALID_DATE\",\"name\":\"utmeldingsdato\"}]",
    "correlationId": "ac4c1a67-a5cc-46fe-8d06-376c67198f33",
    "category": "VALIDATION_ERROR"

The response did not specify which of the contacts in the batch had invalid values.

So we assumed the whole batches were rejected and nothing were saved in HubSpot both in the case of "Property values were not valid" errors and in the case of "DEADLINE_EXCEEDED" errors.

But this wasn't the case. When we tried to run the sync again for the failed batches, we started to get the following errors:

    "status": "error",
    "message": "Contact already exists",
    "correlationId": "d8dcb776-7100-4c41-90da-0eca8d21956c",
    "category": "CONFLICT"

And again, HubSpot did not specify, which of the contacts included in the batch already existed.

It turned out that regardless of the error type some of the contacts included in a batch were still created in HubSpot.

Would it be possible to change the way HubSpot responds to a batch request, so the responce contains detailed information about each contact: if the creation was successful, hubspot should include HubSpot ID of the contact and all the rest data, which normally is included in a successful response; and if creation failed, we need all the properties back and the error message.

Many thanks!

Kamil_KK · ‎Apr 19, 2022

Hey HS Support.
I have also encounter issue with batch API, a bit different origin though:
Custom objects
POST
/crm/v3/objects/{objectType}/batch/create

{objectType} defined and works:

x-xxxxxxxxx

body:

{
    "inputs":[
        {"properties":{
            "required_property": "value",
            "required_property2": "value"
            }
        },
        {"properties":{
            "required_property": "value",
            "required_property2": "value"
            }
        }
    ]
}

In order to create 2 instances of custom object XX
however in response I've got only one instance of custom object created:

{
    "status": "COMPLETE",
    "results": [
        {
            "id": "235641335",
            "properties": {
                "hs_createdate": "2022-04-19T06:00:31.888Z",
                "hs_lastmodifieddate": "2022-04-19T06:00:31.888Z",
                "hs_object_id": "235641335",
                 "required_property": "value",
                 "required_property2": "value"
            },
            "createdAt": "2022-04-19T06:00:31.888Z",
            "updatedAt": "2022-04-19T06:00:31.888Z",
            "archived": false
        }
    ],
    "startedAt": "2022-04-19T06:00:31.884Z",
    "completedAt": "2022-04-19T06:00:31.973Z"
}

And I ran out of ideas. I am able to read instances of cusotm object created by id:
POST
/crm/v3/objects/{objectType}/batch/read

{
    "inputs":[
        {"id":"123"},
        {"id":"132"}
        ],
    "properties":["comma,separated,string,of,properties"]
        
}

RES:

{
    "status": "COMPLETE",
    "results": [
        {
            "id": "123",
            "properties": {
                "required_property": "value",
                 "required_property2": "value"
            },
            "createdAt": "2022-04-05T07:18:21.927Z",
            "updatedAt": "2022-04-05T07:18:21.927Z",
            "archived": false
        },
        {
            "id": "132",
            "properties": {
                "required_property": "value",
                 "required_property2": "value"
            },
            "createdAt": "2022-04-01T06:52:05.075Z",
            "updatedAt": "2022-04-01T11:30:10.379Z",
            "archived": false
        }
    ],
    "startedAt": "2022-04-19T05:35:23.526Z",
    "completedAt": "2022-04-19T05:35:23.537Z"
}

But I had no luck to create more than one instance of the custom object via batch API.
Is body imput wrongly defined? If so, where is the issue? (JSON was walidated, it is correct)
I would appreciate help in this matter. Thanks in advance.

apoyas · ‎Jul 7, 2020

We managed to find a workaround. This might be helpful for someone else...

So before sending over the whole batch again we use another API endpoint to check, whether each individual contact already exists in HubSpot, and exclude those already created from the batch.
We use the following API:

GET /crm/v3/objects/contacts/{EmailAddressOfTheContact}?idProperty=email

Of course for this to work you need to have contacts with mandatory email addresses, and email address must be unique for each contact.

Derek_Gervais · ‎Jul 6, 2020

Hey @apoyas ,

Thanks writing up such a thorough post documenting your experience using the CRM Objects API. I want to make sure I'm just as thorough in my response, so I'll need a bit of time to touch base with my team and investigate the behavior(s) you're describing.

In the meantime, if you have any other information you think would be relevant, or any other questions for me, please don't hesitate to follow up on this post.

jrauser · ‎Aug 25, 2020

We have also run into this issue and can be replicated with this body in postman using the v3 contacts batch add:

{
"inputs": [
{"properties": {"firstname": "Test Pass 1", "email": "test1pass5@hotmail.com"}},
{"properties": {"firstname": "Test Pass 2", "email": "test2pass2@hotmail.com"}},
{"properties": {"firstname": "Test 3 Fail", "email": "email3failhotmail.com"}},
{"properties": {"firstname": "Test 4 After Fail", "email": "test4pass@hotmail.com"}}
]
}

I am getting the expected 400 error for contact 3 but contacts 1,2,4 are still being added. The docs in v3 do not indicate the expected behavior on an error.

The workaround provided is working but would like some confirmation regarding the behavior. Is this expected or is this a bug?

APIs & Integrations