topic Re: Occasional non-association of a contact and a deal via Ecomm bridge in APIs & Integrations

Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Sat, 13 Apr 2019 20:20:04 GMT

I ran a simple test recently sending about a 100 deals to our test portal via Ecomm bridge. Out of those, 32 deals ended up in Hubspot with no contact associated.

I checked, and each and every contact for those 32 deals was created (Original source drill-down 1 = INTEGRATIONS_SYNC), it's just the association isn't there.

I'm having a hard time pinpointing a problem, and all API requests were successful, maybe someone has encountered these issues before? Any advice is welcome, thanks!

Details: portal ID 4455895, sample deal ID 678862257 and customer vid 4651.

Re: Occasional non-association of a contact and a deal via Ecomm bridge

IsaacTakushi — Wed, 17 Apr 2019 16:56:48 GMT

Hey, @balabanov.

Apologies for the delayed response.

I haven't seen reports of this issue recently, but I'd like to investigate further.

Do you happen to have logs of the sync messages you sent for some of the objects missing associations? I'd like to at least see the externalObjectId values you used in an attempt to associate contacts and deals.

For example:

{
  "storeId": "your-store-id",
  "objectType": "CONTACT",
  "messages": [
    {
      "action": "UPSERT",
      "externalObjectId": "contact-id-1234",
      "properties": {
        "firstname": "abc",
        "lastname": "xyz",
        "email": "redacted@example.com"
      },
      "associations": {
        "DEAL": [
          "deal-id-5678"
        ]
      }
    }
  ]
}

and

{
  "storeId": "your-store-id",
  "objectType": "DEAL",
  "messages": [
    {
      "action": "UPSERT",
      "externalObjectId": "deal-id-5678",
      "properties": {
        "dealstage": "appointment_scheduled",
        "amount": 1234,
        "name": "Redacted Example Deal"
      },
      "associations": {
        "CONTACT": [
          "contact-id-1234"
        ]
      }
    }
  ]
}

Re: Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Wed, 17 Apr 2019 17:28:13 GMT

Sure thing @IsaacTakushi

[D 2019-04-16 15:52:47.228] Send messages: [{'integratorObjectId': '67ac3a84-4c66-4254-a96a-70be29ff7885', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'firstname': 'Nicole', 'lastname': 'Bailey', 'city': 'West Matthewland', 'email_0': '806210@testlab2.com', 'phone': '+19074795503', 'state': 'GA', 'country': 'US', 'zip': '90466', 'date_of_birth': ''}}]
[D 2019-04-16 15:52:47.317] Send messages: [{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'dealname': '1253', 'ip__ecomm_bridge__order_number': 1253, 'amount': '1562.03', 'createdate': 1555429962000, 'dealstage': 'checkout_completed', 'hs_assoc__contact_ids': '67ac3a84-4c66-4254-a96a-70be29ff7885'}}]
[D 2019-04-16 15:52:47.400] Send messages: [{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef--7b58c9bf-3c8b-4251-a918-678c079763d1', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'hs_assoc__deal_id': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'hs_assoc__product_id': '54fd39aa-1662-4fbe-bd9b-23df058c6274', 'name': 'Angove Organic 2016 Sauvignon Blanc', 'quantity': '45', 'price': '15.00'}}]
[D 2019-04-16 15:52:47.525] Send messages: [{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef--94a9c480-a466-4e53-bd80-4318d141dafa', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'hs_assoc__deal_id': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'hs_assoc__product_id': '38795ba6-6653-4702-b783-2522af3f64fb', 'name': '2017 Chardonnay Carneros', 'quantity': '20', 'price': '11.00'}}]

The relevant ids are:

Portal ID 5750998
Deal ID 683619340
Contact ID 7701

Re: Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Wed, 17 Apr 2019 17:35:16 GMT

Here are more recent ones, from a new test portal 5750998, deal ID 683619340 and contact ID 7701

{'integratorObjectId': '67ac3a84-4c66-4254-a96a-70be29ff7885', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'firstname': 'Nicole', 'lastname': 'Bailey', 'city': 'West Matthewland', 'email_0': '806210@testlab2.com', 'phone': '+19074795503', 'state': 'GA', 'country': 'US', 'zip': '90466', 'date_of_birth': ''}}

{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'dealname': '1253', 'ip__ecomm_bridge__order_number': 1253, 'amount': '1562.03', 'createdate': 1555429962000, 'dealstage': 'checkout_completed', 'hs_assoc__contact_ids': '67ac3a84-4c66-4254-a96a-70be29ff7885'}}

{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef--94a9c480-a466-4e53-bd80-4318d141dafa', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'hs_assoc__deal_id': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'hs_assoc__product_id': '38795ba6-6653-4702-b783-2522af3f64fb', 'name': '2017 Chardonnay Carneros', 'quantity': '20', 'price': '11.00'}}

{'integratorObjectId': '1d579b44-71be-4c8d-ad76-d055476e2aef--7b58c9bf-3c8b-4251-a918-678c079763d1', 'action': 'UPSERT', 'changeOccurredTimestamp': 1555429962000, 'propertyNameToValues': {'hs_assoc__deal_id': '1d579b44-71be-4c8d-ad76-d055476e2aef', 'hs_assoc__product_id': '54fd39aa-1662-4fbe-bd9b-23df058c6274', 'name': 'Angove Organic 2016 Sauvignon Blanc', 'quantity': '45', 'price': '15.00'}}

Re: Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Wed, 17 Apr 2019 17:36:55 GMT

For some reason, I'm getting "This reply was marked as spam and has been removed. If you believe this is an error, submit an abuse report." when trying to post a JSON snippet 😞

I already reported it, any chance you can pull my message out of the spam can? I can use a gist or pastebin if it doesn't work out

Re: Occasional non-association of a contact and a deal via Ecomm bridge

IsaacTakushi — Fri, 26 Apr 2019 18:29:16 GMT

Hi, @balabanov.

Sorry for dropping the ball on this. Thanks for providing those examples!

Are you creating the contact records before the deals?

I just ran some tests where deal sync messages triggered 204 responses even if the contact specified in the hs_assoc__contact_ids property had yet to be created. Per the description for this association property: "Deals are not required to be associated to contacts, so an invalid value for this property will not prevent the successful creation of a deal."

If I created the contacts before syncing the deals, however, the objects were successfully associated.

Re: Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Fri, 26 Apr 2019 19:05:23 GMT

Thanks for your input @IsaacTakushi

Yep, we create contact first, then the deal, then the line item. We're seeing this non-association constantly and frankly, it's a bit disappointing.

We came up with a minimal test case (below) and started introducing delays between contact and deal creation. Up until we ramp up the delay to 5 seconds we see non-assocs. From where I stand, it's just some race condition or just a slow endpoint

import json
import datetime
import time

from urllib.parse import urlencode
from urllib.request import Request, urlopen


BASE_URL = 'https://api.hubapi.com/'
HS_KEY = 'd4e6cdb1-wiped-for-security-reasons'

START_FROM = 50
NUMBER = 10
DELAY = 5.0


def upload_fake(start_from, n):
    for i in range(start_from, start_from + n):
        si = str(i)
        print(si)

        contact_id = 'fake-contact-' + si
        deal_id = 'fake-deal-' + si

        contact = {
            'integratorObjectId': contact_id,
            'action': 'UPSERT',
            'changeOccurredTimestamp': unixtime_now(),
            'propertyNameToValues': {
                'firstname': 'Name' + si,
                'lastname': 'Surname' + si,
                'email_0': 'user' + si + '@fakeusers.com',
                'city': 'Gotham City',
                'state': 'NY',
                'country': 'US',
            }
        }
        ecomm_sync_contacts([contact])
        time.sleep(DELAY)

        deal = {
            'integratorObjectId': deal_id,
            'action': 'UPSERT',
            'changeOccurredTimestamp': unixtime_now(),
            'propertyNameToValues': {
                'dealname': 'deal' + si,
                'amount': '100.00',
                'dealstage': 'checkout_completed',
                'hs_assoc__contact_ids': contact_id,
            }
        }
        ecomm_sync_deals([deal])
        time.sleep(DELAY)


def unixtime_now():
    UNIX_EPOCH = datetime.datetime(1970, 1, 1)
    dt = datetime.datetime.utcnow()
    ut = int((dt - UNIX_EPOCH) / datetime.timedelta(seconds=1)) * 1000
    return ut


def ua_put_json(suburl, payload):
    url = BASE_URL + suburl + '?hapikey=' + HS_KEY
    body = json.dumps(payload).encode()
    req = Request(url,
                  data=body,
                  headers={
                    'Content-Type': 'application/json',
                    },
                  method='PUT')
    resp = urlopen(req).read().decode()
    # print(resp)


# --- Ecommerce Bridge sync -----------------------------------------------
def ecomm_sync_contacts(contacts):
    r = ua_put_json(
        'extensions/ecomm/v1/sync-messages/CONTACT', payload=contacts)
    return r


def ecomm_sync_products(products):
    r = ua_put_json(
        'extensions/ecomm/v1/sync-messages/PRODUCT', payload=products)
    return r


def ecomm_sync_deals(deals):
    r = ua_put_json(
        'extensions/ecomm/v1/sync-messages/DEAL', payload=deals)
    return r


def ecomm_sync_line_items(items):
    r = ua_put_json(
        'extensions/ecomm/v1/sync-messages/LINE_ITEM', payload=items)
    return r


def main():
    upload_fake(START_FROM, NUMBER)


main()

Re: Occasional non-association of a contact and a deal via Ecomm bridge

IsaacTakushi — Mon, 29 Apr 2019 13:42:54 GMT

Hey, @balabanov.

I took a closer look at the contacts and deals you shared and found that the deal records were created before the contact records, causing the failed associations.

Contact 4651, has the createdate value1555172304098 (4/13/2019, 12:18:24 PM), however deal 678862257 was created 6 milliseconds earlier at 1555172304092.

Similarly, although you sent the sync message to create contact 7701 (externalObjectId: 67ac3a84-4c66-4254-a96a-70be29ff7885) first, it was actually created at 1555429968517 (4/16/2019, 11:52:48 AM) while deal 683619340 (externalObjectId: 1d579b44-71be-4c8d-ad76-d055476e2aef) was created 6 seconds earlier at 1555429962000 (4/16/2019, 11:52:42 AM).

The Ecommerce Bridge creates objects asynchronously, so these associations can and sometimes do get dropped in initial syncs (especially since deals tend to be created more quickly than contacts), but I agree that 5-6 seconds is a long time to wait. I will share your pain with the team and will update here with any actions they take.

As a possible alternative, I also know that the team has also made improvements to the speed and flexibility of the Ecommerce Bridge V2, which is now in developer preview.

Re: Occasional non-association of a contact and a deal via Ecomm bridge

balabanov — Mon, 29 Apr 2019 14:17:23 GMT

Thank you @IsaacTakushi , we'll run the tests using v2 asap, and I'll let you know the outcome.

I wonder, is there some reference Ecomm bridge API implementation to compare ourselves against? It's hard to imagine that nobody faced this association issue before, I still second-guess whether we missed something.

Re: Occasional non-association of a contact and a deal via Ecomm bridge

IsaacTakushi — Mon, 29 Apr 2019 19:40:39 GMT

Sounds good, @balabanov.

We currently don't have any example implementations, but I have worked with other integrators who have also encountered association issues because deals were created before contact records. I don't think you've missed anything; it is an unfortunate reality of the asynchronous nature of the Bridge. I'll continue to advocate to the team — I hope we can come up with a better way to handle these types of issues going forward.

Starting tomorrow, I will be out of the office until May 8, 2019. I can address any additional comments when I return, however if you have a pressing issue, please feel free to create a new post and cite this discussion. Thanks for your understanding!