HubDB imports - how to connect the dots

Developer Support APIs & Integrations
,
1

Hoping to get some help on a connection that is eluding me at present - how to create a practical setup to automate the input of HubDB files using custom workflow steps (server side python).

The API docucuments at CMS API | HubDB (hubspot.com) give details of the import API call, suggesting that a file name and an import config file are required. I have no issues with the config file so far, but generating a valid file spec is causing me problems.

I’m attempting to use a user form to upload a file that give me the starting poing for the import. The file property type - after the form submit - gives an https:// file URL which, although complicated with signing, is apparentlyly valid (I can use it to access and download the file that I uploaded via the form.)

But offering this file URL to the import API call (using python requests/file option) simply fails in all the permutations I have tried. So the question (unless I am thinking about this incorrectly) is how to manipulate the file property URL to satisfy the API parameter for the file spec needed in the HubDB import call?

There do not seem to be any fully worked examples of using the import API with a “real” file spec in the community or in a general Google search that I can find.

All help gratefully received.

Steve

2

Hey, @SteveHTM :waving_hand: Thanks for your question. The Import API can be a bit finicky at times.

For future posts, adding an example of your request URL, body + the specific response and errors returns are very helpful for the community to assist you.

Question:

  • Is this the documentation you are following? — Add table rows. Which links to this section? — Import rows from CSV
  • If so, can you double-check your import request header to make sure it includes a value of multipart/form-data? The endpoint will reject your requests without this being set

This might require two steps based on your use case:

  • One step to download the file from the URL into your server’s temporary storage
  • One step to upload the CSV with the value of multipart/form-data

Have fun building! — Jaycee

3

Thank you for the prompts @Jaycee_Lewis. Yes I was working on the HubDB import API Import rows from CSV. There’s a less than normal amount of help for this particluar API since you can’t try it on the endpoint page and there don’t seem to be many examples that are easy to find.

At the stage I wrote the post above, I had not even got to the execution of an API call since the setup for such a call required the import file details that I could not fathom. The documentation references ‘local files’ - but these are not accessible when working in custom code steps on the server side (not even temporary files as its a read-only file system environment.) And you can’t use authenticated API calls like this on the client side. Definitely seems like a need for a more realistic example here in the docs!

The resolution of this for me turned out to come in two steps:

- Access to the uploaded CSV file from the form entry through it’s protected URL. This is read into a local array and decoded appropriately. In python requests format this looks like:

#get the uploaded CSV file as a string

response = requests.request(“GET”, filePath, headers=headers)

response.encoding = ‘utf-8’ # fix for this type of file

csv = response.text

- And then using the content array to substitute for the missing file spec in the import API parameters:

files= {

‘config’: (None, json.dumps(import_config), ‘text/strings’),

‘file’: (fileName, csv, ‘text/strings’)

}

post import request

response = requests.request(“POST”, import_url, files=files, headers=import_headers)

Note that request headers set for this call should only include the authentication and not the content type (otherwise you get a very opaque 400 error). I think the requests package sets things behind the scenes and does not require help.

Now I seem to have unpicked all the little details that I was looking for to connect the dots between the file upload and the following import. I hope the explanation is useful to others.

My wish now is for the config JSON in the import to support full column header label to HubDB column mapping rather than relying on hard coded column index numbers. This would reduce fragility of the user process. But that’s perhaps an idea for another day.