File de-duplication options not working

Developer Support APIs & Integrations
1

Hello,

It seems that in v3 of the Files API, the de-duplication options, although provided as per the documentation, are ignored, and a copy of the file is created every single time.

This happens whether I use the Python client, or CURL.
For example, when I run the following command twice in a row, I get a different file ID, even though the file is the same, as well as all the other parameters. This seems to happen regardless of the folderPath, the access level of the file uploaded, a

curl -XPOST https://api.hubapi.com/files/v3/files --header ‘authorization: Bearer YOUR_API_TOKEN’ -F folderPath=/test -Foptions=‘{“access”: “PUBLIC_NOT_INDEXABLE”, “duplicateValidationScope”: “EXACT_FOLDER”, “duplicateValidationStragegy”: “RETURN_EXISTING”}’ -F file=@/tmp/example.txt

Also, the documentation does not say how de-duplication is performed. Is it using file names, MD5 or SHA1 hashes, or some other method?

2

Hey, @wollivier :waving_hand: Thanks for flagging this. I’ll be glad to set up a test and reproduce on my end.

A few questions,

  • have you tried using ENTIRE_PORTAL vs EXACT_FOLDER?
  • have you tried using REJECT vs RETURN_EXISTING?

I ask out of curiosity, not because you didn’t provide enough information.

Thanks for the additional details! — Jaycee

3

Hi @Jaycee_Lewis ,
I’ve just given the various options a go, and found that whatever the combination of options was, the API would always create a new file, with incrementing IDs.
There was one strange thing I observed, which might be on purpose, but not documented, so thought I’d mention it here:

  • The first 4 copies of the file are renamed by adding a `-{1..4}` suffix to the basename of the file, like so:

    example-1.txt, example-2.txt

    , etc.

  • The following copies of the file are renamed by adding a textual timestamp to the basename of the file, like so:

    example-Apr-26-2023-04-15-53-6762-PM.txt

So it genuinely seems that the feature is not working at all.
Regards,
Will