How to get all records of the contact object connected to Data Factory

[NelsonS]

Hello community,

MARKETING & SALES ENTERPRISE

I have done with satisfaction the connection of the HubSpot application with Data Factory - DataLake, but at the moment of getting the information I only get half of my records of the Contacts object (530.913). How could I do to get 100% of the records of that object?

INPUT:

{
    "source": {
        "type": "HubspotSource",
        "query": "SELECT * FROM CONTACT_PROPERTIES_VALUES where Property_Name IN ('dni', 'firstname','lastname','email','celular','celular_2', 'telfono__celular', 'phone', 'mobilephone','telefono_fijo', 'lastmodifieddate');"
    },
    "sink": {
        "type": "DelimitedTextSink",
        "storeSettings": {
            "type": "AzureBlobFSWriteSettings"
        },
        "formatSettings": {
            "type": "DelimitedTextWriteSettings",
            "quoteAllText": true,
            "fileExtension": ".txt"
        }
    },
    "enableStaging": false,
    "translator": {
        "type": "TabularTranslator",
        "mappings": [
            {
                "source": {
                    "name": "Vid",
                    "type": "String"
                },
                "sink": {
                    "name": "Vid",
                    "type": "String",
                    "physicalType": "String"
                }
            },
            {
                "source": {
                    "name": "Property_Name",
                    "type": "String"
                },
                "sink": {
                    "name": "Property_Name",
                    "type": "String",
                    "physicalType": "String"
                }
            },
            {
                "source": {
                    "name": "Property_Value",
                    "type": "String"
                },
                "sink": {
                    "name": "Property_Value",
                    "type": "String",
                    "physicalType": "String"
                }
            }
        ],
        "typeConversion": true,
        "typeConversionSettings": {
            "allowDataTruncation": true,
            "treatBooleanAsNumber": false
        }
}

OUTPUT: 

{
    "dataRead": 29828810,
    "dataWritten": 20230974,
    "filesWritten": 1,
    "sourcePeakConnections": 1,
    "sinkPeakConnections": 1,
    "rowsRead": 530920,
    "rowsCopied": 530920,
    "copyDuration": 499,
    "throughput": 58.376,
    "errors": [],
    "effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (South Central US)",
    "usedDataIntegrationUnits": 4,
    "billingReference": {
        "activityType": "DataMovement",
        "billableDuration": [
            {
                "meterType": "AzureIR",
                "duration": 0.6,
                "unit": "DIUHours"
            }
        ]
    },
    "usedParallelCopies": 1,
    "executionDetails": [
        {
            "source": {
                "type": "Hubspot"
            },
            "sink": {
                "type": "AzureBlobFS",
                "region": "South Central US"
            },
            "status": "Succeeded",
            "start": "Aug 15, 2022, 6:34:37 pm",
            "duration": 499,
            "usedDataIntegrationUnits": 4,
            "usedParallelCopies": 1,
            "profile": {
                "queue": {
                    "status": "Completed",
                    "duration": 5
                },
                "transfer": {
                    "status": "Completed",
                    "duration": 492,
                    "details": {
                        "readingFromSource": {
                            "type": "Hubspot",
                            "workingDuration": 488,
                            "timeToFirstByte": 1
                        },
                        "writingToSink": {
                            "type": "AzureBlobFS",
                            "workingDuration": 0
                        }
                    }
                }
            },
            "detailedDurations": {
                "queuingDuration": 5,
                "timeToFirstByte": 1,
                "transferDuration": 491
            }
        }
    ],
    "dataConsistencyVerification": {
        "VerificationResult": "NotVerified"
    },
    "durationInQueue": {
        "integrationRuntimeQueue": 0
    }
}

Images:

Contact records already generated

Someone who can help me, please

[Jaycee_Lewis]

Hi, @NelsonS 👋 Let's see if the community can help out here. I know @johnelmer has done some related work. @RJanov @prosa @jmik do you have any experience here?

Best,

Jaycee