In what order are results sorted in emails and engagemts requests?

[darius_veracity]

Hello,

I work on syncing engagements and emails to our database. I have read in some thread that I cannot rely on offset when fetching next results. Maybe those items are sorted by create date? 

https://legacydocs.hubspot.com/docs/methods/engagements/get-all-engagements

https://legacydocs.hubspot.com/docs/methods/email/get_events 

Then I could  save create date of last succesful item, find offset by create date and continue fetching fromt that.

[darius_veracity]

https://legacydocs.hubspot.com/docs/methods/email/get_events

reading this today and noticed

"Events will be returned in reverse-chronological order."

and I see in responce there is only one time field "created" so looks like it is clear about email events endpoint.

Still waiting for clear info about https://legacydocs.hubspot.com/docs/methods/engagements/get-all-engagements

[dennisedson]

Hey @darius_veracity 

Have to concur with the threads you have read on not relying on the offset. 

I do believe that they are returned by create date with oldest first.

@Mike_Eastwood , any additional thoughts?

[darius_veracity]

Found that engagements are not ordered by dates. Made script:

$printTimestamps = false;

$hasMore = true;
$count = 0;
$offset = '';
while ($hasMore) {
    $json = file_get_contents(sprintf('https://api.hubapi.com/engagements/v1/engagements/paged?hapikey=%s&limit=250%s', $hapiKey, $offset));
    $count++;
    $engagements = json_decode($json, true);

    if ($printTimestamps) {
        printTimestamps($engagements);
    }

    checkIds($engagements);

    $offset = '&offset=' . $engagements['offset'];
    $hasMore = $engagements['hasMore'];
    echo "pages count $count \n";
}

function checkIds(array $engagements)
{
    $lastId = null;
    foreach ($engagements['results'] as $result) {
        if ($result['engagement'] > $lastId) {
            $lastId = $result['engagement']['id'];
            echo $result['engagement']['id'] . "\n";
        } else {
            echo sprintf("Echo id not increasing last: $lastId current: %s \n", $result['engagement']['id']);
        }
    }
}

function printTimestamps(array $engagements)
{
    echo "Timestamp created at | Created at | Last updated | Timestamp in date \n";
    foreach ($engagements['results'] as $result) {

        $engagement = $result['engagement'];

        echo sprintf(
            "%s %s %s %s \n",
            $engagement['createdAt'],
            date('Y-m-d H:i:s', $engagement['createdAt'] / 1000),
            date('Y-m-d H:i:s', $engagement['lastUpdated'] / 1000),
            date('Y-m-d H:i:s', $engagement['timestamp'] / 1000)
        );
    }
}

Looks like on my tested small sample they are sordet by ids.

If I turn on printing dates, I get

Timestamp created at | Created at | Last updated | Timestamp in date
1597938433816 2020-08-20 18:47:13 2020-08-29 06:42:59 2020-08-20 18:47:13
1597940016257 2020-08-20 19:13:36 2020-08-29 14:21:27 2020-08-20 19:13:36
1600284997088 2020-09-16 22:36:37 2020-09-16 22:36:46 2020-09-16 22:36:36
1600285094689 2020-09-16 22:38:14 2020-09-16 22:38:16 2020-09-16 22:38:14
1600362665059 2020-09-17 20:11:05 2020-09-17 20:11:17 2020-09-17 20:11:04
1600368412906 2020-09-17 21:46:52 2020-09-17 21:46:56 2020-09-17 21:46:52
1602686923604 2020-10-14 17:48:43 2020-10-14 17:48:49 2020-10-14 17:48:43
1604947383904 2020-11-09 20:43:03 2020-11-09 20:43:22 2020-11-09 20:43:03
1604958011349 2020-11-09 23:40:11 2020-11-09 23:40:22 2020-11-09 23:40:11
1604958061292 2020-11-09 23:41:01 2020-11-09 23:41:19 2020-11-09 23:41:01
1605025503717 2020-11-10 18:25:03 2020-11-10 18:25:31 2020-11-10 18:25:03
1605034800146 2020-11-10 21:00:00 2020-11-10 21:00:08 2020-11-10 21:00:00
1605034057113 2020-11-10 20:47:37 2020-11-10 20:47:47 2020-11-10 20:47:36
1605035381927 2020-11-10 21:09:41 2020-11-10 21:09:49 2020-11-10 21:09:41
1605034557916 2020-11-10 20:55:57 2020-11-10 20:56:02 2020-11-10 20:55:57
1605035339812 2020-11-10 21:08:59 2020-11-10 21:09:04 2020-11-10 21:08:59
1605037138765 2020-11-10 21:38:58 2020-11-10 21:39:04 2020-11-10 21:38:58
1605038837596 2020-11-10 22:07:17 2020-11-10 22:07:23 2020-11-10 22:07:17
1605038871433 2020-11-10 22:07:51 2020-11-10 22:07:53 2020-11-10 22:07:51
1605041282175 2020-11-10 22:48:02 2020-11-10 22:48:36 2020-11-10 22:48:02
1605636489874 2020-11-17 20:08:09 2020-11-18 20:14:24 2020-11-17 20:08:09
1611679332177 2021-01-26 18:42:12 2021-01-26 18:42:34 2021-01-26 18:42:11
1611772374437 2021-01-27 20:32:54 2021-01-27 20:33:00 2021-01-27 20:32:54
1611773125592 2021-01-27 20:45:25 2021-01-27 20:45:40 2021-01-27 20:45:25
1612456669899 2021-02-04 18:37:49 2021-02-04 18:38:15 2021-02-05 17:00:00
1612457253924 2021-02-04 18:47:33 2021-02-04 18:47:35 2021-02-04 18:47:30
1612456589571 2021-02-04 18:36:29 2021-02-04 18:36:33 2021-02-06 02:00:00
1612456589571 2021-02-04 18:36:29 2021-02-04 18:36:33 2021-02-06 02:00:00
1612456589571 2021-02-04 18:36:29 2021-02-10 17:39:09 2021-02-06 02:00:00
1612457847790 2021-02-04 18:57:27 2021-02-04 18:57:30 2021-02-04 18:57:27
1612457172618 2021-02-04 18:46:12 2021-02-04 18:46:35 2021-02-04 18:46:12
1612458520028 2021-02-04 19:08:40 2021-02-10 17:30:21 2021-02-10 17:30:00
1613499188880 2021-02-16 20:13:08 2021-02-16 20:13:25 2021-02-16 20:13:08
1614586467542 2021-03-01 10:14:27 2021-03-01 10:14:27 2021-03-01 10:14:19
1615225031026 2021-03-08 19:37:11 2021-03-08 19:37:50 2021-03-08 19:37:10
1619096007393 2021-04-22 15:53:27 2021-04-22 15:53:34 2021-04-22 15:53:27
1619119173088 2021-04-22 22:19:33 2021-04-22 22:19:38 2021-04-22 22:19:32
1619196726693 2021-04-23 19:52:06 2021-04-23 19:52:28 2021-04-23 19:52:06
1619197276699 2021-04-23 20:01:16 2021-04-23 20:01:23 2021-04-23 20:01:16
1619197262897 2021-04-23 20:01:02 2021-04-23 20:01:20 2021-04-23 20:01:02
1619197292445 2021-04-23 20:01:32 2021-04-23 20:01:38 2021-04-23 20:01:32
1619616423812 2021-04-28 16:27:03 2021-04-28 16:27:04 2021-05-03 17:00:00
1619802398959 2021-04-30 20:06:38 2021-04-30 20:06:38 2021-04-30 20:06:35
1619802407056 2021-04-30 20:06:47 2021-04-30 20:06:57 2021-04-30 20:06:46
1619802776261 2021-04-30 20:12:56 2021-04-30 20:13:02 2021-04-30 20:12:56
1619802855404 2021-04-30 20:14:15 2021-04-30 20:14:51 2021-04-30 20:14:15
1619803093518 2021-04-30 20:18:13 2021-04-30 20:18:28 2021-04-30 20:18:13
1619813167017 2021-04-30 23:06:07 2021-04-30 23:06:18 2021-04-30 23:06:06
1619813048749 2021-04-30 23:04:08 2021-04-30 23:04:24 2021-04-30 23:04:08
1620032488725 2021-05-03 12:01:28 2021-05-03 12:01:30 2021-05-03 12:01:28
1620032363633 2021-05-03 11:59:23 2021-05-03 11:59:25 2021-05-03 11:59:23
1620040246531 2021-05-03 14:10:46 2021-05-03 14:10:47 2021-05-03 14:10:46
1620040314664 2021-05-03 14:11:54 2021-05-03 14:11:55 2021-05-03 14:11:54
1620079892351 2021-05-04 01:11:32 2021-05-04 01:11:37 2021-05-04 01:11:32
1620079522147 2021-05-04 01:05:22 2021-05-04 01:05:36 2021-05-04 01:05:21
1620079830013 2021-05-04 01:10:30 2021-05-04 01:10:33 2021-05-04 01:10:29

Most of the time dates increase, but there is a row where next row dates are lower:

1620032363633 2021-05-03 11:59:23 2021-05-03 11:59:25 2021-05-03 11:59:23