Could anyone help me understand why Hubspot is calculating Adjusted Open Rates as “Unique Reliable Opens / (Delivered - Unreliable Opens)”? I don’t understand the rational behind subtracting unreliable opens from the total delivered. Why not simply take the reliable unique opens and divide by the total delivered? Does an unreliable open inflate the delivery number…doesn’t seem like it would.
Hi @Otterbot
Its an interesting calculation for sure. My understanding is as follows.
The goal of adjusted open rate (%) is to increase the open rate by an amount proportional to the suspected unreliable opens.
The rate can be increased by either increasing the number of opens (for example by adding reliable and unreliable) or by reducing the number delivered.
I imagine HubSpot’s testing has found that the resulting open rate (%) is more accurate if they reduce the number delivered by the unreliable opens than by adding them to the reliable opens.
Hope this helps?
Hey @Phil_Vallender , hey @Otterbot , thanks for bringing up that topic as it is driving me kind of crazy 
Perhaps the problem lays in how the adjusted Open rate calculation is explained in the screenshot. Because after deeply looking into it, my team and I can’t understand how the adjusted Open rate, which excludes unreliable opens, can be higher than the initial rate. By excluding the segment with 100% open, the adjusted Open rate should be lower.
Here are our hypothesis on the topic:
- Unreliable email opens are not counted as “open” in Hubspot. They would only be taken into account for adjusted open rate calculation. Low probability.
- There is a hidden parameter to calculate the “unique delivered” for adjusted open rate, because the shown formula takes a lower number than the actual result. In that case, how is it calculated?
- There is an error in the calculation shown on Otterbot’s screenshot
Whatever the reason, I’ll be really grateful to the person who helps me understand the problem! Thanks in advance 
@AlexChocolatine, I’m not sure I am exactly following what you’re saying but I agree that there is an issue. @Otterbot & @Phil_Vallender In my emails it shows the unique adjusted opens (unique reliable opens, per the HS explanation) > unique opens …if unique reliable opens should be unique opens minus unique unreliable opens, how is it larger? That just doesn’t make any sense to me at all. The numerator should not be going up while the demoninator is going down. They should both be going down by the same amount.
Additionally, they don’t give the number of unique delivered which means you can’t back into this number. Originally I assumed the “delivered” number was unique delivered but if you try to back into unique unreliable opens by:
(unique reliable opens/adjusted open rate) = (unique delivered - unique unreliable opens) = A
Delivered - A = unique unreliable opens --> this answer for both my examples below is greater than the unique opens or unique reliable opens so delivered must include non-unique deliveries also (such as forwards).
I agree the formula should be:
(Total unique opens - unique unreliable opens)/(total unique delivered - unique unreliable opens) which should always give a lower open rate.
Two real examples:
Exmaple 1:
Delivered: 8,575
Exmaple 2:
Delivered: 4663
I would also really love some insight into this as it makes it very difficult for us to compare our open rates here to open rates in other programs and just to see the big picture and all the pieces. Really appreciate it!
Hey @MRussell28 (and everyone), I finally got an answer from Hubspot’s customer service after reaching out to them on that matter : automatic opens from Apple devices with iOS 15 are considered “unreliable” and are not counted as opened in the open rate.
So that explains why the adjusted open rate is always higher.
To take an example: if you send an email to 100 recipients, 60 of them have non-Apple devices (segment 1), and 40 of them have Apple devices (segment 2). For segment 1, you have 30 opens. For segment 2, info is missing, so Hubspot considers it’s a 0% open rate (as the tool is unable to tell you who has opened the email or not).
=> The basic open rate will be 30 / 100 => 30%
=> The adjusted open rate will be 30 / (100 - 40) = 30 / 60 => 50%
Adjusted open rate is probably closer to reality then!
PS: I feel like wording is yet really confusing in the app, and should probably be updated to indicate that automatic email opens are not counted as opens.
@AlexChocolatine Wow that makes much more sense! Thank you for sharing that with us, it’s much appreciated! What I’m still confused about is why my total opens are less than my reliable opens - am I missing something in that regard? If you look in my above examples - unique opens are less than unique adjusted opens.
That would be a good answer, except it doesn’t seem it’s what happens de facto. Checking the opens by person (they write the device and OS), I see many opens from apple mail, safari, etc, and they ARE calculated in the basic open rate. moreover, our open rates are not lower than before the IOS 15, quite the opposite. And with the adjusted open rates the numbers are crazy high. I just can’t understand how the adjusted open rate is higher.
Hey Sim88, perhaps the results you see are people that use Iphones / safari / etc., but not the last iOS ? Or people that use the last iOS but not the privacy option ? (I remeber that when I downloaded it it was a simple yes or no question, and I guess a lot of people are clicking on no - no - no when their iphone is showing pop-up questions… Could it be an explanation ?)
Hey Alex, this is what I finally got from support:
I’d like to clarify a few points about the Adjusted Open rate that would be important to flag here.
- I will not be able to see the exact numbers used to calculate your AOR until Version 2 is released. (yes, we will have an update on this version in a couple of months). I agree that the numbers for Unique adjusted opens and Total adjusted opens are not helping us understand how the AOR is being calculated. This will be corrected over time.
- The team clarified that the Adjusted Open Rate is expected to be higher because we are only counting the human opens and we are taking out the bot activity from the delivered emails before we calculate the open rate.
The Open Rate is showing you all human opens (because you have bot filtering turned on) divided by all the contacts that were delivered an email.
In looking at the Adjusted Open Rate, the denominator (all delivered emails) is first decreased, by subtracting the amount of bot opens (that you can’t see this number) and then that number is used to divide the human opens to calculate the open rate.
To give an example:
You have bot filtering turned on and you successfully deliver an email to 100 contacts. Your bot-filtered email opens show you have 20 human opens.
- 20 opens/100 delivered emails= 20% open rate
The AOR takes the Apple opens into account and subtracts them from the delivered emails to take them out of consideration.
We detect 10 Apple opens. This will decrease the 100 delivered emails to 90. We then calculate the open rate with this new number: 20 human opens / 90 = 22% open rate which is a higher open rate.
I had to sit with a Product Engineer to understand better how they were calculating this.
One thing I think is important to keep in mind for cases like this is that actually all of these numbers are estimates. There will always be contacts at the margin who, for example, did receive and open a customer’s email but whose security systems block image pixels and therefore don’t allow us to capture that open. Similarly, because of bot technology, there may be cases where an open that we deem unreliable was in fact done by a human or vice-versa. This number represents our best effort to give the customer an accurate idea of their engagement, but it’s not perfect that’s just a reality of the technology at work here.