Get A Free Site Audit to Understand the Impacts!
The way things were in analytics before browser privacy updates
Before we get into it, let’s have a quick look at how analytics used to work.
When a user came to the website, they were given a 1st party cookie (a small text file that identifies them between pages). This cookie allowed the user to be identifiable across sessions and so we could run reports that dealt with the concept of time, such as Cohort Analysis.
Over time, four scenarios were identified where this would break down and/or add noise into the reporting, but they were occurring in low enough numbers that we didn’t really worry about the utility of cookies as sticky identifiers for analytics tools. Those scenarios are:
- The user would buy a new device—so the cookies wouldn’t exist on the return visit.
- The user would reinstall their operating system/browser—so the cookies wouldn’t exist on the return visit.
- The user would delete the cookies between visitors—so the cookies wouldn’t exist on the return visit.
Taken together, this could result in a commonly seen 5-10% delta between the platforms and the back end transaction tracking. The important thing to note is that in each of these scenarios the user had to take explicit action in order to not be identified between visits. This is no longer the case.
How the analytics pot has been slowly brought to a boil
Since early 2019, browsers have, in increasing numbers, taken steps that can affect analytics, because they either restrict or delete cookies systematically between visits—possibly without the user even being aware this is happening. But now, rather than isolated incidents, entire user segments are displaying this behavior on a continuous basis. The above-mentioned scenarios have changed.
What’s happening with ITP as of the iOS14 update
Now, depending on how quickly the user returns to the site, there is an increased chance that the cookie will be deleted by Intelligent Tracking Prevention (either within 7 days or 24 hours, depending on the scenario). Today, if some additional step is not taken to leverage a solution such as server set cookies (which transitions how that cookie is set to something more stable) the canned, out-of-the-box analytics tool reports will be riddled with erroneous data.
Here are three ways data could be affected.
1. Retention Reports / Cohort Analysis have become inaccurate
Because of the changes, reports will indicate Retention Visitors numbers to be lower than actual, while “New” users will be higher (sometimes drastically so) than actual.
In a Cohort Analysis this will manifest itself as people “falling out” of an older cohort and appearing in a newer cohort, while actually being the same person in both. Problematic!
The danger here is if the reporting is being used to inform planning on things such as promotional or marketing spend. Spend may be shifted to try to “recapture” some of that retention traffic (even if it didn’t go anywhere to begin with). This could result in an avoidable hit to the bottom line margin.
2. Audience Segmentation intelligence may become diluted
Audiences/ Personas (and so on) are the people that have been identified as the key focus that the business is trying to attract, service, or grow. While the identification of these groups can be done real-time, these groupings are often identified by cross-referencing large amounts of data and ‘tagging’ a specific visitor as having affinity or behavior-modeling that resembles a group such as “Avid Rock Climbers,” “90s Alternative Fan,” or “Cat Video Lover.”
Depending on how the reporting platform does the mapping of these groupings to the customer segments, what may be found is that increasing amounts of these segments “go dark” or become otherwise unreachable over time.
While the cause of this could be the site is scaring away customers, the cause could also be that the underlying identifier that tags whether a user belongs to a group is being reset by the browser. So, on future visits, unless the customer does something to cause them to be segmented the same way, they will effectively “leave” the audience.
The danger here is that, depending on how the above plays out, marketers could mistakenly assume that their efforts are ineffective when they’re actually not. Based on the reports marketers receive, the marketing plan could either shift focus or double down on trying to capture/service a specific market segment when both of these activities could be erroneous. Resources might be wasted. The true best target audience/persona might be lost.
3. Device Identification Reports have become less dependable
Ever wonder where the iPad traffic went after Fall of 2019? Maybe it took a vacation? 🌴
In iPadOS 13, Safari’s user agent changed. Since analytics platforms use the user agent to classify the device, the fact that the user agent changed meant that (in the majority of cases) iPad traffic started to get classified as MacOS.
So it’s not that iPad users are leaving the Internet in droves, nor is it that a specific website is innately hostile to tablet users. The systems likely can’t distinguish between what is an iPad and what is a MacOS Desktop.
If this is a report that gets looked at often, it’s very possible to set a developer team to work attempting to identify what unseen reason that a specific domain gets low tablet traffic. It’s not necessarily that the iPad users are getting a worse user experience (but that is possible), it’s that the system can’t isolate that segment of the user base any longer.
How to mitigate these effects and next steps
Technically, mitigation of these and other effects can be tricky, but it can also be straight-forward, and this will depend on the specific implementations involved.
Generally, the goal is to get the reporting platform to run from a first party domain, storing data in a first party context issued by a server side command or web service. Some of the existing vendors may have solutions which could easily be leveraged, while others may require more engineering effort.
Analytically, you may be able to adjust reporting and internal processes to account for the variability that these changes inflict upon the analytics implementation. For example, you may reasonably expect to ‘not’ research where the iPad traffic went, because you have the proper context that it’s not broken. Having and responding to this increased context of the larger internet could make the difference between having an effective analysis or not.
Our cautionary tale continues
Things have changed, and analytics is different than it was prior to 2019. Now, there are far more edge cases to trip the unexpecting analyst up.
There are the changes outlined above, the changes affecting marketing measurement, and the cascade of recent changes all seeking to play havoc with an analysis, and possibly shift the course of action as a result.