In this post, we get into our Confirmed Delivery feature and help you troubleshoot why you may be seeing low Confirmed Delivery among messages sent to your audience.
A Quick Definition
Confirmed Deliveries are an analytics feature available on our upgraded plans— including Growth, Pro, and Enterprise. This metric tracks the successful receipt of notifications per device.
Confirmed Delivery is available for both mobile and web notifications. This metric provides near-perfect Click-Through Rate (CTR) estimates and gives app developers peace of mind when it comes to understanding whether their notifications have been successfully received by an end user’s device.
How Does our Delivery System Work?
A Confirmed Delivery occurs when a device receives a notification through OneSignal. This metric can be found in the Message Report of a notification.
To fully understand Confirmed Delivery, you first need to comprehend how notifications are set to devices through OneSignal.
Push notification providers like us send notifications through Apple and Google’s push notification services— Apple Push Notification service and Firebase Cloud Messaging. The messages you send are delivered through APNs and FCM servers. However, without device-level message delivery confirmation, OneSignal can only confirm delivery to the servers.
However, with Confirmed Delivery, we are able to successfully acknowledge message receipt to the end user’s device through an extra network request from the device in question back to OneSignal servers.
The Importance of Confirmed Deliveries
Knowing which devices have successfully received your notifications gives you a more accurate understanding of true message delivery. This metric allows you to better benchmark the health of your messaging campaigns and additionally, troubleshoot your implementation.
In the absence of Confirmed Deliveries, you rely on CTRs as a way of understanding how subscribers are reacting to your messages. CTR is computed by dividing the number of confirmed notification opens by the total number of subscriber devices targeted. Unfortunately, some of the devices you target inevitably fall through the cracks..
The issue with CTRs is that they include “phantom subscribers” which are devices that have never directly unsubscribed through an action like deleting your app, that but are no longer active. Common scenarios for this phenomenon include when you stop using a device after switching it out for a new one.
As a result, these permanently inactive users will lead to an increasingly inaccurate CTR as time goes on. Over time, your subscribed users count can diverge from your actual audience count, as users inevitably change sessions, browsers, and devices.
Before we released our Confirmed Delivery feature, we had no way of accurately calculating the number of devices that actually received a company’s notification.
The Role of Our Robust Infrastructure
Confirmed Deliveries require providers to process an additional inbound request for every push notification. As a result, this creates a higher volume of traffic to the provider’s servers and thus requires robust infrastructure. Luckily, our infrastructure handles messages at one million notifications per second and simultaneously handles over one million confirmed deliveries.
Why is Confirmed Delivery Low?
Although Confirmed Delivery is a valuable metric that can inform and drive your push strategies, our users are often puzzled when they see trends of low Confirmed Delivery over time.
There are a handful of reasons you could be experiencing this pattern, so let’s get into some common scenarios.
Currently, we don't have a clear benchmark for the average percentage of users who don't receive a Confirmed Delivery. There are a few factors that may be at play here.
1. Inactive Devices
"In 2016, American smartphone owners used their phones for 22.7 months on average before upgrading. By 2018, that number had increased to 24.7.” ~ CNBC
Taking this statistic into consideration, we can assume that the average user updates their phones every two years. What this means is that any device with "last session greater than 17520 hours" is likely not receiving push. When users upgrade devices and retire their old ones, they don’t usually unsubscribe from all their apps and websites.
As a result, one reason a user may not be receiving your notifications is because their old device has been banished to a dusty drawer or tossed out in the trash. You may even have subscribed devices that haven’t visited your app in years!
Luckily, you can address this issue in OneSignal using Segments to check all the device records that haven’t opened your device in X amount of time using our “Last Session” filter. Once you identify these inactive subscribers, you can remove these devices from your audience.
2. Android Force Quit State
Another scenario in which your confirmed deliveries seem low is when Android devices put apps into a “force quit state.” Many of these devices put induce a "force stop" state when users swipe an app away. This is a unique roadblock, because for the majority of devices, swiping an app away does not stop notifications. However, for these select devices, notifications are stopped until you resend them with the app running.
When it comes to this issue, any push provider you use will encounter similar issues.
Due to the way these devices handle notifications, we sometimes see that Android audiences have lower confirmed deliveries in account of this manufacturer-level issue.
Certain Android device manufacturers have chosen to implement these force quit states in order to conserve device battery. Here are some manufacturers that handle notifications in ways that may disrupt notification delivery.
Huawei
Huawei devices run a modified version of Android OS called EMUI. By default, EMUI versions earlier than 4.1 automatically kill background apps every time the device's screen is off, unless they are marked as Protected Apps in the device's Battery Management page under Advanced Settings. As a result, users cannot receive notifications from apps that aren’t on the Protected Apps list.
Luckily, this is no longer an issue for devices running the EMUI 4.1 and later versions of the OS.
Sony
Some Sony Xperia devices use a Stamina Mode. Like Huawei's Protected App feature, Stamina Mode causes the device in question to automatically close background processes and network connections when the device's screen is off.
Luckily, Sony has eliminated Stamina Mode for devices that run Android 6 later versions of the OS.
ASUS
ASUS Android devices have an Auto-Start Manager that defaults to preventing applications from being woken up to process incoming notifications unless they are allowed in the Auto-Start Manager application.
Xiaomi
Xiaomi devices make sending notifications incredibly complex. For more information, consult expert advice.
Oppo
Oppo devices will close apps in the background or when a device’s screen is off, and prevent those apps from receiving notifications.
Samsung
New Samsung devices don’t show notifications from an app that hasn’t been opened recently unless it is manually white-listed deep in the device's settings.
3. Next Steps
All that said, if you remove older devices, you should generally expect to see is at least 60 percent Confirmed Delivery on Android and 80 percent Confirmed Delivery on iOS.
You may also want to send a push to distinct segments of your Android users and your iOS users and then check your Confirmed counts afterwards.
Start by setting up 2 Segments with:
1 - Last Session less than 720 hours and device type is Android
2 - Last Session less than 720 hours and device type is iOS
Sent a push to the Android users Segment. Then duplicate the push and send it to the iOS users.
After 24 hours check your Confirmed stats.
If you get zero Confirmed Deliveries in either segment, this could signal a significant issue.
Explore Additional Analytics Features
Delivery Status
Delivery status gives feedback on whether your message was successfully delivered, is still in the process of being delivered, or is experiencing another delivery issue related to push, email or SMS servers.
Email Message Reports
When you click an email report, you will see high-level stats like how the message was created, when it started sending, how long it took to fully send, and any throttling limits set.
Outcomes
Track events and actions within your app or site resulting from push notifications and in-apps from OneSignal.
OneSignal allows you to track various Outcomes —events or actions— resulting from push notifications and in-apps.
This includes means you can track important signals such as event counts ( how many users purchased a product), sums (useful for tracking revenue), or unique events (counted only once).
Our Commitment to Reliability
Meanwhile, here at OneSignal we'll continue to do our best to ensure that every notification developers send gets delivered as promptly and reliably as possible, in order to give you the feedback you need to stay on top of trends affecting push delivery.
How to Improve Message Deliverability
Check our documentation for instructions on how to track which devices are successfully receiving notifications.
View Confirmed Delivery Documentation
