OneSignal Webinars: Advanced Analytics

In this webinar, OneSignal's Head of Product, Zack Hendlin, goes over how advanced analytics powered by OneSignal can help increase ROI. The transcript of the webinar can be found below. Register for our webinars today!

[00:00]

Today, we're going to be covering Advanced Analytics. As a little bit of background, we'll talk about OneSignal and how you can figure out what outcomes you want to drive and track, and then how to actually do that at a technical level.

My name is Zack. I run product here at OneSignal. One of the great things that I get to do is go and talk to our customers and figure out which really useful and valuable to folks. One of the gaps we realized is a lot of folks know that sending push notifications is important because they know that's a good way to bring users back to their app or back to their website, but they have a hard time measuring and connecting all of that up with their own analytics systems.

So, if you use Google Analytics, we do have a built-in integration. But if you actually want to ask more sophisticated questions, like how many users are directly clicking on the notification or how many people did they send a notification to and did it directly click on it, but did use my app and close time proximity to that. So, I could still think of it as being influenced by my notification campaign or by my marketing campaign.

So, we're going to cover that today. We'll cover how you can go even beyond that with custom outcomes, which let you track anything that you might want in your app or website. And then we'll open it up for a bit of Q and A.

So, you should all be able to see my screen and the slides I'm going through. And then I'll jump into a live demo in just a second. Cool.

So, 30 seconds about OneSignal. We actually started out as app developers wanting to find a better way to send push notifications. I used to work at Facebook and LinkedIn for a number of years. One of the biggest drivers for getting users back on a daily basis was push notifications, right? To alert users that there was content that was relevant for them, that was timely, that they might want to check out.

So, the same experience was true with the early days of OneSignal where we realized that actually to build apps and get people to use them, you really need to find good ways to keep folks engaged and let them know about timely and relevant information. In fact, at big tech companies, there's teams of dozens of folks just working on optimizing notifications. So, we're trying to take that and make that power and that kind of optimization available to everybody.

Cool. So, let's take a quick view across OneSignal's advanced analytics capabilities. I'm going from sort of simplest on the left to sort of more sophisticated on the right. So, when you send a notification, you can obviously go in our dashboard under the delivery tab and just see all the statistics there. You also can pull that data down using our View Notifications API. And then, it effectively allows you to get all the data that you would want to know at a macro level, how many folks got this notification, how long did it take to send, what was the click-through rate on that notification and so on. So, that's where sort of most folks start by just looking at click-through rates in the dashboard or via the API if they wanted to do it that way.

The next level of sophistication is to actually say, well, I want to be able to know who of my users clicked on a specific notification, who is taking a particular action. The value of that is it lets you actually start updating your data model and your backend systems to build a richer profile of who is interested in what content or responsive to what types of campaigns. So, you could do that with the Notification History API.

Now, the Notification History API is only available on our paid plans as our outcomes, because to store all this data and maintain it takes us a bit more storage and bandwidth than we can provide for the basic offering over here.

So, Notification History API gives you user level who clicked on and who received a message. That's only retained for seven days because it's a lot of data. And typically, folks accessing it via the API would maybe pull it down once a day.

And then outcomes, which is what I'm going to talk about today. The great thing about outcomes is they let you go from who clicked on my notification or how many clicks did I get to actually instrument anything that you want on your website or your app. That can be who added an item to their cart for an e-commerce site, who shared a piece of content if you're on social media, who replied back to this message.

OneSignal Advanced Analytics

[05:04]

So, that actually takes you from looking at notifications. It's just driving clicks or opens up your app or website to actually say, what are the outcomes I care about? Because you could imagine there could be a situation where you have a lower click-through rate. The messaging, the way you've tailored your messaging is really good in sending people to take action. Let's say it was a discount or something like that. Or very specific to a particular style that might not be right for everybody, but for the people who click on it and are interested, it's going to really resonate with them and they're going to be more likely to purchase. So, we'll look at that in just a sec.

The other really interesting thing about outcomes is that they can be measured in three ways using OneSignal. You can say how many people directly clicked on something or how many people took an action - open their app, purchase something within a time window or attribution window of when I sent them a notification - or how many folks were not in either of those two buckets and just took an action beyond what my attribution window was.

I see we've got a quick question. Yes, a recording of this will be made later available on YouTube. We’ll post that to the OneSignal YouTube page. Thanks for the question.

So, the really big thing to think about here is that someone just clicking on your notification is not actually what you're typically trying to drive as a business. You're trying to drive an action. If you're a social media company, you're probably trying to drive time on your site or number of posts that someone views or number of posts that they create. If you're a video site, you care about how many videos they watch or how much time they spend watching video. If you're an e-commerce site, you probably care about items added to cart or dollar value of things purchased. So, outcomes are really easy to visualize and export as you'll see in just a sec. They're going to be available on iOS and web in the next two weeks and they're already available today on Android.

So, these are examples of elements of the conversion funnel that you might want to track as people come to your site or open your app and drive a result that you care about. So, let's dive into the dashboard so we can actually see how all this works. I'm going to go and create a message and it's just going to be, let's say I'm taking an e-commerce example, so 'Thanksgiving sale starting now! Best deals the year - they won't last long!' Great.

If I wanted, I could make this a little bit prettier and can throw in, let's say a quick image. Cool. I've used OneSignal’s embedded image search functionality here just to make this look a little nicer. Great. And then I maybe want to send folks to my website. I could also have this via deep link if I wanted, but for the sake of simplicity, I'll just leave it on my website. Great. So, now I'm going to send this message and I'm going to-- I see that it shows up here as a web push for me. I also see here that it shows up. I see that it shows up on my phone.

So, I've now got this notification and I can look at click-through rates and all that, but I'm actually wanting to drive something more interesting. I want to drive an outcome that I care about. So, I'm going to click on this notification. And then I'm going to drive an outcome of, let's say Add to Cart. So, outcome name is going to be Add to Cart part and I'm going to, let's say add three items to my cart.

[09:49]

Now, when I go and add that item here, I see that there are three items that are added to my cart. Since I actually had my click destination as my website and not my app, but the items that I added to my cart were in the app, I see that those items added to my cart - there's three items - are influenced, not direct because the way we count direct is if someone directly clicks on the notification and then triggers that outcome within that session. In this case, I sent someone to my website and then they later came to my app and triggered these Add to Cart action. So, we call them influenced.

So, a quick way of visualizing this is if a user clicks on a notification and fires an outcome within that session, that is direct. If a user doesn't click on the notification and have the outcome within that session, but does have it within the attribution window, we call that influenced. And then, if they fire an outcome in your app or on your website beyond the attribution window, we say that's unattributed. You’ll be able to see all this stuff in the dashboard.

So, there's three outcomes that we provide out of the gate where you don't have to write any code, number of clicks on a notification, straight forward enough, number of sessions, so the number of times that folks open your app. usually, sessions and clicks will be pretty close, but it's possible that you send someone to your website instead of to your app, in which case, you have a click to your website, but there wasn't actually a session that was created in your app. And then finally, session duration, which is how long someone has spent in your app once they clicked on that notification. So, that's sort of what we build in by default.

I'll give an example here of session duration by sending-- but by looking at this notification I sent and seeing that there's 37 seconds of time that I had the app open for because of that notification, in close time proximity to notification. This will sum across all folks who get this specific notification. I can also cut this here by direct and influenced as well as by different platforms and different time ranges. So, that gives you a quick example.

I'm going to do something now that's a little bit more interesting, which is actually use this to track the dollar value of purchases. So, I'm going to go through, create a new message here. I'm going to say, 'Find out who won today's soccer game'. Let's go back to e-commerce example, so '20% of the best deals of the season'. Okay. So, I'm sending this notification. Again, if I want, I can make it look pretty by throwing in a quick image from web search. Cool. It looks nice. And I'm going to send it.

So, now I'm going to actually fire an outcome on my phone of three different types. So, I'm going to show you what I'm doing on my phone. I'm going to fire an outcome which is purchase amount. So, here you can see purchase amount. I'm going to put, let's say 5 for $5. I'm going to fire that outcome with the value.

So, there are three types of outcomes. There are these regular outcomes, which is a name. There's outcomes with values, which is a name and a value like purchase amount and 5 would be an outcome with the value.

And then there's unique outcomes. Unique outcomes are outcomes where even if I fire that outcome ten times in a session, we're only going to come at once. So, that's useful for saying like, did a user upgrade? Yes or no? Rather than did they hit the upgrade button five times. It's also useful for, did someone invite their friends to the app, whether they invited one friend or 15 friends? I might only want to count the outcomes.

Creating Unique Outcomes

[15:33]

So, I've now got this outcome with a value when I fire it with a value of 5. I'm going to fire it also with a value of 10. I'm also going to create a unique value which says 'Invite friends'. For invite friends, I'm going to do that as a unique outcome. So, I'm going to send that outcome a lot of times and you're actually going to see it's only going to show up once because of this uniqueness property.

So, now let me go back and I'm going to add one other outcome, which is 'Browse product'. Cool. I'm going to fire that one a couple of times. Great. So, now I go and I click into this notification that I just sent, 'Browse product' and 'purchase amount'. There we go. So, purchase amount, what I'm going to do here is go and get the sum. The cool thing here is I sent two purchase amount events, one for $5 and one for $10. I can view the sum here and it will sum together those values. So, I could see the dollar value of all purchases.

Well, let me share my screen again. So, purchase amount here, I can see the sum of these multiple purchase amounts that I sent down. One was for $5 and one was for $10 and I'm able to see that sort of sum here very easily. I can also see this on the graph where I see, okay, in this one-minute time frame, there were three products that were browsed, browse count is three, and there were $15 of products that were purchased.

Now, I can also see for purchase amount the number of events of that type. So, recall there were two purchased amounts: one for $5 and one for $10. So, if I go to count, I'll actually see here that there were two purchase events and that their sum was $15. I'll also see for the invite friends that even though I fired that event, like five, ten times, it only shows up once here. You’ll see all of these are showing up in the influenced column because I didn't click the notification. Instead, ignored the notification but went into my app right after I'd gotten it. So, we call it influenced because I opened the app and I triggered these outcomes within close time proximity to when I received that.

I could also then click on the notification directly, which I'm going to do right now and let's say trigger some purchases, outcome with value. So, I'm going to trigger $20 of purchases. So, refresh this, purchase amount. You’ll see here that in purchase amount, I now have this count of two because I fired one of $10 and another one of $10. I'm able to see this breakdown of direct and influenced like we just talked about as well as a total across. We’ll see that since I only actually directly clicked on it once, I only see one click here. Cool.

[20:10]

So, this is letting you take any event that you want to instrument in your app or website and added in visually and easily be able to see it in the OneSignal dashboard. You can also export these events and it will email you that export. So, if you want to do analysis in Excel or Google sheets or something like that, really easy to get this data and be able to slice and dice it in a bunch of different ways.

So, we've sort of gone through outcomes very briefly and how they're useful. A couple of other things we'll chat through and then I'll open it up for any questions. So, there are these three outcomes that we provide without you having to write any code, you just make sure you're on the latest version of our SDK. And then, there are these custom outcomes. You’ll notice when I used purchase amount or invite friends, those were custom outcomes. To add them to your app or website, it's just a line of code, JavaScript on web and align on Android or iOS. That's included in the documentation. So, if you just go to OneSignal documentation and type outcomes, you'll see that we describe these three types of outcomes.

Regular outcomes, or you just send an event. Outcomes with a value, like purchase amount and the dollar amount of the purchase, and unique outcomes, where even if you send that outcome a lot of times, we'll automatically de-duplicate it for that user on that notification. So, that is useful when you're saying, "Hey, I want to know if someone posted content during the session rather than how many pieces of content did they post." There are a lot of cases where you might actually want both, right? So, if you're thinking about building better analytics for your app, you might want to know how many pieces of content someone posts. You may also want to know how many unique sessions contain someone posting content.

So, sendOutcome, you would use to track how many times content was posted. And sendUniqueOutcome, you would use to track how many unique people or sessions were generating content.

Any outcome that has a value can be aggregated with either a count or with a sum. So, the count of sessions is how many sessions existed. The sum of session duration will give me all the seconds spent from folks having clicked that notification if it was direct or in close time proximity to it if it was influenced.

A couple of quick notes. So, outcomes are a new feature we've shipped, which means you need to be on the latest version of the OneSignal SDK. Clicks are backwards compatible and are reported even on really old versions of the SDK because we want to make sure that you could have accurate a click-through rate reporting and statistics. But if you actually want to add these new outcomes or get things like session duration, that requires updating to the latest version of the SDK.

Looking forward a little bit. We're building iOS and web SDK support right now. We expect that to be ready in the next couple of weeks. And then being able to export outcomes via the API, not just via the dashboard, is also a thing that we're working on along with user-level outcomes, so knowing which user triggered which outcome, and finally, being able to fire outcomes from in-app messages.

So, an interesting use case here would be, if I want to get a net promoter score for my app, I might actually want to send folks an in-app message that says, "How would you rate us 1 to 10?" and I'll be able to see in outcomes how many folks voted me a 1, how many folks voted me a 2, how many folks voted me a 3, 4, 5, 6, 7, 8, 9 and so on. So, that's going to allow you to do sort of quick surveys, quick polls without having to actually write code. We'll build that in as something that you can do from the in-app messaging UI in the product.

[24:42]

So, hopefully, that gives you a brief idea of what outcomes are capable of doing, how they can help you understand, drop off at each step of your conversion flow in your app or site, and can help you make better decisions. If you know that a lot of folks are, let's say adding to cart but not purchasing, maybe you want to redo part of your UI to help folks once they've added something to cart be aware of that. Or maybe even better, you want to use OneSignal that trigger abandoned cart reminders for those folks to come back and check out and complete their purchase.