As a product manager in the software space, there’s a buzzword I hear a lot: “engagement.”
You’ve probably seen it everywhere too. From social media companies pulling the strings of your dopamine fix to notifications that pull you into an app you’ve already checked six times today, engagement is a concept that many products use as a success metric. Personally, I loathe it. Not just because of its potential for harm, but also because it’s a pretty poor metric to measure your product’s success.
How Traditional Engagement Is Tracked
Let’s dive into what product teams really mean when they say they’re tracking engagement. Usually, this means counting the number of clicks, mouse hovers or minutes spent on-screen with a new feature. In human terms, this means measuring the amount of attention a user gives to the feature.
The assumption is that, the more attention a user gives a feature, the better the feature is. The product manager can report to leadership that the feature is a success. Everyone can pat themselves on the back and go home happy.
This is where I disagree. Simply grabbing attention is cheap; anything bright, shiny and new can grab users’ attention. If engagement is truly how you are measuring success, then there are a lot of unethical ways you can boost it, like making the feature bigger, adding distracting animations, selecting a brighter color scheme and using click-baity, sensationalized messages.
Another problem with tracking engagement comes from the “newness” problem. Frequently, users will give attention to anything that is new because it’s, well, new. When they click on their favorite app, they’re not expecting a new delivery tracker on the homepage — but there it is! It’s in our human nature to click on things that are unexpected, to investigate and to learn. This “investigation” engagement will skew high when a feature has just been released, but, over time, it will diminish as the newness wears off. Of course, by then, your MVP will be deemed a success, and you’ll be on to the next big feature idea.
Track Success Instead
As product managers, we should build products that add value to our user’s lives. The easiest way to add value is to use your product to solve a user’s problem. Ask yourself this question: “This feature is a solution. What is the problem?” Once you’ve figured out the problem you’re solving, consider if what you are building is the best solution you can provide to that problem. If your new feature or enhancement is the right solution to your user’s problem, you wouldn’t want to just track engagement anyway. You would want to track success.
There are two ways to track success for a new feature: satisfaction score and completion rate. Sometimes, it makes sense to use one or the other; other times, you’ll use both. It all depends on the complexity and use cases for your new feature or enhancement.
1. Satisfaction Scores
This is as simple as asking your user “Is this feature helpful?” You can easily use a third-party software to pop a non-intrusive toast after your user tries out the feature. If you’re unable to integrate third-party software, you could send an email to your user after their first session engaging with the feature (though you may need to incentivize with a gift card raffle). Provide quick-and-easy “yes” or “no” as answer selections. You could even give the users space to optionally type in feedback; qualitative data is great too!
2. Completion Rates
This goes beyond just clicking on the feature. Did the user actually complete the journey as you expected them to? This works great when there is an obvious “end state” to your feature. What percentage of users complete the task? Where in the funnel do they fall out? Analyzing your funnel can help you identify points of friction in the customer experience.
How to Ethically Reframe Engagement
I am going to tiptoe into product marketing for a moment and discuss what I call “ethical engagement.” There is a way to promote your new feature and be competitive in your outreach without going after cheap attention.
Every time you communicate with your user, you are interrupting them and asking for their attention; in exchange, they expect you to deliver them some value. This goes especially for app notifications (which can be extremely disruptive) and promotional emails (which are used so liberally that your users don’t even read them anymore).
If you are going to send out emails or notifications promoting a new feature, consider closely the value, timing and audience of the message. Rather than over-enthusiastically notifying your entire user base about a new delivery tracker — “Hey Andy! We have a new delivery tracker! Check it out!” — instead consider messaging the user immediately after they’ve placed their order, when they’re most likely to be searching for that information: “Your order is on the way! To keep an eye on its arrival time, check out our delivery tracker in the app.”
The Takeaway
I spend a lot of time defining what success means for our users. As someone who can shape my product’s offerings, I want to make sure I’m respecting my users by building features that genuinely add value to their lives.
It would be a shame to measure success by simply using engagement, because that doesn’t accurately capture whether we’ve built something of value. When you’re building something new, you should choose your success metrics based on the best way to capture the value you’re adding to your user’s day.
