For some product managers, mastering user retention means their users are on their app longer. For Product Manager Cissy Chen, it means she’s saving lives.
At Citizen, an app that notifies users when they are in proximity of 911 alerts, Chen’s goal is for users to read push notifications. To do so, she tracks click-through rates to evaluate what notifications are of the most value to their users. By analyzing the data, Chen can curtail what notifications are relevant and balance non-emergency notifications, like software updates, without losing users.
According to an article on ProductCraft, it is 5-25 times more expensive to acquire new users than to retain them. By tracking user interaction with a product, whether through technical tools or customer feedback, product and growth managers can gauge how their product is being utilized and make adjustments to steer customers toward its intended use.
We spoke to seven industry experts, including Chen, about the tools and methods they use to keep users engaged.
Tips For Boosting User Retention
- Use in-app event tracking to hone in on user journeys to optimize their flow
- Use post-purchase surveys, user interviews and behavioral quizzes
- Use data science tools to measure how consumer behavior changes due to different events (time of year, weather, etc.)
- Make sure to constantly experiment to optimize all processes
Citizen
Cissy Chen
PRODUCT MANAGER
Product Manager Cissy Chen works to improve the user experience — and thus, user retention — by making sure the Citizen app sends relevant notifications that keep users safe. Mobile events allow Chen’s team to track what features users value. As the company expanded out of New York City, machine learning and user data helped build new notification models.
What tools or technologies are you using to capture and analyze user behavior data?
We fire mobile events to note when a user views or taps something in-app and to understand how they use Citizen and which features they find the most value in. Because one of our core focuses is protecting our users’ data, we decided to build our own mobile analytics pipeline instead of using a third-party service. Identifiable user data is deleted from the pipeline after 30 days.
What specific behaviors are you keeping an eye out for or do you consider to be most important?
As a mission-driven company, our goal is to help keep people safe, not endlessly scrolling through the app. For that reason, we’re constantly striving to make our notifications more relevant to our users. One way we do that is by analyzing how the click-through rate varies across different incident types in order to make sure we notify users when incidents might affect their safety and avoid bothering them with information that isn’t useful.
We’re constantly striving to make our notifications more relevant to our users.”
Beyond user behavior analysis, what's the most effective method you've used for improving user retention?
Citizen started in New York City but quickly expanded to other cities around the U.S. With this growth, we realized the notification model we’d built for NYC – which defaulted to alerting users within a quarter-mile radius of an incident – didn’t work everywhere. We used machine learning to figure out how far to distribute information based on users’ revealed interest in it over time. For example, in midtown Manhattan, a report of a fire might only notify people within .1 miles, whereas in San Mateo, California, it might notify people two miles away. This shift led to the app delivering more relevant information to users, which led to huge gains in user retention.
Hydrant
Margaret Fortner
HEAD OF GROWTH
According to Head of Growth Margaret Fortner, customer retention is gathered through both feedback and technical data at Hydrant. The customer experience team utilizes post-purchase surveys and behavioral quizzes while Google Analytics and Hotjar help track data across all points of the customer’s journey.
What tools or technologies are you using to capture and analyze user behavior data?
On one side, we use a mix of manual, user-driven tactics such as post-purchase surveys, user interviews and broader behavioral quizzes. We have a very passionate and opinionated customer base. Our customer experience team does a fantastic job engaging them and collecting their feedback to help drive the brand’s future and our product development roadmap. These efforts are more high touch and time consuming than your standard NPS survey, but the qualitative and quantitative insights we can gather make it worth it in the long run.
On the more technical side, we’ve set up event tagging through Google Analytics for as many behaviors and touchpoints as possible to measure key points of friction and funnel drop-off. Maintaining this tracking across areas where customers in all points of the journey will be — from paid social-oriented landing pages to the subscriber portal — allows us to aggregate and segment data appropriately for optimal insights. We’re also able to leverage tools such as Hotjar to closely monitor behavior, particularly around new features or tool releases onsite, so we can iterate and update quickly with our development team as needed.
We have a very passionate and opinionated customer base.”
What specific behaviors are you keeping an eye out for or do you consider to be most important?
For users on their first or second visit to the site, we’re monitoring bounce rate, page scroll depth, what products they’re looking at, ease of engagement with the checkout flow and any cross-sell and upsell modules.
Post-purchase, we’re monitoring how often they’re returning to the site to check order progress, engagement with our subscriber portal, engagement with new blog content and testing of new features. More broadly, we also monitor the users returning to site without being prompted by any marketing or outward brand touchpoints.
Beyond user behavior analysis, what's the most effective method you've used for improving user retention?
Investing time and effort into developing our customer subscriber portal has had some of the most positive results on user retention. When we initially embarked on the project, the out-of-the-box subscriber portal we were using was less than ideal. Our e-commerce and dev teams were able to create a more visually engaging, user-friendly experience that was not only easier to use but more feature-packed.
This portal is an ongoing test and learning process. We’re probably on our third or fourth large update since launch, thanks in large part to the analytical flywheel of our customer monitoring software and the direct customer feedback and recommendations captured by our CX team.
MobilityWare
Eden Simpson
PRODUCT OWNER
MobilityWare Product Owner Eden Simpson said her team tests feature usage in smaller games, and if user data deems the features successful, they port the new features to games with bigger audiences.
What tools or technologies are you using to capture and analyze user behavior data?
In mobile gaming analytics, we are always balancing the trade-offs between the speed at which we get data, and how complete it is when we act on it. So we use tools that excel on each side of this spectrum. To interact with our data quickly — during a live ops campaign, for example — we use data management platforms like Leanplum or Treasure Data. For strategic analysis that requires highly accurate data, we unify data from all our sources into Snowflake, our preferred cloud-based data warehouse. From there, our analysts can write SQL-like queries for analysis and build dashboards in Tableau.
The behavior we watch most closely is how many games people are playing.”
Why did you decide to use these tools over the other options on the market?
We choose data solutions based on a couple of areas of focus, and the first is feature set. The more functions a specific tool does well, the more likely we are to use them. For example, does a data management platform only track in-app message campaigns, or can it execute A/B tests and visualize KPIs as well?
The second focus is ease of use. Does leveraging features require a degree in computer science, or are they intuitive enough for product owners to use them?
The last focus is on implementation and documentation. Vendors will try to wow users with how effective their tool is, but how easy is it to integrate? Engineering resources are precious so we factor in the cost of incorporating and maintaining a given solution.
What specific behaviors are you keeping an eye out for?
We earn most of our revenue from advertising, so we don’t have to search for ways to encourage players to spend money on in-app purchases. When we look at data and adding new features, we are always focused on how to make our games more enjoyable. Due to this methodology, the behavior we watch most closely is how many games people are playing, which are in the billions each day across our titles.
Beyond user behavior analysis, what’s the most effective method you’ve used for improving user retention?
To improve retention, we focus on new game features that give players a reason to come back. Luckily, we have games of varying sizes. This diversity of size allows us to test new features in smaller games and move them to larger ones if the data shows players enjoy them. Adding features this way allows us to try new things and innovate without upsetting our most loyal players.
Our most successful feature is a daily challenge feature, which we were the first to release in card games. It increases the number of days the players interacting with the feature logs in. It was picked up by many of our competitors and is now seen as an expected feature in card games.
Tinder
Jona Cho
PRODUCT MANAGER
Product Manager Jona Cho said Tinder’s user metrics revolve around helping people make real-world connections. Cho said her team uses machine learning and Tinder’s custom-built data platform to act on metrics leading to that end.
What tools or technologies are you using to capture and analyze user behavior data?
We use an experimentation platform we built in-house called Phoenix to both run our experiments and monitor results. For more in-depth analyses, we use Sisense and Mode. Sisense is great for time-series analysis while Mode, with it being a Jupyter Notebook-type tool, provides a lot of flexibility around collaboration and sharing across the organization.
We look for the events that proxy long-term value from the ecosystem of our product.”
What specific behaviors are you keeping an eye out for?
We look for the events that proxy long-term value from the ecosystem of our product — connecting in real life with another person — and keep an eye on leading indicators of that behavior. We iterate on the product to increase the value we deliver to our members to improve long-term retention.
Beyond user behavior analysis, what’s the most effective method you’ve used for improving user retention?
Tinder members can have up to nine photos in their profiles. However, the photos might not be in the best order to increase their chances of receiving likes. By using machine learning, we reorder photos within profiles to optimize the ranking and make sure that the best photo is featured first. This experiment not only helped members receive more likes but also improved retention.
Postmates
Enu Herzberg
VP OF DATA & ANALYTICS
Postmates makes real-time adjustments in response to their users’ behaviors, and those decisions have a wide-reaching impact. “All at once, we’re delivering incremental revenue to our partners, earning opportunities for our fleet, and moving goods to our customers — which means we must build a three-sided ecosystem in harmony.”
What tools or technologies are you using to capture and analyze user behavior data?
We are one of the largest marketplaces connecting consumers, businesses and couriers in real time. So we have to pay close attention to how behaviors, appetites and thirst change on a dime — all while real-world events such as weather changes or national holidays affect utilization of our app. Put another way, we may be a digital app, but we operate in the very dynamic and evolving analog world. Elastic customer demand, an elastic labor supply to fulfill that demand and on-demand cash transfers means that we have to partner with vendors and third parties that can keep pace with our ever-evolving marketplace.
That’s why we are proud partners of mParticle, Amplitude, Braze and CartoDB, amongst others who recognize that latency, scalability and flexibility matters. The tools we deploy are more than just anonymous vendors or third-party applications — they reflect extraordinary partnerships that are able to address and reconcile multiple permutations of use cases that communicate across systems and users to ensure our services map to the needs of a country.
What specific behaviors are you keeping an eye out for or do you consider to be most important?
Postmates is in the business of adapting our delivery systems to the real-time adjustments of our users. If merchants are keen on selling more on a holiday weekend, we must scale workers to fulfill that demand. If customers are keen on ordering more items ahead of a hurricane weekend, we must ensure we have a sufficient number of sellers online. And if we are to encourage more workers to hop on our platforms during a playoff game day rush, we have to detail appropriate incentives to encourage that work because on-demand workers don’t sign up in shifts. We must always account for how real-world events can throw that balance off.
We have proprietary methods to measure and score our ability to surface what customers want via various conversion stats across platforms and channels. And that’s why we must keep a close eye on our market health via custom supply and demand utilization stats that can scale down to the narrowest geographic levels.
Beyond user behavior analysis, what's the most effective method you've used for improving user retention?
Build it into the product. Today’s consumers prefer the advantages of access over the hassles of ownership or even excessively repeated fees. When we launched Postmates Unlimited in 2016, the delivery industry’s first subscription program, we made a bet that delivery should be part of everyday life for all consumers. We bet that families who relied on Friday night meals may want to do so regularly. We bet that customers who were inclined to order with a consistent clip would opt into our service with a bit more loyalty and regularity. And we made a bet that the revenue netted from sales for Unlimited holders would outpace non-subscription holders.
ProfitWell
Neel Desai
PRODUCT LEAD
A bit of background: ProfitWell builds revenue operations products for subscription-based companies. Since 2012, the Boston-based startup’s software has helped customers with reporting, analytics, optimization and churn reduction. While ProfitWell may be small in size — there are about 50 employees at the company — don’t let that fool you. According to the company, it holds the largest amount of subscription data you’ll find.
What tools or technologies are you using to capture and analyze user behavior data?
At ProfitWell, we use a variety of tools to capture and analyze our user behavior data. Some of our favorites include Mixpanel and LogRocket since they offer a lot of functionality and are easy to use. LogRocket is really good for replaying individual user sessions inside of our web app to see how users interact with various elements and how we might be able to improve our UX.
In particular, these tools surface up interactions like “rage clicks” or “dead clicks” that users experience that might lead to frustration. It also ties nicely into our error logging platform that houses bug reports and our customer support chat tool.
MixPanel is great for aggregate-level data analysis. It helps us understand overall product usage, which features are being used more than others, and segment that information into cohorts, user types and more.
What specific behavior are you looking out for?
We track a lot of behaviors throughout the entire customer lifecycle, from the moment the customer is acquired to if and when they churn.
We look for certain triggers early in the customer lifestyle, specifically during onboarding, that we know will lead to higher retention. For example, setting up ProfitWell’s Slack integration, downloading our mobile app or installing our optional JavaScript that unlocks additional features. We know these behaviors typically lead to power users who develop a higher overall affinity for our product.
We track these key metrics in a central dashboard and then assign product managers various metrics to optimize over the course of the quarter alongside their OKRs. Over time, we’ve developed various nudges and call to actions (CTAs) throughout the product experience to get more users to take advantage of these features.
The single most impactful initiative to drive overall user retention was to overhaul our onboarding process.”
Beyond user behavior analysis, what’s the most effective method you’ve used for improving user retention?
The single most impactful initiative to drive overall user retention was to overhaul our onboarding process.
We experimented with multiple email drips, in-app tutorials and human-assisted touches for high profile users. We’ve found that putting a specific emphasis on educating the user on how to use the product and how we might be able to help them made a meaningful difference in helping them find value in the app.
To measure our results, we do a lot of cohort analysis to look at how retention is getting better or worse over time, and develop interventions at specific moments in the customer journey where they become high risk.
Federal Reserve Bank of Boston
Dan Baum
HEAD OF FEDNOW PRODUCT DEVELOPMENT
A bit of background: The Federal Reserve Bank of Boston is a part of the nation’s central bank. As such, it has a number of priorities, like putting policies into place, overseeing other banks and promoting financial success in local communities. Engineers should take note of the work the Federal Reserve Bank of Boston is doing to zap cybersecurity risks and keep payment systems on the cutting edge both regionally and nationally.
What tools or technologies are you using to capture and analyze user behavior data? Why did you decide to use these tools over the other options on the market?
The FedNow Service — whose target release date remains 2023 or 2024 — will offer instant payment services to financial institutions of all sizes and in every community across the United States. Potential FedNow participants not only vary in size but in their existing product offerings, their business needs and where they are in their instant payments journey.
To better support the needs of financial institutions as they consider adding the FedNow Service to their payments roadmap, the Federal Reserve conducted an industry listening tour in the fall of 2019 to interview banks, credit unions, service providers and other industry providers to understand the unique needs of different industry participants and to obtain feedback on the FedNow features and functionality to inform the FedNow product roadmap. The team leveraged traditional customer relationship management (CRM) tracking tools to document insights, which were then leveraged to define the different user personas and inform key implementation considerations for the FedNow Service.
In addition to the industry listening tour, the Federal Reserve continues to conduct market research through customer surveys, focus groups and industry webinars, and roundtables to ensure that its understanding of industry needs remains current and that the industry can continue to provide feedback on features and functionality as we get closer to the launch of FedNow Service.
What specific behavior are you looking out for?
Nationwide reach is the ultimate goal for instant payments. To ensure the service is broadly accessible, the Federal Reserve is working to ensure that the FedNow Service takes into account the unique needs of different market segments and is compatible with other instant payment solutions to minimize the potential burden for financial institutions that may participate in more than one instant payments network.
For example, the Federal Reserve is gaining feedback on product features and functionality, and is seeking to minimize any unnecessary differences to ensure messages contain the information necessary to support financial institution processes and that end-to-end processing is seamless. The Federal Reserve will continue to work with the industry to understand the industry’s evolving needs and new trends in payments.
The Federal Reserve is working to ensure that the FedNow Service takes into account the unique needs of different market segments.”
Beyond user behavior analysis, what’s the most effective method you've used for improving user retention?
To encourage adoption, the Federal Reserve is committed to engaging a robust ecosystem of industry stakeholders that will work alongside the Federal Reserve throughout the development of the service to ensure the instant payments ecosystem can support the FedNow Service and provide an enhanced customer experience for financial institutions and end users.
The Federal Reserve launched the FedNow Community to provide transparency into service development to ensure that interested participants can help to shape the development and implementation of the FedNow Service.
Similarly, the FedNow pilot program will provide opportunities for the industry to engage in advisory, testing and operational activities to support the service’s development, testing and adoption objectives. While the FedNow Service is use case agnostic by design, the FedNow team is collaborating with a broad set of industry participants to ensure the ecosystem can support the use cases that are anticipated to have the greatest market demand at initial launch and that the Service can be a springboard for payments innovation by enabling new ideas and services.
