Now that product is delivered, it is time to monitor the metrics which were defined and also explore many more insights which can only be done once customer start using your product. How they use it, when they use it, what makes them hate it, what makes them love it, were they able to do what they set out to do, did they abandon the process in the middle, why? Lot of these questions and many more can be answered when you monitor and check the user journeys. All the data collected from here needs to be explored very well and in depth. This the stage where some bad assumptions can be made and based on that some bad decisions can be made. So as a product manager, you need to go beyond the broader aspect of numbers to deep dive into a specific metrics to know if all is aligned with what you expected. For instance, if you only monitor the total revenue of the mobile app purchases, you may see some very good numbers. You may feel happy that your total revenue increased 20% from last year, but deep diving a bit more, you may realize that your churn rate has increased by 30%. The latter is much more worrisome scenario as lost customers will impact you heavily in coming years. by further analysis, you may realize that you promoted heavily on homepage which led to lot of purchase, but also made some people not happy about promotion of products which they do not need and left before they could find their products.
Here is a twist, with futher exploration, you may realize that the lost customers were not part of your target segment, so you may not be in a bad situation. All these things are part of analysis which you will do to ascertain whether a change in metrics or numbers is really a good sign or there is much more than meets the eye.
The key difference between metrics setting is that, you will measure the metrics after delivery to know what is happening in reality against the goals or metrics set earlier. Earlier you just added a theory, it is now time to measure what happened once you launched. you will need tools to measure these numbers and make sense out of it.
Here are some links to help you learn which metrics to capture and how to look at product performance post delivery.
- Product monitoring metrics: Read which metrics should you track.
- How to measure product performance: Good to read about how to measure the metrics.
- What can Product usage metrics tell you about your product: one more good read on what to measure and how is it needed.
- What is funnel analysis: Understand what the funnel analysis means.
- How to analyze funnel: read this article to understand how to measure funnel analysis. Crucial for user journey montioring and where users drop off.
- Funnel Conversion trends analytics: Good watch on how exactly to do the funnel analysis to see the user journey.
- Funnel analysis in GA4: try out the GA4 funnel analysis with this video.
- Better approach to analysing activation funnel: Very good video to understand funnel analysis for activation.
- Ecommerce funnel analysis: Good example of the E-commerce funnel analysis.
- Server performance metrics: Technical measurements which would be pivotal in good user experience.
- 33 critical KPIs: inclues all metrics of technical and non technical for SaaS.
- How to Analyze heatmaps: Heatmps are crucial parts of product analysis and need consideration on how users are interacting with various parts of the product. Understand how to analyze it.
- Heatmaps expertize: if you want to become expert in heatmaps.
- How to use heatmaps for web and mobile: Check this for understanding different behavior tracking for mobile and website.
- Heatmaps, how to increase conversion with heatmaps: Watch this video to influence conversions with heatmap data.
- Conversion rate optimization for Ecommerce: watch ecommerce example for heatmap analysis and recordings.
- AB testing from Udacity: The best free course to become expert in AB testing from Udacity.
- A/B testing from Optimizely: Learn it from one of the tools for A/B testing.
- A/B testing vs Multivariate testing: Watch the difference between multivariate and A/B testing.
- Cohort Analysis: good explanation for cohort analysis.
- Beginner’s guide to Cohort analysis: A good guide from Appcues to explain Cohort analysis.
- Step by step guide for cohort analysis and churn rate: Amplitude defines it well.
- Cohort Analysis 101: detailed explanation and use case for cohort analysis.
- Cohort analysis by VCs: check out this video to know how VCs calculate cohort analysis.
- Cohort analysis in Tableau: learn how to do cohort analysis in Tableau.
- Cohort analysis in Google sheet: learn how to do in Google Sheets.
- Cohort Analysis with Google Analytics: watch a short video for understanding cohort analysis and how to do in Google analytics.