Product Management

CIRCLES Method
(how to approach design problem)
CIRCLES Method
(how to approach design problem)
1) Comprehend Situation
2) Identify Customer
3) Report Customer Needs
4) Cut through prioritization
5) List solutions
6) Evaluate trade-offs
7) Summarize Recommendations
CTR
Click Thru Rate
-1.3 to 3% is target depending on industry
-ratio of number of clicks to impressions

(Total Clicks on Ad) / (Total Impressions)

4 Components to Good Product Strategy
1) Real Pain Points means “Who is it for?” and “What is their pain point?”

2) Design refers to “What key features would you design into the product?” and “How would you describe your brand and voice?”

3) Capabilities tackles the “Why should we be the ones doing it?”and ” What is our unique capability?”

4) Logistics is the economics and channels, like “What’s our pricing strategy?” and “What’s the medium through which we deliver this?”

MVP Alternatives
-SLC
-RAT
SLC – Simple, Lovable, Complete
-adds value the way it is, doesn’t need a v2
-Snapchat started w/simple photo
-simple is good, incomplete is not

RAT – Riskiest Assumption Tests
-only build enough to test biggest unknown
-“Is this the smallest thing we can do test our riskiest assumption?”

CSAT
Customer Satisfaction
-measures short term happiness (transactional)
-does not address how user feels about company as a whole
-typically use 1 to 5 scale
-# of satisfied customers (4/5) / # of responses

“How would you rate your overall satisfaction with the service [or product] you received?”

Diff w/NPS
-NPS is about long term happiness
-NPS measures loyalty

Qualities of Great Product Manager
1. Strategic Thinker — Understanding the current needs of the product and how it fits with the company’s overall vision.
2. Passion for Products — Love what you make, love what you do.
3. Empathizes with the Customer — and understand the customer’s wants and needs.
4. Interviews Customers — Gather insights from your customers, learn their pain points and discover opportunities.
5. Aspires to Build Great Experiences — Understands that user experience is paramount for product success.
6. Keeps Score — Identify the key metrics and track performance, establish baselines to measure success.
7. Ability to Prioritize — Keep things in order, understand what needs to be completed first and what needs to wait. Use customer feedback for prioritization.
8. Collaborative Leader — Management is a collaborative process, you cannot be dictatorial and expect to inspire your team.
9. Execution — Action-oriented, you get things done.
Survey Checklist (best practices)
-My survey has a clear objective
-My survey has a defined target audience
-I know what I want to achieve with my survey
-My survey questions are not ambiguous
-I know where people will respond to my survey (at home, at work, on the bus…)
-I know what I will do with the responses and how to action them
-I know how I will distribute my survey (via email, social media…)
Survey Best Practices
1) Keep the survey short
Max 5 minutes to fill out
Include progress bar
2) One question at a time
3) Explain the survey to users in beginning and thank them at the endd
4) Order the questions properly
Add sensitive questions at the end
Causes for a Bad Product
1) Tunnel Vision
2) Unclear Goals
3) Product Neglect
4) Customer Avoidance – spend at least 3hrs per week
5) Inability to Say “No”
Make an App “Sticky” From the Start
1) Let the user try it with no strings attached
2) Keep away from indirect competitors
3) Advertise the possibilities and benefits first
4) Less is more
5) Bridge the gap between the physical and digital worlds
Steps for Improving Product Question and Favorite Product
1) Goals
2) Problems
3) Solve Problems
4) Validate Success

1) What problems does it solve?
2) How does it accomplish goals?
3) How does it compare to alternatives?
4) How would you improve it?

Empathy Map
Empathy Map
-A way to understand the user’s needs / pains…etc
Steps for Designing Product Question
1) Ask questions about problem (clarify)
2) Provide structure
3) Identify users / customers
4) Use cases? Why use product?
5) How well is product w/use cases?
6) What features / changes to improve?
7) Wrap up

*Remember to define MVP. Then other features are improvements do ROI

Rule of 72
How long until something doubles

-increase x% /yr
-double after 72 / x yrs

e.g. 72 / 8 (compound int rate) = 9 yrs to double

Orders of Magnitude
A*B = n
digits (A) + digits (B) = digits (A*B)

823 * 1032 has 6 or 7 digits

Key Product Metrics
(users, conversion…etc)
1) Users / Traffic
2) Conversion
3) Referral Rates / Types
4) Engagement
5) Retention
6) Revenue
7) Costs
Key Product Factors
(e.g. users / goals…etc)
1) Users and Goals
2) Strengths
3) Challenges
4) Why Why Why
5) Priorities / Values
6) Competitors
7) Trade offs
4 P’s Framework
(marketing plan)
1) Product (establish uniqueness / value)
2) Price
3) Promotion (where to advertise)
4) Place (where to sell)
SWOT Analysis
(strength / weaknesses and threats)
SWOT Analysis
(strength / weaknesses and threats)
1) Strengths
2) Weaknesses
3) Opportunities
4) Threats

Strengths: characteristics of the business or project that give it an advantage over others

Weaknesses: characteristics of the business that place the business or project at a disadvantage relative to others

Opportunities: elements in the environment that the business or project could exploit to its advantage

Threats: elements in the environment that could cause trouble for the business or project

Y-Axis: Internal / External
X-Axis: Helpful / Harmful

*Strengths / Weaknesses can be about product offering not just organization

The 5 C’s
(product pricing)
1) Company
2) Competitors
3) Customers
4) Collaborators
5) Climate
BCG Matrix
BCG Matrix
Cash Cow
-generates more cash than is needed to maintain market share so excess should be reinvested

Dogs
-are worthless, low market share and low growth – only value in liquidation

Question Marks
-are high growth, low market share -require far more cash than they can generate
-become Dogs when the growth stops
-liability

Stars
-are high market share, high growth -always generates profits
-becomes Cash Cow
all products become Cows or Dogs -all dependant on obtaining a leader share of market before growth slows

Balanced Portfolio has:
Stars – high share and high growth assure future
Cash Cows – supply funds for growth
Question Marks – can be converted to stars w/added funds

The 3 C’s Marketing
Customer
-unmet need?, segment?, price sensitive?

Competition
-strengths / weaknesses, concentration, potential substitutes

Company
-strengths / weaknesses, value chain, value add?

Porter's 5 Forces
Porter’s 5 Forces
1) Rivalry among competitors
2) Buyer Power
3) Supplier Power
4) Threat of Substitutes
5) Threat of New Entrants
How to Gather Metrics
(specific ways)
1) Usability Testing
2) Customer Feedback
3) Traffic Analysis
4) Logs
5) A/B Testing
Root Cause Analysis (5 Whys)
Root Cause Analysis (5 Whys)
1) Ask why problem happens and write down answer below problem
2) Ask why again if it doesn’t identify root cause
3) … repeat

E.g.
Q: Why did your car stop?
A: Because it ran out of gas
Q: Why did it run out of gas?
A: Because I was lazy

What does being a PM mean to you?
Leader
Define Goal
Explain Vision

-responsible for guiding the success of a product and leading the team that is responsible for improving it
-liaison between stakeholders and implementation teams
-PMs are extremely valuable to build the bridge between the outside (market, customers, external stakeholders, partners) and the inside (all the sources around the product)

Why Metrics?
1) Maintain focus
2) Set common goal for team
3) Hold everyone accountable
4) Object decision making
GAME Framework
-How to establish metrics for a project

G -> Goals (user and business)
A -> Actions
M -> Metrics (use ARM)
E -> Evaluations

MOSCOW
Must Have
Should Have
Could Have
Would Have
Story Points
-Why?
-Key points
Why?
-Avoid scope creep
-Establish team velocity
-Quickly establish estimates
-Eliminate subject hour estimation based on developer skill

-Complexity, Effort, Doubt
-Not based on hours

Base 2 sequence:
0, 1, 2, 4, 8, 16

ROI
ROI
Return on Investment

-Ratio btw business value and complexity

Bottom Up vs Top Down
Bottom Up: ideas come from devs / PMs

Top Down: ideas come from execs that supply long term vision

ARM Framework
-Questions to help establish metrics

A -> Acquisition and Activation
-what actions do users take to get value and come a customer?

R -> Retention and Engagement
-what gets my users back to product?

M -> Monetization and Revenue
-what format of payment does product take?

TAM
Total Addressable Market

-Quantify market size / opportunity

-Bottom Up: figure out how many toothbrushes sold in stores and for what cost

-Top Down: market share and total market size

ARPU
Average Revenue Per User

-figures out value users place on your product

-Total Revenue / # Users in Period

Cohort Analysis
-how users engage with product over time
-shows how much users like your product
-“cohort” is a group of users
-look at retention of cohorts after 6-12 months
Key Usage Metrics
(actual metrics to look for)
1) Time on Site
2) Repeat Visits
3) Unique Visitors
4) Active Users
5) Feature Usage
Ways to Change Competitive Strategy
1) Diversifying revenue sources
2) Building barriers to entry
3) Being the “One-Stop Shop for ___”
4) Being the Low-Cost Leader
5) Reducing Reliance on a Key Buyer or Supplier
6) Testing a New Market
Topics to Discuss for Marketing Strategy
(understand A, B, C…)
1) Understand the Company
2) Understand the Competition
3) Understand the Customers
4) Understand the Landscape
5) Market Your Product
Topic to Covers for Launch Strategy
1) Target Market
2) User Types
3) MVP or Full Product
4) Distribution
5) Rollout
6) Buzz
7) Partnerships
8) Risks
Pricing Strategies
1) Cost Plus Pricing
2) Value Pricing
3) Competitive Pricing
4) Experimental Pricing
Pricing Models
1) Free, Ad-Supported
2) Freemium: basic is free but premium costs $$
3) Tiered
4) A La Carte
5) Subscriptions
6) Free Trial
7) Razor Blade Model
Ad Pricing
1) Pay-Per-Click
2) Pay-Per-Impression: sees ad but doesn’t have to click
3) Pay-Per-Action: clicks on ad and pays for product
Most an Advertiser Would Pay for Ad
-expected profit from click
-conversion rate typically 2-5%

max for ad
= expected profit per click
= conversion rate * profit per conversion

Common Problems with Products
1) Falling profit
2) Falling revenue
3) Falling sales volume
4) Declining new customers
5) Increase in costs
6) Decline in traffic
7) Decline in new visitors
10 Principles of Good Design (and best ones)
**Innovative
**Makes a product useful (utility)
Aesthetic
**Understandable (ease of use)
Unobtrusive
**Honest
Long lasting
Thorough / detailed
Environmentally friendly
As little design as possible
1) US population
2) Ave people per household
3) # households in US
4) Life expectancy US
5) Life expectancy World
6) Storage cost per GB
1) 300mm
2) 2.5
3) 100mm
4) 80yrs
5) 65-70yrs
6) AWS is 0.023/gb
1) World population
2) European population
3) Asia population
4) Hours in a year
5) Minutes in a year
6) San Fran / NYC population
7) # airports in US
8) # cities in a state
1) 7.2 billion
2) 700mm
3) 4 billion
4) 9000
5) 500,000
6) SF = 850k NYC = 8.3MM
7) 5000
8) 100
Market Entry Checklist
-Characteristics
-Market Size
-Market Growth
-Profit Margins
-Market Trends
Market Entry Checklist
-Competitive Environment
-Number of competitors
-Competitors resources (human capital, $$)
-Competitors unique competences (distribution channels)
Market Entry Checklist
-Company Fit
-Expertise
-Economies of scale
-Access to distribution channels
-Good relationships with suppliers
-Related to existing brand promise
1) US GDP
2) US GDP Growth Rate
3) US Corporate Tax Rate
4) US Smartphone Penetration
5) US % Under 18
6) US % Over 65
1) 17 trillion
2) 2%
3) 35%
4) 70%
5) 23%
6) 13%
Market Size Question Solving Steps
# of people in population
* % target customers
* purchase frequency
* quantity purchased
* price per unit
Revenue Estimation Question Solving Steps
1) Estimate revenue for a single store / product
2) Estimate total # of store (total units)
3) Multiple 1 and 2 for revenue
Lifetime Value Question Solving Steps
= Ave contribution per sale
* Sales per year
* # of years per customer
TCP/IP Data Flow
Computer
-> Application Layer (HTTP)
-> Transport Layer (TCP)
-> Network Layer (IP)
-> Data Link Layer
Ethernet
Questions to Ask a Company
-How do all of the products fit together?
-Who are the competitors? How does the company differentiate?
-What is the revenue model?
-What do customers love / hate about product?
-What are the key metrics? How many users… conversion rate… growth rate?
-What are your product weaknesses?
-What are the product’s competitive advantages?
-What threats are there impending?
Questions to Ask About Role
**What are top 3 qualities looking for in a candidate
-How technical is the role?
-How much time on writing requirements… managing timelines… building roadmap… etc
-Where do ideas come from? Bottom-up or Top-down?
-Manage innovation? Is product relatively change averse or willing to take informed risks?
-How do you make design decisions? Surveys… studies… A/B testing?
-What is most challenging about being a PM here?
Agile Key Tenants
-Advantages / Disadvantages
-Respond to change quickly
-Iterative and Incremental
-Communication, transparency, constant feedback
Advantage: respond and deliver quickly
Disadvantages: hard to scale, difficult to create roadmap
INVEST
-story best practices
Independent
Negotiable
Valuable
Estimable
Small
Testable
SCRUM
-Scrum Master duties
-Product Owner duties
Scrum Master
-establishes scrum practices / rules
-ensures team is fully functional
-sets up meetings
-help with the backlog

Product Owner
-defines features of product; focus on ROI
-track value and return
-scope management / prioritize backlog
-accept or reject work

Key SCRUM Meetings
Key SCRUM Meetings
-Daily Scrum
-what’d you do yesterday?
-what will you do today?
-any blockers?
-Sprint Review
-1/2 day; demo work to stakeholders
-acknowledge achievement
-Sprint Retrospective
-Sprint Planning
-1/2 day; 60% must haves
-Backlog Refinement
Agile, Lose Big Picture Focus
-Why?
-How to fix
Agile, Lose Big Picture Focus
-Why?
-How to fix
Why
-lack of clear roadmap from PM
-myopic view of team (only focused on one section of product)

Fix
-Story mapping

Agile Issues
Agile Issues
-Not enough design time
-Sprints are too short
-Not enough user feedback
-No sprint planning
-Lacking big picture
-Poor communication
-Dependency issues
What Makes a Good PM?
What Makes a Good PM?
-Ownership of product
-Convincing and persuasive (lead / influence)
-Be optimistic and positive
-Get technical
-Celebrate others
-Be fearless when discussing ideas
Common PM Duties
-Analyze log files for data trends
-Prioritization of backlog
-Launch preparation
-Post launch analysis
-Create mock-ups
Prioritization Matrix
Prioritization Matrix
Types of Metrics
Qualitative: usability testing
Quantitative: traffic analysis
Comparative: A/B testing
Competitive: monitor competitors
Decision Matrix
Decision Matrix
-Benefits / Cost / Mitigations
-Option A / B
Heart Framework
-and Goals Signals Metrics
Heart Framework
-and Goals Signals Metrics
-Categories used to help define metrics
-Try to have a few metrics in each category

Happiness: measure of user attitude (via survey)… net promoter score, satisfaction

Engagement: level of user involvement (# of visits per week)
Adoption: new users of product / feature

Retention: rates at which users are returning (churn)

Task Success: behavioral metrics such as efficiency, effectiveness and error rate

Customer Oriented Metrics
-Product usage or adoption
-Percentage of prospects who take an action in response to campaign
-Percentage of users who take action in product
-Customer retention or churn rate
-Quality (# of bugs)
Business Oriented Metrics
-Cost to acquire a new customer (CAC)
-Customer lifetime value (LTV)
-Monthly recurring revenue generated (MRR)
-Average revenue per user
Conversion
Cohort Metrics
-Number of sessions per user (cohort of users), over time
-Session duration for a cohort, over time.
-Number of user actions per session, for cohort, over time.
-Action completion rate & differences between different cohorts
KPIs
-Key Performance Indicator
-Do not answer why
-Measure performance, monitor progress, data for decision making
-KPIs are derived from the product vision and tell you how well your product is meeting the vision

Examples
-Deviation of set hours of work
-Planned budget deviation
-Missed milestones
-Cost variance metrics

SCAMPER
(brainstorming)
Substitute
-What part can be substituted without affecting whole?

Combine
-How to merge two idea in innovative way (e.g. phone tech and camera)

Adapt
-How to tweak and adjust for better output

Modify
-Change to release more innovative capabilities

Eliminate
-Remove a part of process / product to improve it

Reverse

P-Value
P-Value
-used to determine statistical significance of scientific test
-A/B testing validation
-reflects risk tolerance and confidence in test
-X < 0.05 -larger sample size the more confident in result -does not always mean its an accurate assumption
Persona
-what to include
-what makes a good one?
Persona
-what to include
-what makes a good one?
Behaviors
-degree of loyalty / usage

Demographics
-age, gender, income, education
-country, climate, population

Needs & Goals

Include on Card…
-Name / Title
-Goals
-Attributes (demographics)
-Needs / Challenges
-Skills / Behaviors

Good Personas are…
-reflect patterns from Research
-focus on current state
-are realistic
-help understand users

Personas
-Why?
-Develop a deeper understanding of customer needs and how to solve for them
-Guide product development by creating features that help them achieve their desired outcomes
-Prioritize which projects, campaigns, and initiatives to invest time and resources on
-Create alignment across the organization and rally other teams around a customer-centric vision
How to Build Trust w/Stakeholders?
1) Be willing to learn and let others know
2) Understand the industry and tech
3) Use metrics / stats to reduce subjectivity
4) Communicate and be transparent…keep stakeholders up-to-date
Estimate Question Math
-Volume of Sphere
Volume of Sphere
-4/3(pie)(r^3)
Financial Terms
-Fixed Cost (examples)
-Variable Cost (examples)
-Gross Profit
-Market Penetration
-Operating Margin
Fixed Cost
-Insurance
-Rent
-Utilities
-Advertising

Variable Cost
-Materials / Labor
-Shipping

Gross Profit = Revenue – Cost of Goods Sold
-In accounting, gross profit, gross margin, sales profit, or credit sales is the difference between revenue and the cost of making a product or providing a service, before deducting overheads, payroll, taxation, and interest payments

Market Penetration = Customer / Potential Customers

Operating Margin = Operating Income / Total Revenue

Funnel
-Uber example
Funnel
-Uber example
-start broad then narrrow
-broad inputs that narrow down into final product or output
-develop, measure and optimize
-a funnel is made up of the measurement of the key event at each step of the flow or user journey
-By putting these measurements together you can see where the leakage is in the funnel. The leakage is where people stop completing the task they set out to do

1. Discover the pain points
2. Find the reason for the problem
3. What is the impact of the problem

Uber’s 2 Funnels
1) on boarding
2) booking transportation

Steps to Plan Analytics Implementation
1) Define the product vision
2) Define the KPIs that meet the product vision
3) Define the metrics that allow you to hit your KPIs
4) Define the funnels (via user journeys) that affect your metrics
Segmentation
Segmentation
1) Geographic
2) Demographic
3) Psychographic
4) Behavioral
Managing Change
-Kotter’s 8 Steps
1) Establish sense of urgency
2) Create guiding coalition
3) Develop a vision and strategy
4) Communication change vision
5) Empower broad-based action
6) Generate short-term wins
7) Consolidate gains and produce more change
8) Anchor new approaches in culture
Change
-conflict
-time
-communication
-Conflict is inevitable with change. How you respond determines your reactions
-Success with change takes time to obtain, it is not a straight line path.
-Communication should be a priority for every manager at every level of the firm
-When you are so sick of talking about something that you can hardly stand it, your message is finally starting to get through
Core User Properties
Top of Mind: come directly, not because of reminder or google search…etc

Recurring: do they come back again and again?

Referring: do they share with others?

Issue Trees
-e.g. How can we increase our profitability?
Issue Trees
-e.g. How can we increase our profitability?
-Use in Why Why Why
-Start with high level question and break down via a tree
User Journeys
User Journeys
-Break down into stages of interacting with the product
-For each stage, write down the key actions

Good User Journeys help to…
1) Empathize with the customer
2) Discover product improvement opportunities

5E’s Framework
-Entice. What event triggers a user to enter into the UX funnel?

-Enter. What are the first few steps in the UX funnel?

-Engage. What task(s) is the user trying to accomplish?

-Exit. How does the user complete the task?

-Extend. What follow-up actions occur after the user completes the task?

A/B Testing
-names
-best practices
-A.k.a split testing or bucket testing
-Version A (control) vs Version B (challenger) to see which has higher conversion rate
-“cooked” term for when test is complete
-Lift = increase

Best Practices
-Test should run until at least 95% chance to beat control
-Test for a minimum of 7 days (takes into account weekdays and weekends
-Do not focus on just one metric. Make inventory of important metrics before and after every test you run
-Create segments and run A/B tests on them separately to gauge big picture

Multi-Variate Testing
-Multivariate involves testing multiple differences. Instead of 2 versions of a page, create N variations with different version of elements on each page
-Can show that a footer has no effect on conversion
-Really good for redesign efforts
-Requires a lot of traffic / data to run variations
Net Promoter Score
-How likely are you, on a scale of 0 (not at all likely) to 10 (extremely likely), to recommend this product or service to a colleague or friend?
-A company’s Net Promoter Score is the percentage of promoters less the percentage of detractors

-“Detractors” answer from 0 to 6
-“passives” answer 7 or 8
-“promoters” answer 9 or 10

Common Issues Facing PMs
-No direct authority; have to convince
-No measurement
-Rapid shuffling of teams / projects
-Success theater
-Infrequent acknowledge failures
-No connection to core metrics
-No PM retrospectives
-No tweaking / iteration
-Shiny objects
Types of Roadmaps
-Gantt
-Scrum Sprints
-Story Map
KPI
-holy trinity
1. Retention
2. Growth
3. Montization