Problem Definition Canvas

Seven questions most problem statements skip. The weakest cells ARE your research priorities.

5 Whys: a tree, not a line

Don't peel in a single chain. Branch when "why" has multiple valid answers. Real causes are almost always multi-factorial.

Most common failure: solution disguised as root cause

"Why are users churning? Because we don't have feature X." That's a prescription, not a root cause. Keep asking "what goal can't the user achieve?" until you reach the pain.

Jobs-to-be-Done: 3 dimensions of every job

"Doing nothing" is usually the strongest competitor

If users have a workaround that sort-of works, the problem is real but switching cost is nonzero. If they're not doing the task at all, either the pain isn't severe enough, or you have a new-market opportunity. Both are critical signals.

Opportunity sizing: Frequency × Severity × Breadth × Willingness

Score each dimension 1-5. Composite ≥ 300 = critical opportunity. Under 100 = validate before committing.

Problem-Solution Fit: the 6 tests

Prioritization anti-gaming rules

1. Score dimensions independently. Don't reason backward from desired priority.
2. Document evidence behind each score.
3. Disclose the scorer. Accountability enables honesty.
4. Re-score when new evidence arrives.
5. Never average across scorers without discussing divergence. Impact=5 vs 2 averaging to 3.5 hides genuine disagreement that IS the finding.

The 6 evidence tiers

Triangulation requires independence

Five interviews with the same segment, same researcher, same interview guide are one source, not five: Independence requires different methods, different populations, or different time periods.

Frequency × Intensity Matrix

The most useful decision tool for interview synthesis.

Sample size: what you can and cannot claim

The biggest blind spot: the non-consumer

The O-I-R-C-W cascade (for every recommendation)

1. Observation: What the evidence shows (with tier)
2. Implication: Why it matters (the mechanism)
3. Response: What to do (specific action + owner + timeline)
4. Confidence: H/M/L + the key assumption
5. Watch: Observable signal; if threshold crossed, re-assess

Hamilton Helmer's 7 Powers

Switching costs: decompose, don't rate

"High/medium/low" is meaningless. The key insight: customer-created switching costs beat vendor-imposed ones. The most durable costs are ones users CREATE (data, workflow embedding, identity), not ones vendors impose (contracts, exit fees). Vendor-imposed moats erode.

Aggregation Theory (Ben Thompson)

Three characteristics of aggregators: (1) direct user relationship, (2) zero marginal cost to serve, (3) demand-driven multi-sided networks with decreasing acquisition costs.

Christensen's Disruption + COAP

Three disruption vectors:

• Low-end: "Good enough" at dramatically lower cost (over-served customers churn down). Detection: customers saying "I don't use half the features."
• New-market: Serves non-consumers (Canva for people who aren't designers).
• Hybrid: Both low-end AND new-market simultaneously (Notion vs Confluence).

Blue Ocean: the ERRC Grid

For each factor the industry competes on: Eliminate, Reduce, Raise, Create: Value innovation = simultaneous differentiation AND low cost. You don't beat competitors; you make them irrelevant.

Crossing the Chasm

Early Adopters buy on vision. Early Majority buy on references from people like themselves. Early Adopters are individualists who don't give referrals that satisfy pragmatists. There's no natural word-of-mouth bridge. You can't grow across; you have to jump.

Win/Loss pattern taxonomy

Sales attributes wins to product quality and losses to price. Both are usually wrong. Real reasons cluster around integration fit, switching costs, trust/brand, references. None show up in CRM notes.

9 failure modes

North Star Metric: The 5-criteria rubric

Score every NSM candidate. A viable NSM scores 4/5 or higher.

Goodhart's Law: 4 variants, 4 counter-strategies

"When a measure becomes a target, it ceases to be a good measure." Not one phenomenon: four distinct failure mechanisms.

Counter-metric pairing

Every primary metric needs a guardrail.

Leading vs Lagging: temporal lag matters

Your monitoring cadence must match the metric's lag. Checking a structural metric daily creates noise anxiety. Checking an immediate metric monthly means you miss fires.

Retention curve shapes

Product-Market Fit (Sean Ellis test)

"How would you feel if you could no longer use [product]?"

12 failure modes (condensed)

F1: Outcome-First

A spec defines a change in the world, not a list of features. Test: what is true after that was not true before?

F2: Acceptance Criteria Taxonomy: 5 types

Most specs only include behavioral criteria. Missing the other four is where scope creep and bugs live.

F3: Scope Boundary: named exclusions

OUT of scope must name specific capabilities, not vague categories.

F4: Executor Context Model

The "mind-reader executor" anti-pattern: writing for someone who shares your full context. Real executors don't.

F5: Failure Conditions (not error handling)

4 types: Data Model, Integration, Process/Workflow, Authorization. For each, pattern: "This spec assumes X. If [observable deviation], STOP and escalate."

The Worst Misinterpretation Test

For every acceptance criterion, ask: "What would the executor build if they interpreted this in the worst possible way?" If the worst reading produces the wrong result, add a failure condition or clarify. Especially critical for AI agents, who interpret literally.

Zero-Question Protocol

The integration test. Simulate reading the spec as the specific executor type. Write down every question they'd ask. Classify by ambiguity type. Resolve or mark TBD with owner.

9 spec failure modes

The 6-element narrative arc

Every strategic narrative needs all six in order. Miss one, and the narrative breaks in a predictable way.

"Why Now": 4 structural preconditions

Differentiation without timing is not a narrative. "We are better" isn't enough. You need "we are better AND the structural conditions just arrived."

• Technology shift: specific milestone, not "AI is improving"
• Market shift: measurable: market size, buyer budget reallocation
• Regulatory shift: specific regulation with effective date
• Behavioral shift: observable with data, not assumed

Audience adaptation: same story, different lead

The invariance principle: the core narrative stays structurally identical. What changes is what you lead with, what evidence you emphasize, what objections you pre-address, what CTA you make.

C-E-I: every claim needs 3 parts

Claim (declarative, no hedging) → Evidence (specific data, with tier) → Implication ("this means...")

SCR structure (Minto Pyramid)

Answer first. Grouped supporting arguments. Evidence last. Execs read top-down and stop when convinced.

Executive audience calibration

Same $2M investment, framed four ways:

Bezos Type 1 vs Type 2 decisions

Option architecture rules

• Always 2-4 options: One = rubber stamp. 5+ = you haven't done the work.
• Options must differ structurally (not just scope/timeline).
• Always include "do nothing." Execs need to understand cost of inaction.
• Name the irreversible commitment for each option.
• State your recommendation: Execs want your judgment.

Narrative failure modes

Three dimensions: Power, Interest, Position

Score each independently. Never conflate.

Four sources of power (Net Power = max of these)

Position taxonomy (5 levels)

Alignment sequencing

The ORDER matters as much as what you say. Each conversation changes the next stakeholder's calculus.

1. Strongest champion: Build momentum. Get explicit endorsement for use elsewhere.
2. Swing voters: Before they hear the blocker's frame. First framing anchors.
3. Neutral high-power stakeholders: With champion endorsements and swing momentum.
4. Skeptics (convertible): With maximum coalition. Address specific concerns.
5. Blockers: LAST, with full coalition. Goal: make opposition costly, not convince.

Decision archaeology: excavating hidden history

Every stakeholder position has a history. A VP who championed a failed initiative like yours will be defensive. These histories are rarely stated.

The "Burned Before" pattern

Five excavation questions (ask trusted insiders, not stakeholders)

1. What happened the last time someone proposed something similar?
2. Who fought hardest for/against, and why?
3. What has changed since then?
4. Who do the key decision-makers listen to on this kind of decision?
5. What would make this initiative feel safe to the skeptics?

Objection types need different levers

Alignment has a shelf life

Success durability ladder

Market Entry Thesis: 3 pillars

Most people only have one (usually "the market is big"). All three are required.

ICP scoring: The beachhead segment

A segment must be specific enough to name 100 prospects. "SMBs" is not a segment. "10-50 person design agencies using Figma who hit collab limits at 15+ users" is.

Channels: start with ONE

Launch gating: each phase answers a question

Kill criteria must pass SMART-K

• Specific: names the exact metric
• Measurable: can be pulled from a dashboard
• Actionable: leads to a specific decision (not "reassess")
• Realistic threshold
• Time-bound: has a deadline
• Kill-linked: specifies what stops if triggered

Pricing model: value drives the model, not the market leader

Good-Better-Best packaging

Product strategy: the vision-to-roadmap cascade

Portfolio balance

NOT-Doing as integrity check

Strategic tensions: signal, not noise

A roadmap where everything coexists peacefully means hard decisions were avoided. Five tension types:

• Resource: Two bets need the same people/budget
• Strategic: Bets pull in different directions (enterprise vs self-serve)
• Timing: Both need to go first
• Cannibalization: One bet's success undermines another
• Platform: Architecture for one constrains another

Structure (9 steps)

1. Clarify (3 to 5 questions). Ask focused questions to narrow the scope before solving. Cover region (US or global), platform (mobile, web, native), timeline (near-term improvement or 0-to-1), what the company cares about right now, which part of the product you are focused on, and any unmet needs worth flagging.
2. State the mission and pick a goal. Lead with the company mission, then tie your answer to a specific, measurable goal. The goal should be narrow enough to drive your segment and solution choices. For example: "Uber's mission is reliable rides for everyone. I will focus on improving rider retention in suburban markets."
3. Identify and prioritize segments. List 2 to 3 meaningful user segments with short descriptions, then pick one to design for. Shortlist by market size, fit with the stated goal, where the company has a unique right to win, and which segment is most underserved. The strongest answers pick by differentiation potential, not by sheer size.
4. Map pain points along the journey. Walk through the user journey stage by stage (Discover, Evaluate, Use, Return) and name what is slow, confusing, or missing at each stage for your chosen segment. Aim for 3 to 5 concrete pain points, not abstractions.
5. Prioritize the pain. If you have multiple strong pain points, rank them by severity and frequency. Pick the top 1 or 2 and briefly say why these matter more than the rest.
6. Generate solutions (3 to 5 per pain). For each top pain, generate a range: Basic (low effort, incremental), Middle (meaningful change, moderate effort), and Moonshot (high effort, high reward, creative). Add a third column for each solution noting technical risk, adoption risk, or edge cases.
7. Pick one solution with the Impact vs Complexity matrix. Place your candidates on a 2x2: high-impact, low-complexity is the winner. Briefly explain why this one beats the others and reference the specific pain it unblocks.
8. Define success metrics with formulas. Name metrics across Activation, Engagement, Retention, Revenue, and Guardrails. For each, include how you would calculate it (not just the name). Pick one North Star that directly measures the behavior change you are trying to drive.
9. Name risks and a launch plan. Call out 1 or 2 real risks (adoption, technical, unintended consequences) and suggest a launch approach: A/B test on a specific segment, phased rollout, or beta with design partners. Close with a concise wrap-up restating goal, segment, solution, and metric.

The 5 tips that transfer across every answer

Featured examples

Common failures

• Solutionizing too early. spend time on who and why before jumping to what
• "DAU / time spent" metrics: vanity. The NSM should measure behavior change.
• Picking segments by size, not by differentiation potential
• Five shallow solutions instead of one deep one
• Forgetting guardrails: optimizing one metric always risks breaking another

Structure (9 steps)

1. Clarify and briefly restate the product. Confirm what the product does, who uses it, whether it is B2B or B2C, and what stage it is in. A one-sentence restatement grounds your answer and signals you are not just reciting a framework.
2. Company goal plus product goal. State the company-level goal (revenue, growth, market share) and explain how this product contributes to it. This is where your strategic thinking score comes from. For example: "Alphabet's goal is growing ad revenue. YouTube's product goal is being the preferred platform for creators and viewers, which directly feeds ad revenue."
3. Name the product stage. Is it in Launch, Growth, Maturity, or Decline? The stage determines which metrics matter most. Early-stage products care about activation and PMF signals. Mature products care about retention and monetization.
4. Segment users meaningfully. Break users into 2 to 3 segments and explain why the segments matter for metrics. A power user and a new user should not be measured the same way. In marketplaces, define segments on both sides (creators and viewers, buyers and sellers).
5. Map the user journey and funnel (the bulk of the answer). Use AARRR (Acquisition, Activation, Retention, Revenue, Referral) as your backbone. At each stage, define metrics with: name, formula, what behavior it captures, and why it matters for the goal.
6. Pick one North Star metric. Choose the single metric that best captures the core value your product delivers. A good NSM is broad enough to reflect overall health, specific enough to be actionable, and tied to the value users actually get (not just business revenue).
7. Pick 3 to 5 supporting metrics. These are the levers that feed the North Star. Each should be directly actionable by a team, causally linked to the NSM (not just correlated), and non-overlapping (each captures a different lever).
8. Name guardrail metrics. Which metrics must NOT get worse as you optimize the North Star? Cover performance (latency, crashes), cannibalization of other products, quality (support tickets, NPS), and user health (opt-outs, uninstalls, reported content).
9. Wrap up in one sentence. Restate the North Star, the key supporting metrics, the guardrails, and tie it all back to the product goal. This is where the judgment score comes from.

5 transferable tips

Common pitfalls

• Vanity metrics: Total downloads, registered users, page views. Prefer ratios and active measures.
• Rising ratio, shrinking base: Conversion 30% → 50%, but users 100 → 50. You're celebrating while business shrinks.
• No guardrails: Optimizing one metric always risks breaking another.
• B2B/B2C blind spot: B2B activation might be "team invited 3+ members"; retention is contract renewal; buyer ≠ user.
• Numbers without meaning "Track conversion rate" without explaining what it captures or why it matters here.

Featured example: YouTube

Supporting: avg watch time per DAU, D7 viewer retention, upload frequency (established+ creators), recommendation CTR, Shorts watch time % of total.

Guardrails: creator RPM (creator economics), viewer NPS, ad load per hour (UX), content policy violation rate (safety), creator 90-day retention (supply pipeline).

The non-obvious NSM: Google Maps

The 5-branch RCA tree

Metric changed
 |-- Technical: Bugs, infra outage, logging errors, pipeline
 |-- Product: Feature/UI change, A/B gone wrong, OS update
 |-- User: Segment shift, behavioral change, fatigue, seasonal
 |-- Business: Competitor move, pricing change, marketing spend
 |-- External: Regulation, news event, economic shift, partner outage

Structure

1. Sanity-check the data before investigating. Make sure the signal is real before you start hunting for causes. Confirm the metric calculation has not changed, logging is working, and the movement is statistically meaningful. Is it sudden (suggests a specific event) or gradual (suggests a trend)? Is it across all platforms, geographies, and segments, or concentrated? Could it be seasonal?
2. Rule out external factors. Work through things outside your control before blaming something internal. Check for competitor moves (new launches, viral moments, price cuts), regulatory changes, macro events, shifts in indirect competition (TikTok pulling time from YouTube), changes in entry points (SEO rankings, app store, paid channels), and partner outages.
3. Investigate internal factors. Now look at things within your control. Check for recent releases, whether other metrics also moved (multiple metrics shifting together usually means systemic, not feature-level), spikes in crashes or errors, cannibalization from new features, conversion breaks at each funnel step, pricing or policy changes, and capacity or latency issues.
4. Synthesize and rank causes. State the most likely root cause and explain why you believe it over alternatives. If you cannot pinpoint one, rank the top 2 or 3 by likelihood. Say explicitly what data you would need to confirm.
5. Propose fix and prevention. Name a quick fix (this week), a deeper investigation (this sprint), what metrics you would monitor to confirm the fix worked, and a process change that prevents recurrence (better testing, staged rollouts, anomaly detection).

5 tips

Featured examples

AI-specific RCA: the relevance × groundedness 2×2

When the metric that dropped is AI quality (thumbs-down, task completion, hallucination flags), don't reach for the general tree first. This 2×2 tells you which layer of the system to investigate.

One of the most reusable AI-PM frames. For deeper eval methodology, see Evals Deep Dive.

High-level mental model (every sub-type)

Skip Company → answer disconnected from business. Skip Ecosystem → building in a vacuum. Skip Execution → consultant with no plan.

5 tips

Marketplace cold start — memorize the trio

For any chicken-and-egg or two-sided platform question, drop the matching example:

• Tinder threw college parties to seed both sides at once. Localized density problem solved by a localized event.
• Uber subsidized drivers so supply existed before demand could complain. Subsidize the harder side.
• Airbnb scraped Craigslist to bootstrap supply. Tap an existing pool, then pull onto your platform with a better experience.

The pattern behind all three: pick one side, subsidize or import it, until the other side has a reason to show up. Two-sided launches never start two-sided.

Multi-homing kills lock-in. When users can be on Uber and Lyft simultaneously, you don't win on switching costs. You win on density (cheaper / faster) or experience (Uber Eats integration, ride memory).

The 3 question types

The core mental model

Competing options or signals
    ↓
  Product Goal / NSM
    ↓
  Data to inform the decision
    ↓
  A/B test to validate
    ↓
  Ship / iterate / kill

Structure (for conflict and prioritization questions)

1. Clarify the problem. Restate the question in your own words and confirm your assumptions about the product, users, and goal. A minute of clarification saves five minutes of solving the wrong problem.
2. Identify the high-level goal or North Star. Before evaluating trade-offs, anchor on what the product is ultimately optimizing for. Every subsequent argument ties back to this.
3. Run the hypothetical extreme. Mentally push the metric to 100% and then to 0. What breaks at each extreme? This reveals which direction matters more and exposes second-order effects you would otherwise miss.
4. Define the data or experiment. For metric conflicts, name the data you would pull to diagnose (funnel metrics, cohort analysis, segment breakdowns). For prioritization, describe the A/B test that would settle the question.
5. Make the decision. Use the data or experiment framework to make a concrete call. Do not hedge: say what you would ship, what you would kill, and why.
6. Next steps and guardrails. Name follow-up experiments, monitoring metrics, and guardrails. A strong answer leaves the interviewer with a clear sense of how you would operate this decision week over week.

Experiment design toolkit

Setup checklist

1. Write a falsifiable hypothesis. Format: "If we do X, metric Y will change by Z% because [mechanism]." If the hypothesis cannot be wrong, it is not a hypothesis.
2. Control vs treatment (one variable only). Change exactly one thing. If you change two, you cannot attribute the result to either.
3. Audience design. Randomize, stratified by key dimensions (geography, tenure, platform) so the control and treatment groups are comparable.
4. Sample size computed, not guessed. Use power analysis to determine how many users you need. "Let's run for two weeks and see" is not an experiment, it is a wish.
5. Duration covers real behavior cycles. Run for at least 1 or 2 full behavior cycles, which typically means 2 to 4 weeks minimum. B2B experiments often need 4 to 6 weeks to capture procurement rhythms.

Metric tiers

5 tips

Featured example: TikTok: sponsored post vs follow suggestion

Hypothetical extremes: All sponsored = ad fatigue, DAU drops, revenue falls. All follows = no direct revenue but high engagement.

Experiment: Control 50% sees sponsored, treatment 50% sees follow. Stratified by geography, tenure, usage. 4-6 weeks. P0: ARPU (7-day, 30-day). P1: CTR, follow-through, retention. P2: satisfaction, report rates.

Decision: ARPU higher + retention flat → ship ads. ARPU similar + retention better → ship follows (retention compounds). ARPU higher + retention drops → calculate breakeven.

Hybrid: Ads for high-engagement users, follow suggestions for new/low-engagement. Set a kill switch on retention.

Structure

1. Clarify the scope. Agree on boundaries before doing any math. Confirm geography (US, global, a specific city), time period (per day, per year), definition (what exactly counts as "revenue" or "user"), and any constraints (physical limits, operating hours, capacity).
2. Pick your approach and name it. Top-down starts from a known large number (population, total market) and filters down through percentages and segments. Bottom-up starts from a single unit (one store, one user, one hour) and scales up. If time allows, do both and cross-check; if they are 10x apart, one of your assumptions is wrong.
3. State the top-line equation. Write out the high-level formula before breaking anything down. For example: "Revenue = Number of users × Transactions per user × Average transaction value." This gives the interviewer your mental model up front.
4. Build the estimation tree. Break each component into sub-components. For each, label what you are estimating, state the arithmetic, explain the logic, and make assumptions explicit. Use segmentation (age brackets, urban vs rural, weekday vs weekend) when a number feels too big to estimate directly.
5. Calculate cleanly. Work through the math step by step. Round generously; false precision is worse than fuzzy truth.
6. Sanity-check against a different angle. Compare your answer to a known benchmark, a per-capita ratio, or an intuitive sense. If it feels off, identify which assumption is most likely wrong and adjust. State your confidence level explicitly.

5 sharp moves

Useful reference numbers

Featured example: Piano tuners in Chicago (~300)

Structure

1. Name the product (10 seconds). Pick something you genuinely use and have real opinions on. Do not pick Google or Apple to sound smart; pick a product where you can go deep on specific design decisions and business model choices.
2. Say what it is (30 seconds). Give a one-line description of what it does and who it is for. Do not over-explain, the interviewer probably knows the product.
3. Explain why it is good (1 to 2 minutes). This is where you show product taste. Go beyond features to talk about the core insight the product got right, specific design decisions that make it work, the moat (network effects, data, habit, ecosystem), and how making the product better also makes the business better. Aim for 2 or 3 specific reasons, not a laundry list.
4. Name 2 or 3 user segments (30 seconds). Show that you think about different users with different needs, not just your own use case. This signals PM-level thinking rather than personal preference.
5. Propose one focused improvement (2 to 3 minutes). Pick ONE weakness and go deep: name the pain point specifically, propose a scoped solution, explain why this improvement over others, and define the metrics you would track. One focused improvement shows more skill than five shallow ideas.
6. Close with competitive context (30 seconds). Name who else plays in the space, what the product's edge is, and what the biggest threat to that edge is. This signals you think about products as businesses, not just apps.

5 transferable tips

Picking your products: a mix of 2-3

• One you use daily: Deepest opinions
• One less obvious: Shows breadth. A niche tool, physical product, B2B. Interviewers hear "Instagram" all day.
• One relevant to the company: Social company → social product. B2B → enterprise product.

Featured picks

15 terms — define each in one sentence

ELI5 — three to rehearse aloud

Problem-solving prompts (drill list)

1. Scale: How would you scale ChatGPT's image-generation queue when demand 10×'s overnight?
2. Cost: A new feature uses 5× more LLM tokens than expected. How do you investigate and respond?
3. Latency: P95 latency spiked 2× overnight with no code change. Diagnose.
4. Privacy: A user reports their data appeared in someone else's session. Walk through your response.
5. System design (PM-shaped): How would you let enterprise customers fine-tune a foundation model on their private data without us seeing it?

Defendable claims — my own shipped tech work

When the interviewer asks "tell me about a technical decision you made" — anchor in these. Each is one sentence a smart engineer would respect.

Why AI questions feel different

Frame 1: Latency × Accuracy × Cost (pick two)

Every AI product trades these three off. Name your point on the triangle and why.

Say it out loud: "This is a chat surface, so I'd take the latency hit for accuracy because users wait for the answer." That single sentence differentiates a generic AI answer from a senior one.

Frame 2: Autonomy spectrum (L1 → L5)

For any agent question, place it on the ladder and justify. The autonomy level is the design.

Frame 3: Three-layer guardrails (pre / in / post)

When asked about safety or trust, structure your answer in three layers. No single layer is enough.

AI vs rules — when to choose which

Default to the simplest thing that works. "AI" for a problem rules can solve is bad engineering — slower, more expensive, less reliable. Saying this out loud in an interview is a hire signal.

The eval-first roadmap (the senior-round move)

Lines worth memorizing

Anti-patterns (no-hire signals)

How to plug the overlay into a normal answer

The mindset shift: bounded → unbounded

Traditional software is bounded. The user clicks buttons, the app responds in predictable ways. QA = unit tests for known input → expected output.

AI products are unbounded. User types into an empty box. Models return different answers to the same prompt. So you stop trying to map every point. You start profiling the distribution.

The four PM skills

1. Guidelines — what does good look like? (qualitative). Most teams skip this because it's the hardest.
2. Metrics — how do I quantify "good"? (qualitative → numeric)
3. Eval sets — over what space do I measure? (sample the distribution)
4. Way of working — how does this run continuously, not once? (process)

Guidelines — the qualitative foundation

Ask five people "what is a good Copilot answer?" → seven answers. Until the team agrees, no eval is trustworthy. Guidelines are the PRD for an AI product.

Two anchors: Google's Search Rater Guidelines (~150 pages, ~300 conditions) and OpenAI's Model Spec (hard tradeoffs explicit). The move: go from coarse ("good search result") to granular ("trustworthy source + relevant + visually clean + not clickbait + fresh enough").

Metrics — three axes

Quality correlates with behavior. As quality climbs, usage climbs, retention climbs, revenue climbs. The eval scorecard is upstream of the business scorecard — which is why PMs own it.

Eval sets — sampling the distribution

The tightness ladder — how confidence is built

RAG diagnostic 2×2 — the most reusable AI-PM frame

Improving accuracy — the ladder

1. Prompt engineering — cheapest, fastest, surprising headroom
2. In-context learning — few-shot examples in the prompt
3. Grounding (RAG) — retrieve real context
4. Fine-tuning — last resort; expensive, less general

Try in this order. Fine-tuning is rarely the right first move.

Common pitfalls

Two lines worth remembering

Showing this through projects

Demonstrate by pointing at where I've done it, not by reciting frameworks.

• M365 Copilot evals stack — SEVAL platform, GroundLeo + LMChecklist scorers, 7-attribute rubric, golden set design, calibration target.
• Voice Notes eval infrastructure — how the rubric, query set, and WER targets came together.
• Daily Brief AI evals — same methodology, different feature shape.

Knowing the theory is table stakes. Showing the artifact — a real rubric, a real scorecard, a real shipped decision gated on a real delta — is what an AI-era PM interview actually tests for.

1: Behavioral > Stated preference

When what users say conflicts with what users do, behavior wins, not because users lie, but because they describe who they want to be, not who they are. They anchor on recent experiences. They want to help and agree with your implied hypothesis.

2: Evidence tiers (T1-T6)

Grade every claim by how you know it. T1 (behavior) beats everything. Five T5 sources are still T5, not T2.

3: The O-I-R-C-W cascade

Every recommendation must trace: Observation (with tier) → Implication (mechanism) → Response (action + owner + timeline) → Confidence (H/M/L + key assumption) → Watch (signal if wrong). A finding without a mechanism, action, and falsification signal is trivia.

4: Calibrated confidence (no weasel words)

5: Counter-metrics (Goodhart's 4 variants)

Every metric can be gamed. The 4 mechanisms. Regressional, Extremal, Causal, Adversarial. each need different counter-metrics. No primary metric without a guardrail.

6: Leading vs Lagging: match cadence to lag

Immediate metrics (errors, latency) → real-time alerts. Short (DAU, signups) → daily. Medium (activation, NPS) → weekly. Long (retention, LTV) → monthly. Structural (market share) → quarterly. Checking a structural metric daily creates noise anxiety.

7: Cross-framework contradictions are signal

When 7 Powers says "strong moat" but COAP says "profits migrating away" that tension IS the insight. Don't paper over. Don't average. Surface explicitly. Reconciling prematurely hides the most valuable finding.

8: Adversarial self-critique (mandatory)

After any analysis, list 3+ genuine weaknesses, not generic "research could be better." Specific, steelmanned arguments against your own conclusion. If you can't argue against your recommendation, you haven't stress-tested it.

9: Assumption registry

Every plan rests on assumptions. List them. For each: what it underpins, confidence, what would invalidate it. The difference between "the plan failed" and "assumption #3 was wrong, which means we need to revise sections 4 and 7."

10: Inevitability over assertion

The best narratives, exec docs, and alignment campaigns all share one property: they make the right conclusion feel inevitable, not because you said so, but because the structural argument leaves no other option.

11: Sequence creates momentum

In alignment: champion before blocker. In exec writing: answer before evidence. In narrative: tension before resolution. In compression: source before derivative. Order matters as much as content.

12: Alignment decays: refresh it

Verbal agreement from 4+ weeks ago without reinforcement = expired. Narratives built on 6-month-old data need re-validation. Positions shift when you're not looking.

13: Steelman everything

Objections in narratives. Counterarguments in writing. Stakeholder resistance in alignment. State the opposition's case stronger than they would. If your response only works against the weak version, you're not prepared.

14: The "do nothing" competitor

The largest competitor for most products is not another product: it's inertia. Spreadsheets, manual processes, "just winging it." If users are doing nothing, either the pain isn't severe enough or you have a new-market opportunity. Both are critical.

15: Customer-created beats vendor-imposed

The most durable switching costs are ones users CREATE through use (data accumulation, workflow embedding, identity), not ones vendors impose (contracts, exit fees). Companies relying on imposed switching costs sit on erosion-prone moats.

16: Decomposition + segmentation = diagnosis

Every RCA follows: decompose the metric into moving parts, segment by platform/user/geography/version, identify which segment moved, drill down. Turns "mystery 15% drop" into "one code change on one platform" in hours. Mechanical and repeatable.

17: Prevention is the inverse of RCA

RCA finds what broke. Prevention builds the system that catches it before users see it next time. Every RCA output should include a prevention system change.

18: The underserved segment beats the largest segment

Most candidates pick by size. The winners pick by differentiation potential. Alarm for blind users. Airbnb for low-intent browsers. Headspace for rookies. The segment where you can create the most pull.

19: Upstream pain beats downstream friction

Preparation friction (forgot items) stops people from engaging at all. Post-engagement friction just makes it worse. Upstream blockers compound.

20: Fixing supply fixes demand

Two-sided marketplaces: fix the supply side (admins, sellers, creators) and demand follows. The supply side is usually the lever, even though demand is what's measured.

By company

By product stage

Porter's 5 Forces

Measures how much profit an industry allows. More forces working against you = less attractive.

5C Analysis

Broader than Porter. full situation analysis.

• Company: Strengths, capabilities, brand, culture
• Customers: Segments, needs, willingness to pay
• Competitors: Direct, indirect, potential entrants
• Collaborators: Partners, suppliers, distributors
• Climate: Political, economic, social, technological (PESTEL)

4P Marketing Mix

GTM / pricing questions: Product / Price / Place / Promotion

SWOT

Match: use Strengths to capture Opportunities, watch where Weaknesses meet Threats.

PESTEL

Macro scan for regulation-heavy or new-country questions. Political / Economic / Social / Technological / Environmental / Legal.

Ansoff Matrix (growth)

Profit Tree

Profit = Revenue - Costs
Revenue = Users × ARPU = (Traffic × Conversion) × (Frequency × Avg Price)
Costs = Fixed (rent, salaries, infra) + Variable (COGS, marketing, support)

Break whichever side the question points to. If revenue is flat, decompose revenue. If margins are shrinking, decompose costs.

Jobs-To-Be-Done (JTBD)

Reframes products as hired to do a job, not bought because of features. Use when "what is this product actually for" feels shallow or when competitive lines blur.

Three questions:

1. What job is the user hiring this product to do? Make it specific — "kill 30 minutes in a checkout line" beats "be entertained."
2. What were they hiring before? Often non-obvious — Netflix's competitor wasn't HBO, it was sleep.
3. Functional + emotional + social dimensions? A diet product has all three (lose weight / feel in control / be seen as healthy).

Marketplace Dynamics

• Supply vs Demand which side is harder to acquire? That's your constraint.
• Chicken-and-egg: Subsidize the hard side first. Uber subsidized drivers. Airbnb seeded from Craigslist. Tinder seeded with college parties.
• Ownership vs Aggregation: Netflix owns originals (higher margin, more control). YouTube aggregates (lower margin, more scale).
• Trust & safety: Ratings + insurance + identity verification = marketplace trust. Without it, no transactions.
• Multi-homing: Can users be on multiple platforms? High multi-homing = low switching costs = hard to build a moat.

When to reach for what

Acquisition patterns

Engagement patterns

Retention patterns

Monetization patterns

Populations & households

Technology

Economy

Time

Utilization & cycle anchors

SaaS / business benchmarks

Tier 1 — Recruiter

Knows: process, timing, comp, role mechanics, hiring bar. Can't answer deep product/culture.

• What does the rest of the interview process look like, and what should I prep for each round?
• Who would I report to, and who are the other PMs on the team?
• How is the team structured — solo PM, pod, multiple PMs?
• What's the team's hiring history — recent hires from where, what backgrounds?
• What's the timeline for a decision?

Tier 2 — Hiring Manager

Knows: team strategy, what success looks like, what gap they're hiring for.

• What does success look like at 6 months, 12 months, 24 months?
• What's the most important problem this team is working on right now? Why is it hard?
• What's the gap you're hoping this hire fills?
• How do you make build/buy decisions on this team?
• Where does the team's roadmap come from — top-down, bottom-up, customer-led?
• What's a recent decision the team got wrong, and how did you respond?

Tier 3 — Peer PM

Knows: day-to-day reality, eng partnership, politics, unwritten rules. This is where you get the real picture.

• Walk me through a recent launch — what was the hardest part?
• How do you decide what's on your roadmap vs what isn't?
• What's something you wish you'd known before joining?
• How do PMs grow here — what does promotion look like?
• What do you spend the most time on that you didn't expect to?

Tier 4 — Cross-functional (Eng / Design / Data)

Knows: how PM decisions translate into work, where PMs typically fail.

• To Eng: What's the best PM you've worked with done differently? Where do PMs slow you down? Where do they add the most value?
• To Design: How is research scoped — PM-led, designer-led, shared? When does PM bring you in — problem stage or solution stage?
• To Data: How are experiments designed — PM-led hypothesis, data-led? What's a metric the team chases that you think is wrong?

Tier 5 — VP / Skip-level

Knows: company strategy, org dynamics, the next 18 months. Doesn't want day-to-day talk.

• What's the most important bet this org is making over the next 18 months that isn't obvious from the outside?
• What's the biggest constraint on this org — talent, capital, distribution, technology?
• How is the company thinking about [major industry shift relevant to the role]?
• What did you used to believe about building great products that you've changed your mind on?
• Brave one: What's the question you wish a candidate would ask you that they never do?

Avoid (any tier)

How to deploy

AI / Evals (senior round closers)

Strategy

Metrics

RCA

Product sense

Favorite product

Experimentation

Cross-cutting

Product Sense "How would you increase Airbnb bookings?"

Metrics "Define success metrics for YouTube"

RCA "Instagram newsfeed engagement -10%"

Strategy "Grow Netflix 3x in 5 years"

Experimentation "Design an experiment to increase Target revenue"

Guesstimate "Piano tuners in Chicago" (~300)

Favorite Product "Wispr Flow"