Experiment Factory: Idea → Design → Launch → Postmortem (Repeatable)

GrowthMarketing AnalyticsExperimentationProduct Optimization

Problem Statement

The workflow should transform raw growth ideas into structured, testable experiments with clear hypotheses, expected impact, requirements, and measurement plans. It must also support prioritization through ICE/RICE scoring, generate a launch-ready runbook with checklists, and later produce a high-quality postmortem once results are available. The goal is to make experimentation systematic, repeatable, and aligned with the North Star metric and guardrails.

Steps

1

Ingest Inputs

INGEST

Collect the North Star metric, guardrails, funnel baselines, channel constraints, and the list of raw ideas provided by the user. Parse all inputs into structured fields so they can be used consistently across the workflow.

2

Retrieve Contextual Benchmarks

EXECUTE

Pull in any known experiment heuristics, benchmark lift ranges, or historical experiment results when provided or available. This helps anchor hypotheses and expected lift estimates.

3

Generate Structured Experiments

EXECUTE

Turn each raw idea into a structured experiment with hypothesis, expected lift, required changes, risks, and measurement plan. Score each experiment using ICE or RICE to create a prioritized backlog.

4

Generate Launch Runbook

EXECUTE

Create the top three fully designed experiments, including launch checklists and tracking QA checklists. Ensure wording is operational, clear, and actionable.

5

Validate Postmortem Inputs

EXECUTE

Check experiment result inputs for completeness and consistency before drafting a postmortem. Flag missing metrics, unclear attribution, or unclear experiment status.

6

Generate Postmortem

EXECUTE

Draft the postmortem summarizing what happened, why, next steps, and reusable learnings. Ensure the tone is analytical and constructive.

Prompt Configurations

Experiment Structuring Prompt

Model: gpt-4oTemp: 0.7
You are an expert growth experiment designer. Turn the user-provided North Star metric, guardrails, funnel baselines, channel constraints, and raw growth ideas into a structured experiment backlog. For each idea: create a clear hypothesis, expected lift (with a reasonable numeric estimate), required changes, risks, and a measurement plan aligned with funnel baselines. Then score each idea using ICE or RICE. Finally, output the backlog and identify the top 3 highest-scoring experiments. Use a professional, analytical tone. Do not invent metrics not tied to provided or typical growth heuristics. If guardrails conflict with an idea, flag it but still return the structured experiment.

Launch Runbook Prompt

Model: gpt-4oTemp: 0.6
Generate a launch-ready runbook for the top 3 experiments. Include: a launch checklist, tracking QA checklist, key owners, timelines, and success metrics. Ensure all instructions are actionable, unambiguous, and compliant with the provided guardrails. Maintain a professional but clear tone suitable for execution teams.

Postmortem Drafting Prompt

Model: gpt-4oTemp: 0.4
Using the provided experiment results, create a concise but thorough postmortem. Include: what happened, why it happened, what to do next, risks or learnings, and what should be documented for future reuse. Do not invent data—only interpret what is provided and apply common experimentation principles. Use a neutral, analytical tone.

Example Inputs/Outputs

Input:

North Star: Paid activations per week. Guardrails: No SMS usage, no discounts. Funnel baseline: 12% signup-to-activation. Channel constraints: 80% traffic from SEO. Ideas: 'Improve onboarding tooltip', 'Create pre-signup quiz', 'Add exit intent modal'.

Output:

Generated an experiment backlog scoring each idea (e.g., tooltip ICE: 7.3, quiz RICE: 5.1, exit intent RICE: 4.2). Produced detailed hypotheses such as 'If we clarify onboarding steps via contextual tooltips, users will activate more often because friction is reduced'. Expected lift estimates tied to funnel baselines (e.g., +0.8–1.5pp). Created top 3 full designs with launch checklists and tracking QA. Provided a reusable postmortem template.