Problem Statement
The workflow assists analytics engineering and data platform teams in triaging data quality alerts and producing structured, stakeholder-ready communications. It turns raw alert inputs into an incident summary, a prioritized triage plan, decision guidance, and two communication drafts. The workflow ensures accurate, blameless, operational updates while preventing premature conclusions about root causes.
Steps
Ingest Alert Inputs
INGESTCollect and parse the alert text, timestamp, affected pipeline or job, recent code changes, and known downstream consumers. Normalize these into a structured representation for further processing.
Retrieve Context & Blast Radius
EXECUTESearch for metadata such as lineage, downstream dashboards, and other affected assets. Assess which decision-making processes might be using possibly incorrect data.
Generate Incident Summary
EXECUTEDraft a concise summary describing what failed, when, initial impact, and any immediate risks. Maintain a calm, operational tone.
Generate Triage Plan & Validations
EXECUTEProduce a 10–20 minute triage plan with fast isolations, likely and possible root causes, and explicit validation steps without asserting causality prematurely.
Validate Recommended Actions
EXECUTECheck that suggested next-best actions follow operational constraints such as stopping bleeding, pausing schedules, or avoiding risky backfills. Ensure recommendations align with evidence.
Draft Communications
EXECUTEProduce two communication drafts: one short update for stakeholders and one detailed internal incident-thread update. Follow tone, word limits, and guidance.
Prompt Configurations
Data Quality Incident Triage Prompt
You are an analytics engineering incident responder. Given the alert text, timestamp, affected pipeline or job, recent changes, and downstream assets, produce: 1) A clear incident summary (what failed, when, blast radius). Maintain calm, operational, blameless tone. 2) A 10–20 minute triage plan with prioritized checks. Include fast isolations first. Do not claim root cause; label hypotheses as LIKELY or POSSIBLE. 3) A next-best-actions decision tree (fix forward vs rollback vs backfill). Include stop-the-bleeding steps when relevant. 4) Two comms drafts: a stakeholder update under 120 words and a detailed internal incident-thread update. Include ETA only if supported. Constraints: - Never invent data not provided. - Only use supplied metadata and reasonable inference. - Always call out correctness risks and downstream decision impacts. Tone: calm, operational, action-oriented, blameless.
Example Inputs/Outputs
Alert: dbt test failure - nulls found in primary key for model dim_customer. Timestamp: 2026-01-31 14:10 UTC. Affected job: nightly_dbt_run. Recent change: PR #482 added new merge logic. Downstream: customer360 dashboard and churn model.
Incident Summary: At 2026-01-31 14:10 UTC, dim_customer failed a primary-key null check during nightly_dbt_run. Downstream analytical assets (customer360 dashboard and churn model) may currently display or propagate incorrect customer records. Triage Plan: Fast isolations include checking recent merge logic diff, inspecting row counts for unexpected spikes, and reviewing upstream staging freshness. POSSIBLE causes: merge condition change; upstream late-arriving data. LIKELY cause cannot yet be determined without validating source timestamps and merge predicates. Next Best Actions: If upstream issue is confirmed, pause downstream refresh. If merge logic is flawed, rollback PR #482. If data is only partially affected, consider backfill. Comms: Stakeholder update under 120 words summarizing impact and ongoing triage, plus detailed internal incident-thread update including checks underway and validations planned.