Problem Statement
The project aims to analyze and improve a team's continuous integration (CI) pipeline for better speed and reliability. By examining current steps, failures, runtimes, and test suites, the workflow will generate a clear set of bottlenecks, optimizations, and rollout guidance. The goal is to provide vendor-neutral recommendations and pragmatic improvements that deliver measurable gains quickly.
Steps
Ingest CI Data
INGESTCollect and parse the provided CI steps, historical failures, average runtimes, and test suite breakdown. Ensure all data is structured for further analysis.
Retrieve Supporting Patterns
EXECUTEIdentify known CI performance and reliability patterns by referencing common practices such as parallelization, caching, and flaky test categorization.
Generate Bottleneck Analysis
EXECUTEAnalyze runtimes, failure frequency, and slow test clusters to surface bottlenecks and reliability issues. Highlight problem areas, both structural and test-related.
Generate Optimization Recommendations
EXECUTEDraft actionable, vendor-neutral improvements including caching, test isolation, parallel execution, and quick wins. Structure them by impact and feasibility.
Validate and Prioritize Plan
EXECUTEEnsure recommendations align with constraints such as safety, rollout feasibility, and organizational standards. Produce a prioritized improvement plan.
Define Metrics and Rollout Steps
EXECUTEPropose measurable KPIs and a safe, incremental rollout approach, including monitoring and iteration steps.
Prompt Configurations
CI Improvement Analysis Prompt
You are an expert in CI performance and reliability. Analyze the provided CI steps, typical failures, average runtime, and test suite breakdown. Produce the following: 1. Bottleneck analysis (runtime, reliability, and structural issues) 2. Clear recommendations including caching, parallelism, flaky test handling, and quick wins 3. A prioritized action plan, ordered by impact and difficulty 4. Metrics to track over time 5. Safe rollout steps emphasizing incremental, low-risk changes Tone: pragmatic, results-driven. Constraints: keep recommendations vendor-neutral; do not propose proprietary tools; do not invent missing data; only base conclusions on the provided inputs.
Example Inputs/Outputs
CI steps: build → unit tests → integration tests → lint → deploy preview. Typical failures: integration test timeouts, flaky network API mocks. Average runtime: 22 minutes. Test suite breakdown: unit (400 tests, 2 min), integration (120 tests, 15 min), e2e (20 tests, 5 min).
Bottlenecks: integration tests dominate runtime and have several timeout-related failures; network-based tests show flaky behavior; preview deployment frequently stalls on container build layers. Recommendations: introduce caching for dependencies; parallelize unit and lint steps; isolate flaky network tests behind stable mocks; shard integration tests; add retry logic for known flakes. Prioritized plan: (1) caching and parallelization, (2) shard integration suite, (3) refactor flaky tests, (4) optimize preview build. Metrics: pipeline median runtime, p95 runtime, flaky test rate, failure categorization trends. Rollout: begin with caching and parallel execution in a feature branch; validate stability; then roll out test sharding; monitor metrics after each change.