CI/CD Workflow Watchdog and Remediation

I'll help you set up automated CI/CD workflow monitoring and failure remediation. Let me understand your needs:

## Understanding Your CI/CD Monitoring Needs

**Repository and Workflow Information:**
- Which repository's workflows should I monitor?
- What is the default branch? (main, master, develop)
- How frequently do workflows run? (on commits, PRs, scheduled)
- Are there specific workflows that need priority monitoring?

**Failure Response Requirements:**
- Should I automatically fix issues or alert you first?
- What types of failures are most common? (test failures, build errors, deployment issues)
- Do you want immediate notifications or periodic summaries?
- Are there workflows that should never be auto-fixed?

**Analysis Depth:**
- How detailed should failure analysis be? (basic summary, comprehensive root cause, token-efficient)
- Do you need historical pattern analysis?
- Should I track recurring issues?
- Do you want performance metrics for workflows?

**Remediation Strategy:**
- What should happen after a fix? (auto-commit, create PR, create issue)
- Should fixes be tested before committing?
- Do you need approval workflows for certain changes?
- How should successful remediations be documented?

**Communication Preferences:**
- Who should be notified of failures? (team, specific individuals)
- What notification channels? (GitHub issues, Slack, email)
- How verbose should reports be?
- Do you want success notifications too?

---

Based on your answers, I'll provide:

## 1. Workflow Monitoring System

Continuous monitoring capabilities:
- **Latest Run Evaluation**: Automatic checking of most recent workflow runs
- **Status Detection**: Success, failure, cancelled, timed-out states
- **Branch Tracking**: Default branch and PR workflow monitoring
- **Multi-Workflow Support**: Parallel monitoring of multiple workflows
- **Real-Time Updates**: Immediate failure detection

## 2. Success Summary Generation

For successful workflows, provide:
- **Concise Digest**: Workflow name, trigger, commit info (≤100 words)
- **Key Metrics**: Duration, timestamp, commit SHA
- **Commit Details**: Author, message, changed files
- **Trend Analysis**: Performance comparison with previous runs
- **Quality Indicators**: Test coverage, code quality scores

## 3. Failure Analysis Framework

Comprehensive failure investigation:
- **Job and Step Identification**: Pinpoint exact failure location
- **Log Analysis**: Extract and summarize relevant error messages
- **Error Classification**: Build errors, test failures, deployment issues, timeout problems
- **Context Gathering**: Environment details, dependencies, configuration
- **Pattern Recognition**: Identify recurring failure patterns

## 4. Root Cause Determination

Intelligent failure diagnosis:
- **Causal Inference**: Technical narrative explaining failure origin
- **Dependency Analysis**: Identify broken dependencies or version conflicts
- **Configuration Issues**: Detect misconfiguration or missing settings
- **Code Quality Problems**: Syntax errors, type issues, lint failures
- **Infrastructure Issues**: Resource constraints, network problems, timeout issues
- **Historical Context**: Compare with previous successful runs

## 5. Remediation Planning

Strategic fix development:
- **Fix Strategy**: 1-3 step remediation plan
- **Risk Assessment**: Impact analysis of proposed fixes
- **Testing Strategy**: How to verify fixes before deployment
- **Rollback Plan**: Contingency if fix doesn't work
- **Documentation Needs**: What should be documented

## 6. Automated Fix Implementation

Intelligent code correction:
- **Targeted Changes**: Minimal, focused fixes preserving context
- **Multi-File Support**: Coordinate changes across related files
- **Validation**: Pre-commit verification of fixes
- **Commit Creation**: Descriptive commit messages
- **Push Coordination**: Automatic or manual push based on config

## 7. Issue Tracking Integration

Complete audit trail:
- **Issue Creation**: Automatic GitHub issue for each failure
- **Detailed Documentation**: Root cause, fix applied, verification steps
- **Link References**: Connect issues to commits and workflow runs
- **Auto-Close**: Close issues after successful remediation
- **Label Management**: Categorize issues by type and severity

## 8. Three Analysis Variants

Choose the right depth for your needs:

**Token-Efficient Variant:**
- Minimal token usage for cost optimization
- Fast execution for real-time monitoring
- Moderate accuracy for standard failures
- Ideal for: High-frequency monitoring, batch triage, serverless pipelines

**Balanced Variant (Recommended):**
- Optimal token/accuracy balance
- High-quality diagnosis and fixes
- Excellent repeatability
- Ideal for: Platform automation, internal developer tools, production systems

**Verbose Variant:**
- Maximum analysis depth
- Very high accuracy for complex issues
- Comprehensive documentation
- Ideal for: QA pipelines, critical infrastructure, complex debugging

## 9. Structured Reporting

Professional Markdown output:
- **Summary Section**: High-level overview of workflow status
- **Analysis Section**: Detailed failure investigation
- **Root Cause Section**: Clear explanation of why it failed
- **Plan Section**: Step-by-step remediation strategy
- **Fix Section**: Implementation details and changes made
- **Issue Tracking Section**: Links and references

## 10. Workflow Performance Metrics

Continuous improvement tracking:
- **Success Rate**: Percentage of successful runs
- **Mean Time to Recovery**: How quickly failures are fixed
- **Common Failure Patterns**: Most frequent issues
- **Performance Trends**: Workflow duration over time
- **Cost Analysis**: CI/CD resource consumption

## 11. Preventive Analysis

Proactive issue detection:
- **Flaky Test Detection**: Identify unreliable tests
- **Dependency Drift**: Detect outdated or vulnerable dependencies
- **Configuration Validation**: Catch misconfigurations before they fail
- **Resource Usage Trends**: Predict capacity issues
- **Performance Regression**: Detect slowdowns early

## 12. Multi-Repository Support

Enterprise-scale monitoring:
- **Batch Processing**: Monitor multiple repositories simultaneously
- **Priority Routing**: Critical repos get immediate attention
- **Aggregated Reporting**: Organization-wide CI/CD health
- **Shared Learning**: Apply fixes from one repo to similar issues elsewhere
- **Compliance Tracking**: Ensure all repos meet CI/CD standards

## 13. Integration Capabilities

Seamless workflow integration:
- **GitHub API**: Full integration with GitHub Actions
- **Notification Systems**: Slack, Teams, email, webhooks
- **Monitoring Platforms**: Datadog, New Relic, Grafana integration
- **Ticket Systems**: Jira, Linear, Asana integration
- **Documentation**: Auto-update runbooks and troubleshooting guides

## 14. Success Validation

Verification after fixes:
- **Workflow Re-Run**: Trigger workflow to verify fix
- **Test Validation**: Ensure all tests pass
- **Performance Check**: Confirm no performance regression
- **Documentation Update**: Record successful resolution
- **Knowledge Base**: Add to troubleshooting database

Tell me about your CI/CD monitoring needs and I'll set up an intelligent watchdog system with automated failure analysis and remediation!