Skip to content
Skills Library
DevOps & cloud · Review & audit

mlops

/mlops

Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.

Licence
Apache-2.0
Last repository change
24.10.2026

Assessment and security

84/ 100Class: good
Format9 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
0
Deprecation

No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.

Original author description

Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.

How it works

  1. Outcome Contract
  2. MLOps Maturity Levels
  3. Six Pillars of MLOps
  4. Audit Template
  5. Common Anti-Patterns
  6. Tool Recommendations
  7. Process
Original SKILL.md headings · 189 lines, 688 words
SKILL.md preview
---
model: opus
description: Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.
effort: xhigh
---

# /craftsman:mlops - ML Production Readiness Audit

## Outcome Contract

- **Outcome**: a production-readiness verdict on the ML system, with gaps named and prioritized.
- **Done when**: each MLOps dimension is assessed against what exists in the repository, not assumed, and every gap has a concrete remediation.
- **Evidence**: the files and pipelines inspected, and the assessment per dimension.

Audit ML projects against production best practices.

## MLOps Maturity Levels

```
Level 0: Manual           - Notebooks, manual deployment
Level 1: ML Pipeline      - Automated training, manual deploy
Level 2: CI/CD for ML     - Automated training + deployment
Level 3: Full MLOps       - Automated retraining, monitoring, feedback
```

## Six Pillars of MLOps

### 1. Data Management

```markdown
## Data Checklist

- [ ] Data versioning (DVC, Delta Lake)
- [ ] Data validation (Great Expectations)
- [ ] Feature store (Feast, Tecton)
- [ ] Data lineage tracking
- [ ] Privacy compliance (PII handling)
```

### 2. Model Development

```markdown
## Development Checklist

- [ ] Experiment tracking (MLflow, W&B)
- [ ] Reproducible training (seeds, configs)
- [ ] Model registry
- [ ] Baseline comparisons
- [ ] Hyperparameter tuning automation
```

### 3. Model Deployment

```markdown
## Deployment Checklist

- [ ] Model serving (TorchServe, TF Serving, FastAPI)
- [ ] Containerization (Docker)
- [ ] Orchestration (K8s, ECS)
- [ ] A/B testing capability
- [ ] Canary deployments
- [ ] Rollback mechanism
```

### 4. Monitoring

```markdown
## Monitoring Checklist

- [ ] Model performance metrics
- [ ] Data drift detection
- [ ] Concept drift detection
- [ ] Prediction latency
- [ ] Resource utilization
- [ ] Alerting thresholds
```

### 5. CI/CD for ML

```markdown
## CI/CD Checklist

- [ ] Automated testing (unit, integration)
- [ ] Model validation gates
- [ ] Automated retraining triggers
- [ ] Shadow deployments
- [ ] Feature flag integration
```

### 6. Governance

```markdown
## Governance Checklist

- [ ] Model documentation
- [ ] Bias/fairness audits
- [ ] Explainability (SHAP, LIME)
- [ ] Audit trail
- [ ] Access control
```

## Audit Template

```markdown
# MLOps Audit: [Project Name]

## Summary
- **Maturity Level:** [0-3]
- **Critical Issues:** [Count]
- **Recommendations:** [Count]

## Pillar Assessment

### 1. Data Management [Score: X/10]

| Aspect | Status | Notes |
|--------|--------|-------|
| Versioning | ✅/⚠️/❌ | |
| Validation | ✅/⚠️/❌ | |
| Feature Store | ✅/⚠️/❌ | |

**Issues:**
- [Issue 1]
- [Issue 2]

**Recommendations:**
- [Recommendation 1]

### 2. Model Development [Score: X/10]
[Same structure]

### 3. Deployment [Score: X/10]
[Same structure]

### 4. Monitoring [Score: X/10]
[Same structure]

### 5. CI/CD [Score: X/10]
[Same structure]

### 6. Governance [Score: X/10]
[Same structure]

## Risk Assessment

| Risk | Probability | Impact | Mitigation |
|------|-------------|--------|------------|
| Model drift undetected | High | High | Implement monitoring |
| Data pipeline failure | Medium | High | Add validation |

## Roadmap

### Quick Wins (1-2 weeks)
- [ ] [Action 1]
- [ ] [Action 2]

### Medium Term (1-3 months)
- [ ] [Action 1]
- [ ] [Action 2]

### Long Term (3-6 months)
- [ ] [Action 1]
- [ ] [Action 2]
```

## Common Anti-Patterns

| Anti-Pattern | Problem | Solution |
|--------------|---------|----------|
| Notebook-to-prod | No reproducibility | Pipeline extraction |
| No versioning | Can't reproduce | DVC + MLflow |
| Train/serve skew | Different preprocessing | Feature store |
| No monitoring | Silent failures | Drift detection |
| Manual deployment | Slow, error-prone | CI/CD pipeline |

## Tool Recommendations

| Category | Tools |
|----------|-------|
| Experiment Tracking | MLflow, Weights & Biases, Neptune |
| Data Versioning | DVC, Delta Lake, LakeFS |
| Feature Store | Feast, Tecton, Hopsworks |
| Model Serving | BentoML, Ray Serve, Seldon |
| Monitoring | Evidently, WhyLabs, Arize |
| Orchestration | Kubeflow, Airflow, Prefect |

## Process

1. **Identify current maturity level**
2. **Audit each pillar**
3. **Identify gaps and risks**
4. **Prioritize improvements**
5. **Create actionable roadmap**
Files in this package (2)
  • .craftsman-pack-source 1 KB
  • SKILL.md 4 KB

Versions

RepositoryScoreLicence
This page · buldee/ai-craftsman-superpowers84Apache-2.0
aws-samples/sample-claude-code-plugins-for-startups100MIT

Database snapshot: 13.11.2026 · package version 9b3f3bea1538 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 100mlops-engineer
    ML infrastructure automation and production ML lifecycle management. Use when building ML pipelines, setting up experiment tracking, implementing CI/CD for models, or managing model deployments.
  • 100ml-pipeline-workflow
    Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod...
  • 100ml-pipeline-workflow
    Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod...
  • 100mlops-engineer
    Provides MLOps patterns for ML CI/CD pipelines, model registries, monitoring, and data drift detection. Use when setting up ML infrastructure or when the user mentions MLOps, model deployment, ML pipeline, or model monitoring.
  • 100mlops-engineer
    Senior MLOps Engineer with 8+ years ML systems experience. Use for model serving & inference infrastructure, AI/ML pipelines, training-data pipelines, model deployment & monitoring, and AI cost optimization at the infrastructure level. For app-level LLM product features (RAG,…
  • 100code-audit
    Use when reviewing code or a codebase for DevOps readiness, security issues, and missing best practices

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.