mlops
/mlops
Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.
- Author
- buldee ↗
- Repository
- ai-craftsman-superpowers ↗
- Licence
- Apache-2.0
- Last repository change
- 24.10.2026
Assessment and security
- Files in this package
- 2
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 0
No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.
Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.
What it does and when to use it
Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.
Original author description
Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.
How it works
- Outcome Contract
- MLOps Maturity Levels
- Six Pillars of MLOps
- Audit Template
- Common Anti-Patterns
- Tool Recommendations
- Process
SKILL.md preview
--- model: opus description: Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices. effort: xhigh --- # /craftsman:mlops - ML Production Readiness Audit ## Outcome Contract - **Outcome**: a production-readiness verdict on the ML system, with gaps named and prioritized. - **Done when**: each MLOps dimension is assessed against what exists in the repository, not assumed, and every gap has a concrete remediation. - **Evidence**: the files and pipelines inspected, and the assessment per dimension. Audit ML projects against production best practices. ## MLOps Maturity Levels ``` Level 0: Manual - Notebooks, manual deployment Level 1: ML Pipeline - Automated training, manual deploy Level 2: CI/CD for ML - Automated training + deployment Level 3: Full MLOps - Automated retraining, monitoring, feedback ``` ## Six Pillars of MLOps ### 1. Data Management ```markdown ## Data Checklist - [ ] Data versioning (DVC, Delta Lake) - [ ] Data validation (Great Expectations) - [ ] Feature store (Feast, Tecton) - [ ] Data lineage tracking - [ ] Privacy compliance (PII handling) ``` ### 2. Model Development ```markdown ## Development Checklist - [ ] Experiment tracking (MLflow, W&B) - [ ] Reproducible training (seeds, configs) - [ ] Model registry - [ ] Baseline comparisons - [ ] Hyperparameter tuning automation ``` ### 3. Model Deployment ```markdown ## Deployment Checklist - [ ] Model serving (TorchServe, TF Serving, FastAPI) - [ ] Containerization (Docker) - [ ] Orchestration (K8s, ECS) - [ ] A/B testing capability - [ ] Canary deployments - [ ] Rollback mechanism ``` ### 4. Monitoring ```markdown ## Monitoring Checklist - [ ] Model performance metrics - [ ] Data drift detection - [ ] Concept drift detection - [ ] Prediction latency - [ ] Resource utilization - [ ] Alerting thresholds ``` ### 5. CI/CD for ML ```markdown ## CI/CD Checklist - [ ] Automated testing (unit, integration) - [ ] Model validation gates - [ ] Automated retraining triggers - [ ] Shadow deployments - [ ] Feature flag integration ``` ### 6. Governance ```markdown ## Governance Checklist - [ ] Model documentation - [ ] Bias/fairness audits - [ ] Explainability (SHAP, LIME) - [ ] Audit trail - [ ] Access control ``` ## Audit Template ```markdown # MLOps Audit: [Project Name] ## Summary - **Maturity Level:** [0-3] - **Critical Issues:** [Count] - **Recommendations:** [Count] ## Pillar Assessment ### 1. Data Management [Score: X/10] | Aspect | Status | Notes | |--------|--------|-------| | Versioning | ✅/⚠️/❌ | | | Validation | ✅/⚠️/❌ | | | Feature Store | ✅/⚠️/❌ | | **Issues:** - [Issue 1] - [Issue 2] **Recommendations:** - [Recommendation 1] ### 2. Model Development [Score: X/10] [Same structure] ### 3. Deployment [Score: X/10] [Same structure] ### 4. Monitoring [Score: X/10] [Same structure] ### 5. CI/CD [Score: X/10] [Same structure] ### 6. Governance [Score: X/10] [Same structure] ## Risk Assessment | Risk | Probability | Impact | Mitigation | |------|-------------|--------|------------| | Model drift undetected | High | High | Implement monitoring | | Data pipeline failure | Medium | High | Add validation | ## Roadmap ### Quick Wins (1-2 weeks) - [ ] [Action 1] - [ ] [Action 2] ### Medium Term (1-3 months) - [ ] [Action 1] - [ ] [Action 2] ### Long Term (3-6 months) - [ ] [Action 1] - [ ] [Action 2] ``` ## Common Anti-Patterns | Anti-Pattern | Problem | Solution | |--------------|---------|----------| | Notebook-to-prod | No reproducibility | Pipeline extraction | | No versioning | Can't reproduce | DVC + MLflow | | Train/serve skew | Different preprocessing | Feature store | | No monitoring | Silent failures | Drift detection | | Manual deployment | Slow, error-prone | CI/CD pipeline | ## Tool Recommendations | Category | Tools | |----------|-------| | Experiment Tracking | MLflow, Weights & Biases, Neptune | | Data Versioning | DVC, Delta Lake, LakeFS | | Feature Store | Feast, Tecton, Hopsworks | | Model Serving | BentoML, Ray Serve, Seldon | | Monitoring | Evidently, WhyLabs, Arize | | Orchestration | Kubeflow, Airflow, Prefect | ## Process 1. **Identify current maturity level** 2. **Audit each pillar** 3. **Identify gaps and risks** 4. **Prioritize improvements** 5. **Create actionable roadmap**
Files in this package (2)
- .craftsman-pack-source 1 KB
- SKILL.md 4 KB
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · buldee/ai-craftsman-superpowers | 84 | Apache-2.0 |
| aws-samples/sample-claude-code-plugins-for-startups | 100 | MIT |
Database snapshot: 13.11.2026 · package version 9b3f3bea1538 (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 100mlops-engineer
ML infrastructure automation and production ML lifecycle management. Use when building ML pipelines, setting up experiment tracking, implementing CI/CD for models, or managing model deployments. - 100ml-pipeline-workflow
Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod... - 100ml-pipeline-workflow
Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod... - 100mlops-engineer
Provides MLOps patterns for ML CI/CD pipelines, model registries, monitoring, and data drift detection. Use when setting up ML infrastructure or when the user mentions MLOps, model deployment, ML pipeline, or model monitoring. - 100mlops-engineer
Senior MLOps Engineer with 8+ years ML systems experience. Use for model serving & inference infrastructure, AI/ML pipelines, training-data pipelines, model deployment & monitoring, and AI cost optimization at the infrastructure level. For app-level LLM product features (RAG,… - 100code-audit
Use when reviewing code or a codebase for DevOps readiness, security issues, and missing best practices