Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

MLOps (Machine Learning Operations)

Overview

MLOps is a set of practices that combines Machine Learning, DevOps, and Data Engineering to deploy and maintain ML systems in production reliably and efficiently. It bridges the gap between model development and production deployment.

Why MLOps?

graph LR
    A[Data Science] -->|Without MLOps| B[Manual Deployment]
    B --> C[Inconsistent Results]
    C --> D[Production Failures]
    
    A -->|With MLOps| E[Automated Pipeline]
    E --> F[Reproducible Models]
    F --> G[Reliable Production]

The gap: ~85% of ML projects never make it to production. MLOps addresses this by providing the engineering practices needed to deploy, monitor, and maintain ML systems reliably.

MLOps vs DevOps

AspectDevOpsMLOps
ArtifactCodeCode + Data + Model
TestingUnit/integration testsData validation + model evaluation
VersioningGitGit + DVC (data) + MLflow (models)
DeploymentBuild → DeployTrain → Evaluate → Register → Deploy
MonitoringCPU, memory, latency+ Data drift, model performance
RollbackCode rollbackModel rollback + data rollback
IterationFeature releasesModel retraining cycles

MLOps Maturity Levels

graph TB
    L0["Level 0: Manual Process<br/>Manual Jupyter notebook → manual deploy<br/>No automation, no monitoring"]
    L1["Level 1: ML Pipeline Automation<br/>Automated training pipeline<br/>Continuous training (CT)"]
    L2["Level 2: CI/CD for ML<br/>Automated testing & deployment<br/>Full MLOps maturity"]
    L0 --> L1 --> L2
LevelAutomationReproducibilityMonitoringTypical Team
0ManualLowNoneData scientists only
1Training pipelineMediumBasicDS + ML engineers
2Full CI/CD/CTHighComprehensiveDS + MLE + Platform

ML Lifecycle

graph TD
    A[1. Problem Definition] --> B[2. Data Collection & Labeling]
    B --> C[3. Data Validation & Exploration]
    C --> D[4. Feature Engineering]
    D --> E[5. Model Training & Tuning]
    E --> F[6. Model Evaluation]
    F --> G{Meets Criteria?}
    G -->|No| D
    G -->|Yes| H[7. Model Registration]
    H --> I[8. Model Deployment]
    I --> J[9. Monitoring & Alerting]
    J --> K{Drift Detected?}
    K -->|Yes| B
    K -->|No| J

Core Components

1. Data Management

ComponentPurposeTools
Data VersioningTrack changes to datasetsDVC, LakeFS, Delta Lake
Data ValidationCheck schema, distributions, qualityGreat Expectations, Pandera
Data PipelineAutomate data ingestion/transformationAirflow, Prefect, Dagster
Feature StoreCentralized feature managementFeast, Tecton, Hopsworks

2. Model Training

ComponentPurposeTools
Experiment TrackingLog params, metrics, artifactsMLflow, W&B, Neptune
Hyperparameter TuningSearch optimal configOptuna, Ray Tune, SigOpt
Training OrchestrationDistributed trainingHorovod, DeepSpeed, PyTorch DDP
Model RegistryVersion and stage modelsMLflow Registry, Vertex AI

3. Model Deployment

PatternDescriptionWhen to Use
BatchRun predictions on a scheduleRecommendations, reports
Real-timeServe predictions via APIFraud detection, search
StreamingProcess data in real-timeEvent-driven systems
EdgeRun on deviceMobile, IoT

4. Monitoring

CategoryWhat to TrackTools
Data DriftInput distribution changesEvidently, Whylogs
Model PerformanceAccuracy, latency, throughputPrometheus, Grafana
System HealthCPU, memory, GPU utilizationDatadog, CloudWatch
Business MetricsRevenue, conversion, engagementCustom dashboards

CI/CD/CT for ML

graph LR
    A[Code Change] --> B[CI: Build & Test]
    B --> C[CT: Retrain Model]
    C --> D[Evaluate Model]
    D --> E{Quality Gate?}
    E -->|Pass| F[CD: Deploy Model]
    E -->|Fail| G[Alert & Investigate]
    F --> H[Monitor in Production]
PipelineTriggerWhat It Does
CICode commitRun tests, lint, build containers
CTData change or scheduleRetrain model on new data
CDModel passes quality gateDeploy to staging → production
CategoryToolStrengths
Experiment TrackingMLflowOpen-source, widely adopted
Weights & BiasesBeautiful UI, team collaboration
OrchestrationAirflowMature, large community
PrefectModern Python, easy to use
DagsterAsset-centric, type-safe
Feature StoreFeastOpen-source, lightweight
TectonManaged, real-time features
Model ServingTritonMulti-framework, GPU optimized
vLLMLLM-specific, fast inference
BentoMLEasy packaging and deployment
MonitoringEvidentlyOpen-source, drift detection
ArizeManaged, production monitoring
PlatformKubeflowKubernetes-native ML platform
Vertex AIGCP managed ML platform
SageMakerAWS managed ML platform

Key Design Decisions

Model Registry Strategy

graph LR
    A[Experiment] -->|Promote| B[Staging]
    B -->|Validate| C[Production]
    C -->|Supersede| D[Archived]
StageDescriptionAccess
None/DevelopmentExperiment modelsData scientists
StagingPassed offline eval, A/B testStaging environment
ProductionServing live trafficProduction
ArchivedReplaced by newer modelRead-only

Retraining Strategy

TriggerDescriptionUse Case
ScheduledRetrain every N hours/daysStable data distributions
Performance-basedRetrain when metrics dropDynamic environments
Data-basedRetrain when new labeled data arrivesActive learning setups
Drift-basedRetrain when drift detectedNon-stationary data

Common MLOps Anti-Patterns

Anti-PatternProblemSolution
Notebook-to-productionUntested, unversioned codePackage as modules, add tests
Training-serving skewDifferent code in training/servingShare feature pipelines
Data leakageFuture data in trainingPoint-in-time joins
No monitoringSilent model degradationImplement drift detection
Manual deploymentHuman errors, slowAutomate with CI/CD
No versioningCan’t reproduce resultsVersion code, data, models

Interview Questions

  1. What is MLOps and why is it important? MLOps applies DevOps principles to ML systems. It’s important because ML systems have unique challenges: data dependencies, model drift, non-deterministic training, and complex reproducibility requirements. Without MLOps, most ML projects fail in production.

  2. How does MLOps differ from DevOps? MLOps manages three artifacts (code, data, model) instead of just code. It requires data versioning, experiment tracking, model evaluation gates, drift monitoring, and retraining pipelines on top of standard CI/CD.

  3. What is training-serving skew and how do you prevent it? When the feature computation logic differs between training and serving, causing models to see different data in production. Prevention: (1) Share feature computation code. (2) Use a feature store. (3) Validate features in both pipelines.

  4. How do you version ML models? Track model artifacts (weights, config), training code version (git commit), data version (DVC hash), hyperparameters, and evaluation metrics. Store in a model registry with metadata linking all these together.

  5. Design an ML system that retrains automatically. Scheduled or drift-triggered retraining → automated training pipeline → evaluation against production model → quality gate (must beat baseline) → canary deployment → monitoring → rollback if degraded.

Common Mistakes

  • Treating ML projects like software projects without considering data drift
  • Not versioning data alongside code and models
  • Skipping monitoring in production
  • Manual deployment processes leading to human errors
  • No quality gates before deployment

Summary

MLOps is essential for moving ML from research to production. It ensures reproducibility, scalability, and reliability of ML systems through automation, monitoring, and best practices borrowed from DevOps and Data Engineering.

References

  • Sculley, D. et al. (2015). “Hidden Technical Debt in Machine Learning Systems” — Classic paper on ML system complexity
  • Google. (2020). MLOps: Continuous delivery and automation pipelines in machine learning — MLOps whitepaper
  • Kreuzberger, D. et al. (2022). “Machine Learning Operations (MLOps): Overview, Definition, and Architecture” — Comprehensive survey
  • Huyen, C. (2022). Designing Machine Learning Systems (O’Reilly) — Best practical MLOps book
  • ml-ops.org — MLOps community resources

Cross-References