Skip to main content

MLOps with Harness

MLOps (Machine Learning Operations) and DevOps (Development Operations) represent two crucial paradigms in modern software development, each addressing distinct but interconnected aspects of the software lifecycle. The Harness vision is to reduce toil and enable Developer, DevOps, and MLOps teams to deliver functionality to end users quickly, reliably, and efficiently.

This guide explains how to get started with MLOps in Harness.

MLOps and DevOps

DevOps emphasizes the integration and collaboration between development and operations teams, aiming to automate and streamline the delivery pipeline from code commit to deployment. It emphasizes practices such as continuous integration, continuous delivery (CI/CD), infrastructure as code (IaC), and monitoring, fostering a culture of rapid iteration, feedback, and accountability. DevOps ensures that software is delivered quickly, reliably, and with high quality, enabling organizations to respond swiftly to market demands and deliver value to customers efficiently

MLOps extends the principles of DevOps to the realm of machine learning, focusing on the operationalization and management of machine learning models throughout their lifecycle. MLOps integrates data engineering, model development, deployment, monitoring, and governance into a cohesive workflow, enabling organizations to efficiently develop, deploy, and manage machine learning applications at scale. MLOps addresses challenges unique to machine learning, such as data drift, model decay, versioning, and reproducibility, by applying DevOps practices like automation, version control, and collaboration.

By bridging the gap between data science and operations, MLOps ensures that machine learning models are deployed reliably, monitored effectively, and continually improved to meet evolving business needs.

To learn more about MLOps benefits and best practices, go to MLOps best practices

Accelerate velocity with MLOps and Harness

Developer velocity is about removing barriers and friction points by adopting the best tools and practices to enable your developers to achieve their full potential. Modern apps use a combination of traditional apps and ML/Generative AI models. Teams have to collaborate together from development to testing and through deployment. A modern toolset must help all these personas collaborate and deliver modern apps to end users quickly, reliably, efficiently, and securely.

A typical MLOps project lifecycle includes:

  1. Exploration (Problem framing, data collection, data exploration)
  2. Development (Feature engineering, model training)
  3. Model validation
  4. Model deployment
  5. Monitoring (Performance monitoring, drift monitoring, alerts/feedback loops)

DevOps complements MLOps. For example, here's a typical application flow combining MLOps and DevOps:

MLOps process diagram

Harness MLOps demo

Harness can help orchestrate and govern the practices of deploying secure ML code. Try it for yourself using the following demo. For a longer, step-by-step tutorial, go to Tutorial - End-to-end MLOps CI/CD pipeline with Harness and AWS.

To get started, you need to:


If you don't have a Harness account yet, you can create one for free at

Train model

In this team, a data scientist has built an ML model using scikit-learn in Jupyter Notebook. Data scientists can use any tools/framework, such as TensorFlow, PyTorch, scikit-learn,Keras , Jupyter Notebook, Pandas, NumPy, and Matplotlib, for data manipulation, analysis, and visualization.

Training can be done in Harness or through native integrations with any popular data science platforms, such as AWS SageMaker, Google Vertex AI, and more.

             - step:
type: Run
name: train
identifier: train
shell: Sh
command: |-
pip install --upgrade pip
pip install -r requirements.txt
pytest --nbval-lax credit_card_approval.ipynb --junitxml=report.xml

Deploy model (optional)

Deploying a machine learning model involves making it accessible for use in real-world applications, where it can make predictions or classifications based on new input data. This process varies from feature-engineering to exporting models to deployment platform to serving models to real-time monitoring.

Data scientists can use any tool/library for feature extraction and export models to common formats (such as pickle, joblib, TensorFlow, ONNX). Models can be deployed to cloud-based platforms, on-premises servers, or edge devices.

Harness can natively deploy to servers, containers, or serverless functions on Docker Hub, AWS, Google, Azure, and more.

This example shows how to use ECR as model registry and deploy a model to AWS Lambda. Note that you need your cloud provider's credentials to deploy and serve models.

YAML example: Deploy model to AWS Lambda
             - step:
type: BuildAndPushECR
name: Harness Training
identifier: BuildAndPushCCAprovalECR
connectorRef: someawsconnector
region: us-east-2
account: "12345"
imageName: ccapproval
- latest
dockerfile: Dockerfile_Training_Testing
- step:
type: Run
name: Integration Tests
identifier: runtest_ccapproval_deploy
connectorRef: someawsconnector
shell: Sh
command: pytest --nbval-lax credit_card_approval.ipynb --junitxml=report.xml
- onFailure:
- AllErrors
type: Ignore
- step:
type: BuildAndPushECR
name: Export Model
identifier: BuildAndPushCCApprovalECR
connectorRef: someawsconnector
region: us-east-2
account: "12345"
imageName: ccapproval-deploy
- latest
caching: false
dockerfile: Dockerfile_Inference_Lambda
stageStatus: Success
- step:
type: Run
name: Deploy Model
identifier: Deploy_Model_via_Lambda
shell: Sh
command: |-
aws lambda update-function-code --function-name lambda-python --image-uri
aws lambda wait function-updated-v2 --function-name lambda-python

aws lambda invoke --function-name lambda-python response.json
cat response.json
AWS_ACCESS_KEY_ID: <+secrets.getValue("aws_access_key_id")>
AWS_SECRET_ACCESS_KEY: <+secrets.getValue("aws_secret_access_key")>
AWS_SESSION_TOKEN: <+secrets.getValue("aws_session_token")>
AWS_REGION: us-east-2

Next steps

Monitoring and managing machine learning models in production involves ensuring their reliability, performance, and alignment with business goals over time.

This involves setting up KPIs, monitoring infrastructure, drift detection, monitoring model performance, alerts, A/B Testing, automated deployments, track model experiments, model freshness/retirement, and so on.

This video demonstrates how Harness can automate the process of training, deploying, and monitoring a machine learning model for a sample credit card loan application.

Harness integrates with common ML tools, including: