Understanding End-to-End Machine Learning Process (Part 5 of 5)

By Ashwin Venugopal -

 





To read part 1, please click here
To read part 2, please click here
To read part 3, please click here
To read part 4, please click here







Deploying Models

This step is also known as inferencing or scoring a model. You can clearly see the deployment and operation of an ML pipeline when the model is tested on live data in production, which is generally done to get deeper insight and data to improve the model continuously. So, if you collect the model's performance over time, it will guarantee the model's improvement in it. There are two main architectures for ML-scoring pipelines:
  • Batch scoring using pipelines- It's an offline process in which you can evaluate an ML model against a batch of data. The result of this scoring is usually not time-critical, and the to be scored data is also larger than the model.

  • Real-time scoring using a container-based web service endpoint- It's a technique in which you can score single data inputs (common in stream processing for scoring single events in real time), which makes this task highly time-critical, and the execution is blocked until the resulting score is computed. 

Developing & Operating Enterprise-Grade ML Solutions

In order to operationalize ML projects, we have to use automated pipelines and development-operations (DevOps) methodologies like Continuous Integration (CI) and Continuous Delivery/Continuous Deployment (CD), also known as MLOps. Hence, the two different automated pipelines are:
  • Training pipeline- This one contains loading datasets, transformation, model training, and registering final models; and could be triggered via changes in the datasets or possible detected data drift in a deployed model.

  • Deployment pipeline- In this one, we have loading of models from the registry, creating and deploying Docker images, creating and deploying operational scripts, and the final deployment of the model to the target; and could be triggered by any new versions of an ML model.

Now, after having these pipelines, we can look upon Azure DevOps (and other tooling) to create a life cycle for our ML projects containing the following parts:

  • Creating or retraining a model- Here, training pipelines are used to create or retrain our model while version-controlling the pipelines and the code.

  • Deploying the model & creating scoring files & dependencies- In this one, we can use a deployment pipeline to deploy a specific mode version while version-controlling the pipeline and the code.

  • Creating an audit trail- Through CI/CD pipelines and version control, we can create an audit trail for all assets while ensuring integrity and compliance.

  • Monitoring model in production- Here, we can monitor the performance and possible data drift, which might trigger retraining of the model automatically. 








To read part 1, please click here
To read part 2, please click here
To read part 3, please click here
To read part 4, please click here
















































Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different groups of computers, re
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning of alert rule parameter
Read more

Design Planning (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Complex Organizational Structures SIEM can be an expensive security control, hence, it is common multiple businesses to contribute to the overall expense. Various organizational units may require a level of access to specific dashboards or sets of data. Microsoft Sentinel offers the ability to assign table-level permissions and limit the level of access to the minimum required to perform requisite job functions.  Role-based Access Control (RBAC) Requirements Microsoft Sentinel offers an extensive list of Azure built-in roles that can be used to provide granular access according to the job requirements and permitted level of access. Some of them are various Microsoft Sentinel dedicated roles: Microsoft Sentinel Contributor- Can perform all engineering-related configuration, such as creating alert rules, configuring data connectors, and additional similar tasks.  Microsoft Sentinel Reader- Can query the log data stor
Read more