Exploring the Azure Machine Learning service

By Ashwin Venugopal -

 







Analyzing the Deployed Services

There are three services that are deployed along with the main mldemows workspace, i.e., Storage account, Key vault, and Application Insights. Although, an Azure container registry will be required later on but it doesn't need to there during initial deployment of the workspace. Now, let's have a look at them:

The Storage Account for an ML Workspace

The storage account is generally known as the default storage account and it's the main datastore for the workspace. This one is vital for the operation of the service as it can store experiment runs, models, snapshots, and source files (such as Jupyter notebooks). 

Azure Key Vault for an ML Workspace

It is a cloud-managed service that can store secrets like passwords, API keys, certificates, and cryptographic keys either in a software vault or in a managed Hardware Security Module (HSM). You can easily access the key vault via a so-called managed identity, which gives the workspace (the app) itself an identity to assign rights to. 

Application Insights for an ML Workspace

It is a module of Azure monitor used to monitor infrastructure and applications, that can store as well as surface infrastructure metrics such as CPU usage and log files of applications. The Azure ML workspace uses Application Insights to store compute infrastructure logs, ML script logs, and defined metrics of the ML model runs and is therefore required for the operation of the workspace.

Azure Container Registry (ACR) for an ML Workspace

It is a service based on the Docker Registry to store as well as manage Docker container images and artifacts. It is required while running training on or deploying models to a compute that's not our local machine for which a container is packed and registered to ACR, which can then be tracked and utilized in ML scripts or by deploying pipelines. 












































Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning...
Read more