Manage Security Operations (part 2 of 3)

By Ashwin Venugopal -

 



To read part 1 please click here
To read part 3 please click here


Metrics and Logs

All data that Azure Monitor collects fits into one of the two fundamental type- metrics or logs.

What are metrics?

Metrics are numerical values that describes some aspect of a system at a particular time and are also collected at regular intervals as well as useful for alerting because they can be sampled frequently, and an alert can be fired quickly with relatively simple logic. 

Different ways of using metric data in Azure Monitor:

  • Analyze- You can use metric explorer to analyze collected metrics on a chart and compare metrics from different resources. 

  • Visualize- Pin a chart from metrics explorer to an Azure dashboard and create a workbook to combine with the multiple sets of the data in an interactive report. Export the results of a query to Grafana to leverage its dashboarding and combine with other data sources.

  • Alert- Configure a metric alert rule that sends a notification or takes automated action when the metric value crosses a threshold.

  • Automate- Use Autoscale to increase or decrease resources based on a metric value crossing a threshold.

  • Export- Route Metrics to Logs to analyze data in Azure Monitor Metrics together with data in Azure Monitor Logs and to store metric values for longer than 93 days. 

  • Retrieve- Access metric values from a command line using PowerShell cmdlets and from custom application using REST API. 

  • Archive- Archive the performance or health history of your resource for compliance, auditing, or offline reporting purposes.

What are Azure Monitor Logs?

Logs in Azure Monitor contains all the different kinds of data organized into records with different sets of properties for each type while also containing the numeric values like Azure Monitor Metrics but typically contains text data with detailed descriptions; they also vary from metrics in their structures and they are often not collected at regular intervals.

A common type of log entry is an event, which is collected sporadically. Metric data can also be stored in the logs to combine them with the other monitoring data for trending and other data analysis.

Different ways of using Logs in Azure Monitor:

  

  • Analyze- You can use Log Analytics in the Azure portal to write log queries and interactively analyze log data using the powerful Data Explorer analysis engine, whereas the Application Insights Analytic console in the Azure portal can be used to write log queries and interactively analyze log data from Application Insights.

  • Visualize- Pin query results rendered as tables or charts to an Azure dashboard and create a workbook to combine with the multiple sets of data in an interactive report. 

  • *Alert- Configure a log alert rule that sends a notification or takes automated action when the results of the  query match a particular result. 

  • Export- Build a workflow to retrieve log data and copy to an external location using Logic Apps.  

  • Retrieve- Access log query results from a command line using Azure CLI, PowerShell cmdlets, and from a custom application using REST API.

Log Analytics

Log Analytics is a part of Microsoft Azure's overall monitoring solution which helps you to monitor cloud and on-premises environments to maintain availability and performance and is the primary tool in the Azure portal for writing log queries as well as interactively analyzing their results.

In addition to interactively working with log queries and their results in Log Analytics, areas in Azure Monitor where you will use queries include the following:

  • Alert rules- They proactively identify issues from data in your workspace and each alert rule is based on a log search that is automatically run at the regular intervals. 

  • Dashboards- You can pin the results of any query into an Azure Dashboard which allows you to visualize log and metric data together as well as optionally share with the other Azure users.

  • Views- You can easily create the visualizations of the data to be included in user dashboards with the View Designer. Log queries provides the data used by tiles and visualization parts in each view.

  • Export- When you import the log data from Azure Monitor into Excel or Power BI, you create a log query to define the data to export.

  • PowerShell- You can run a PowerShell script from a command line or an Azure Automation runbook that uses Get-AzOperationlInsightsSearchResults to retrieve the log data from the Azure monitor. This cmdlet requires a query to determine the data to retrieve.

  • Azure Monitor Logs API- It allows any REST API client to retrieve log data from the workspace. The API request includes a query that is run against Azure Monitor to determine the data to retrieve.      

At the center of the Log Analytics is the Log Analytics Workspace, which is hosted in Azure. you can use the Log Analytics Workspaces menu to create a Log analytics Workspace using the Azure portal. Each workspace has its own data repository as well as configuration, and data sources and solutions are configured to store their data in a particular workspace. You will require a Log Analytics Workspace if you intend on collecting data from the following sources:

  • Azure resources in your subscription
  • On-premises computers monitored by System Center Operations Manager  
  • Device collections from Configuration Manager
  • Diagnostics and log data from Azure Storage




To read part 1 please click here
To read part 3 please click here


Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning...
Read more