Connect Data to Azure Sentinel Using Data Connectors

By Ashwin Venugopal -

 



Ingest Log Data with Data Connectors

You should connect your data sources with Azure Sentinel Connectors to collect the log data whose page displays a growing list of the connectors provided by Azure Sentinel, after selecting the Open connector page (the detailed connector page has a left and right blade). 

The information about the connector, the connector's status, and the last time a log was received if connected is provided by the left blade. Whereas the right tab has two tabs- Instructions and Next Steps. The instructions tab can be different based on the connector having Prerequisites and Configuration which can be followed to connect to the data source. The Next Steps tab offers a quick reference to workbooks, query samples, and analytical templates. Data connectors can only be disconnected/deactivated, not deleted.

Note- The connector does not install Workbooks and Analytical Templates as they are already available in the Sentinel environment for out of the box connectors.

Understand Data Connectors Providers

Microsoft 365 Defender

The alerts and data that has already been normalized as well as used in the Microsoft 365 Defender portal, are provided by the Microsoft 365 Defender and related data connectors which includes products like:

  • Microsoft Defender for Endpoint
  • Microsoft Defender for Identity
  • Microsoft Defender for Office 365
  • Microsoft Cloud App Security   

Microsoft/Azure Services

The connectors for Microsoft and Azure-related services includes (but are not limited to):
  • Azure Active Directory- audit logs and sign-in logs
  • Azure Activity
  • Azure AD Identity Protection
  • Azure DDOS Protection
  • Azure Defender to IoT
  • Azure Information Protection
  • Azure Firewall
  • Azure Security Center- alerts from Azure Defender solutions
  • Azure Web Application Firewall (WAF)
  • Cloud App Security 
  • Domain name server
  • Office 365
  • Windows firewall
  • Security events

Vendor Connectors

Azure Sentinel provides an ever-growing list of vendor-specific data connectors that primarily use the CEF and Syslog connector. 

Custom Connectors Using the Log Analytics API

You can use the Log Analytics Data Collector API to send log data to the Azure Sentinel Log Analytics workspace.

Logstash plugin

You are able to send any log you want through Logstash directly to your Log Analytics workspace in Azure Sentinel with the help of Azure Sentinel's output plugin for the Logstash data collection engine. The logs are written to a custom table that can be defined using the output plugin.

CEF & Syslog Connector

The generic Common Event Format (CEF) or Syslog connector can be used if there is no vendor-provided connector which is known as an open event logging protocol that is common to Linux. Messages will be send by the Applications that may be stored on the local machine or delivered to a Syslog connector. Whereas CEF is an industry-standard format on top of Syslog messages, extensively used by many security vendors to allow event interoperability among different platforms.

Syslog vs. Common Event Format

CEF is always considered as a superior choice because the log data is parsed into predefined fields in the CommonSecurityLog table while Syslog always provides header fields, but its raw log message is stored in a field named Syslog Message in the Syslog table. You have to write a parser to extract the specific fields for the Syslog data to be queried. 

Connector architecture options

To connect the CEF or Syslog Collector to Azure Sentinel, the agent must deploy on a dedicated Azure VM or an on-premises system to support the appliance's communication with Azure Sentinel, automatically or manually. Automatic deployment is available only if your dedicated machine is a VM in Azure. Alternatively, you can  always manually deploy the agent on an existing Azure VM, on a VM in another cloud, or an on-premises machine.  

View Connected Hosts

The Data Connector page clearly shows the connectors that are connected and the amount of Windows as well as Linux hosts connected with the Log Analytics agent is available in the Log Analytics worksapce. To see your connected hosts you can do the following steps:

  • Select Settings
  • Workspace Setting (this will transfer you to Log Analytics)
  • In Log Analytics Settings area select Agents Management
  • There are two tabs to view- one for Windows other for Linux     










Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning...
Read more