Amazon Web Services Cloud

By Ashwin Venugopal -


 




AWS Management Console

You can access as well as manage AWS through the AWS Management Console, which is a simple and intuitive user interface while also using the AWS Console Mobile Application to view the resources on the go quickly.

AWS Command Line Interface 

It's a unified tool that's used to manage your AWS services and also helps you to control multiple AWS services as well as automate them through scripts with just one tool to download and configure.

Software Development Kits

They cam simplify using AWS services in your applications with the help of an Application Program Interface (API) tailored to your programming language and interface.

Analytics

Amazon Athena 

It's an interactive query service making it easy to analyze data in Amazon S3 with the help of standard SQL and does not require any infrastructure to manage as Athena is serverless and you only pay for the queries you run.

Athena is easy to use and does not require any kind of complex extract, transform, and load (ETL) jobs in order to prepare your data for analysis which makes it easy for anyone with SQL skills to quickly analyze large-scale datasets. 

It's an out-of-the-box integrated with AWS Glue Data Catalog, which helps you to create a unified metadata repository across various services, crawl data sources to discover schemas and populate your Catalog with new as well as modified table and partition definitions, and maintain schema versioning.

Amazon CloudSearch 

It's a managed service which is simple as well as cost-effective to set up, manage, and scale a search solution for your website or application. It is capable of supporting 34 languages as well as popular search features like highlighting, autocomplete, and geospatial search.

Amazon Elasticsearch Service

It readily helps to deploy, secure, operate as well as scale Elasticsearch to find, analyze, and visualize data in real-time. It offers easy-to-use APIs and real-time analytics capabilities to power use-cases like log analytics, full-text search, application monitoring, and clickstream analytics, along with an enterprise-grade availability, scalability, and security. It can also readily integrates with the other AWS services like Amazon Virtual Private Cloud (Amazon VPC), AWS Key Management Service (AWS KMS), Amazon Kinesis Data Firehose, AWS Lambda, AWS Identity & Access Management (IAM), Amazon Cognito, and Amazon CloudWatch, so that you can go from raw data to actionable insights quickly.

Amazon EMR

It's an industry-leading cloud big data platform for processing vast amounts of data with the help of open source tools like Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto. It allows you to run petabyte-scale analysis at less than half of the cost of traditional on-premises solutions and over three times faster than standard Apache Spark while also helping you to run workloads on Amazon EC2 instances, on Amazon Elastic Kubernetes Services (EKS) clusters, or on-premises using EMR on AWS Outposts. 

Amazon FinSpace

It's a data management and analytics service purpose-built for the Financial Services Industry (FSI). It can effectively reduce the time spent on finding as well as preparing petabytes of financial data to be ready for analysis from months to minutes. Financial services organizations analyzes data from internal data stores like portfolio, actuarial, and risk management systems along with the petabytes of data from third-party data feeds, like historical securities prices from stock exchanges. 

FinSpace can remove the heavy lifting of building and maintaining a data management system for financial analytics and you can easily share as well as discover data across your organization according to your compliance requirements. It also contains a library with 100+ functions, such as time bars and Bollinger bands, for you to prepare data for analysis.

Amazon Kinesis

It helps you to easily collect, process, as well as analyze real-time, streaming data so you can get timely insights and react quickly to new information. Hence you can easily ingest real-time data like video, audio, application logs, website clickstreams, and IoT telemetry data for machine learning, analytics, and the other applications. Amazon Kinesis allows you to process as well as analyze data as it arrives and respond instantly instead of waiting until all your data is collected before the processing can even begin. It currently provides four services- Kinesis Data Firehose, Kinesis Data Analytics, Kinesis Data Streams, and Kinesis Video Streams. 

Amazon Redshift

It's one of the most widely used cloud data warehouse which is fast, simple, and cost-effective for analyzing all your data with the help of standard SQL as well as your existing Business Intelligence (BI) tools. Here you can run complex analytic queries against terabytes to petabytes of structured and semi-structured data using sophisticated query optimization, columnar storage on high-performance storage, and massively parallel query execution.

Amazon QuickSight

It's a fast, cloud-powered Business Intelligence (BI) service that helps you to easily deliver insights to everyone in your organization while also allowing you to create as well as publish interactive dashboards that can be accessed from browsers or mobile devices. QuickSight can easily scale to millions of users without any need of installing a software, deploying servers, or managing an infrastructure.

AWS Data Exchange

It helps you to easily search, subscribe, as well as use third-party data in the cloud and after subscribing to a data product, you can use AWS Data Exchange API to directly load data into Amazon S3 and then analyze it using various AWS analytics as well as machine learning services. Data providers can easily reach to millions of AWS customers migrating to the cloud by removing the requirement to build as well as maintain infrastructure for data storage, delivery, billing, and entitling, with the help of AWS Data Exchange. 

AWS Data Pipeline

It's a type of web service that can reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specific intervals. It helps you to easily create complex data processing workloads that are fault tolerant, repeatable, and highly available without even have to worry about ensuring resource availability, managing inter-task dependencies, retrying transient failures or timeouts in individual tasks, or creating a failure notification system. AWS Data Pipeline also helps you to move and process the data that was previously locked up in on-premises data silos.

AWS Glue

It's a fully managed Extract, Transform, and Load (ETL) service which is easier for the customers to prepare as well as load their data for analytics. You just have to point AWS Glue to the data stored on AWS so that it can discover your data to store the associated metadata in the AWS Glue Data Catalog. Once cataloged, your data can be instantly searchable, queryable, and available for ETL.

AWS Lake Formation

This service allows you to easily set up a secure data lake in days which is a centralized, curated, and secured repository that can store all your data, both in its original form and prepared for analysis. It enables you to break down data silos and combine various analytics to gain insights and guide better business decisions.

It is as simple to create a data Lake Formation as defining where your data is located as well as what data access and security policies you want to apply. Lake Formation collects as well as catalogs data from databases and object storage, moves them to into your new Amazon S3 data lake, cleans and classifies it with the help of machine learning algorithms, and secure access to your sensitive data. Users can leverage these data sets according to their choice of analytics and machine learning services, such as Amazon EMR for Apache Spark, Amazon Redshift, Amazon Athena, SageMaker, and Amazon QuickSight. 

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

It's a fully managed service that allows you to easily build as well as run applications that use Apache Kafka to process streaming data which is an open-source platform for building real-time streaming data pipelines and applications. You can use Apache Kafka APIs with the help of Amazon MSK, to populate data lakes, stream changes to and from databases, and power machine learning and analytics applications.

You can create highly available Apache Kafka clusters on Amazon MSK console with settings as well as configuration based on Apache Kafka's deployment best practices. Amazon MSK can automatically provision as well as run your Apache Kafka clusters while also continuously monitoring cluster health and automatically replacing unhealthy nodes with no downtime to your application; besides it can also protect your Apache Kafka cluster by encrypting the data at rest.     















































  

  























Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning...
Read more