Container Security (part 3 of 3)

By Ashwin Venugopal -

 



To read part 1 of 3 please click here
To read part 2 of 3 please click here

Azure Kubernetes Service (AKS)

Kubernetes is a rapidly evolving platform that manages container-based applications and their associated networking as well as storage components while supporting both the stateless and stateful applications as teams progress through the adoption of microservices-based applications. As an open platform, it also allows you to build your applications with your preferred programming language, OS, libraries, or messaging bus. 
AKS provides a managed kubernetes service that reduces the complexity for deployment and core management tasks, including coordinating upgrades. AKS is built on top of the open-source Azure Kubernetes Service Engine (aks-engine) and you only pay for the AKS nodes that run your applications. 

Features of AKS

  • Fully managed
  • Public IP and FQDN (private IP option)
  • Accessed with RBAC or Azure AD
  • Deployment of containers
  • Dynamic scale containers
  • Automation of rolling updates and rollbacks of containers
  • Management of storage, network traffic, and sensitive information  

AKS Architecture

Kubernetes cluster architecture

Cluster Master
When you create an AKS cluster, a cluster master is automatically created and configured and there is no cost for the cluster master, only for the nodes that are part of the AKS cluster. The cluster master includes the following core kubernetes componenets:
  • kube-apiserver- The API server is how the underlying Kubernetes APIs are exposed. This component provides the interaction for management tools, such as kubect1 or the kubernetes dashboard.
  • etcd- To maintain the state of your kubernetes cluster and configuration, the highly available etcd is the key value stored within kubernetes.
  • kube-scheduler- When you create or scale applications, the Scheduler determines what nodes can run the workload and start them.
  • kube-controller-manager- The Controller Manager oversees a number of smaller Controllers that perform actions such as replicating pods and handling node operations.  
The managed cluster master means that you do not need to configure components like a highly available store, but it also means that you cannot access the cluster master directly, but if you need to configure the cluster master in a particular way or need direct access to them, then you can deploy your own kubernetes cluster using aks-engine.

Nodes and node pools

An AKS cluster has one or more nodes, which is an Azure VM that runs the Kubernetes node component and container runtime:
  • The kubelet is the Kubernetes agent that processes the orchestration requests from the control plane and scheduling of running the requested containers.
  • Virtual networking is handled by the kube-proxy on each node. the proxy routes network traffic and manages IP addressing for services and pods.
  • The container runtime is the component that allows containerized applications to run and interact with additional resources such as the virtual network and storage. In AKS, Moby is used as the container runtime.
In AKS, the VM image for the nodes in your cluster is currently based on Ubuntu Linux or Windows Server 2019. Agent nodes are billed as standard virtual machines, so any discounts you have on the VM size you are using (including Azure reservations), are automatically applied.

Node Security

AKS nodes are Azure VMs that you can manage and maintain. The Azure platform automatically applies OS security patches to Linux nodes on a nightly basis and you can manually reboot the Linux nodes, or a common approach is to use Kured, an open-source reboot daemon for Kubernetes which are managed across the cluster using the same cordon and drain process as a cluster upgrade.

For Windows Server nodes, Windows Update does not automatically run and apply the latest updates. You should perform an upgrade on the Windows Server node pool(s) in your AKS cluster which creates nodes that run the latest Windows Server image and patches, then remove the older nodes. 

Nodes are deployed into a private virtual network subnet, with no public IP addresses assigned and to provide storage, the nodes use Azure Managed Disks as well as improve the redundancy, these disks are also securely replicated within the Azure datacenter. 

To protect your customer data as you run application workloads in AKS, the security of your cluster is a key consideration. 

AKS Networking

To allow access to your applications, or for application components to communicate with each other, Kubernetes provides an abstraction layer to virtual networking. In Kubernetes, Services logically group pods to allow for direct access via an IP address or DNS name and on a specific port. Security and filtering of the network traffic for pods is possible with Kubernetes network policies.  
The Azure platform also helps to simplify virtual networking for AKS clusters. As you open network ports to pods, the corresponding Azure network security group rules are configured. For HTTP application routing, Azure can also configure external DNS as new ingress routes are configured. 

Services

Kubernetes uses Services to logically group a set of pods together and provide network connectivity. The following service types are available:
  • Cluster IP- Creates an internal IP address for use within the AKS cluster. Good for internal-only applications that supports other workloads within the cluster.
  • NodePort- Creates a port mapping on the underlying node that allows the application to be accessed directly with the node IP address and port.
  • LoadBalancer- Creates an Azure load balancer resource, configures an external IP address, and connects the requested pods to the load balancer backend pool. 
  • ExternalName- Creates a specific DNS entry for easier application access. 

The Network Policy feature in Kubernetes lets you define rules for ingress and egress traffic between pods in a cluster. 

Volumes

Traditional volumes to store and retrieve data are created as Kubernetes resources backed by Azure storage. These data volumes can use Azure Disks or Azure Files:
  • Azure Disks can be used to create Kubernetes DataDisk resource. They are mounted as ReadWriteOnce, hence are only available to a single pod. For storage volumes that can be accessed by multiple pods simultaneously, use Azure Files.
  • Azure Files can be used to mount an SMB 3.0 share backed by an Azure Storage account to pods. Files let you share data across multiple nodes as well as pods and can use Azure Standard storage backed by regular HDDs, or Azure Premium storage, backed by high performance SSDs. 

Storage Classes

To define the different tiers of storage, such as Premium and Standard, you can create a StorageClass which also defines the reclaimPolicy. In AKS two initial StorageClasses are created:
  • default- Uses Azure Standard storage to create a Managed Disk. The reclaim Policy indicates that the underlying Azure Disk is deleted when the persistent volume that used it is deleted.
  • managed-premium- Uses Azure Premium storage to create Managed Disk. The reclaim Policy again indicates that the underlying Azure Disk is deleted when the persistent volume that used it is deleted. 
If no StorageClass is specified for a persistent volume, the default StorageClass is used.

Kubernetes Service Accounts

One of the primary user types in Kubernetes is a service account while the credentials for service accounts are stored as Kubernetes secrets, which allows them to be used by authorized pods to communicate with the API server. Most API requests provide an authentication token for a service account or a normal use account.
Normal user accounts allows more traditional access for human administrators or developers, not just services and processes while the external identity solutions can be integrated into Kubernetes, and this integrated identity solution is Azure Active Directory.

Azure Active Directory Integration

The security of AKS clusters can be enhanced with the integration of Azure Active Directory through which you can integrate on-premises identities into AKS clusters to provide a single source for account management and security. The Azure AD-integrated AKS clusters can grant users or groups access to Kubernetes resources within a namespace or across the cluster while providing a single source for user account management and password credential. The user can only access the resources as defined by the cluster administrator.

AKS and RBAC

One additional mechanism for controlling access to resources is  Azure Role-Based Access Controls (RBAC). With Azure RBAC, you can create a role definition that outlines the permissions to be applied. A user or group is then assigned this role definition for a particular scope, which could be an individual resource, a resource group, or across the subscription.

Roles and ClusterRoles

Roles are used to grant permissions within a namespace. But if you need to grant permissions across the entire cluster, or to cluster resources outside a given namespace, you can instead use ClusterRoles. A ClusterRole works in the same way to grant permissions to resources, but can be applied to resources across the entire cluster, not a specific namespace.

RoleBindings and ClusterRoleBindings

Role bindings are used to assign roles for a given namespace. This lets you logically segregate a single AKS cluster, with users only able to access the application resources in their assigned namespace. If you need to bind roles across the entire cluster, or to cluster resources outside a given namespace, you can instead use ClusterRoleBindings. This approach lets you grant administrators or support engineers access to all resources in the AKS cluster. 

Kubernetes Secrets

A Kubernetes Secret is used to inject sensitive data into pods, such as access credentials or keys. Secrets are only provided to the nodes that have a scheduled pod that requires it, and it is stored in tmpfs, not written to disk. They are stored within a given namespace and can only be accessed by the pods within the same namespace.

The use of Secrets reduces the sensitive information that is defined in the pod or service YAML manifest. Instead, you can request the Secret stored in Kubernetes API server as a part of your YAML manifest that only provide the specific pod access to the Secret. However, the raw secret manifest files contains the secret data in base64 format, therefore, this file should be treated as sensitive information, and never committed to source control.  

The secrets are written in the clear text on the node's volume which means the customers have to do two things:
  • Use file ACLs to secure the secrets file location.
  • Use volume-level encryption using BitLocker.    
   

 
To read part 1 of 3 please click here
To read part 2 of 3 please click here













Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Project Resourcing (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here Engineering - SIEM The SIEM engineers configures Microsoft Sentinel, including Log Analytics, Logic Apps, workbooks, and playbooks. They are also responsible for the following high-level tasks: Initial configuration of Azure tenant, including provisioning required resources, assigning access roles, and configuring workspace parameters like log retention, resource tagging, and blueprints.  Deployment and configuration of syslog/CEF log collection agents in appropriate locations to collect logs from on-premises devices. Generally, it will be joint effort of both system engineer and SIEM engineer, involving configuration and potential troubleshooting.  Working with system owners to enable log forwarding and configuring any required parsing of log data in Log Analytics.  Working with security operations to create and deploy KQL analytic rules to offer detections for SOC/Computer Security Incident Response Team (CSRIT) use. Tuning...
Read more