Very Large Database (VLDB) Migration to Azure (part 2)

By Ashwin Venugopal -

 




Migration Monitoring

In a VLDB migration, some of the important components are monitoring, logging, and diagnostics that can be configured during Development, Test as well as "dry run" migrations. It is essential for optimizing the migration and planning production cutover as it will help in achieving safe, repeatable, consistent as well as low downtime migrations with the guarantee of no data loss. You have monitor the following aspects while running OS/DB migration:
  • OS level parameters on DB and R3load hosts- CPU per thread, Kernel time per thread, Free Memory (GB), Page in/sec, Page out/sec, Disc I/O reads/sec, Disc I/O write/sec, Disk read KB/sec, Disk write KB/sec.

  • DB level parameters on SQL Server target- BCP rows/sec, BCP KB/sec, Transaction Log %, Memory Grants pending, Locks, Lock memory, locking/blocking.

Network monitoring is generally handled by network team while its configuration is totally depended on customer specific situation.

VLDB Migration Do's & Don't

Do's:

  • First of all, survey and inventory the current SAP landscape for identifying the Support Pack levels as well as finding if patching is needed or not to support the target DBMS.

  • Make a list of SAP OSS notes needed to be applied in the source system like updates for SMIGR_CREATE_DDL while also upgrading the SAP Kernels in the source systems to avoid a large change during migration to Azure.

  • Find a High Availability and Disaster Recovery solution through detailing the HA/DR concept in a document which should readily break-up the solution into the DB layer, ASCS layer, and SAP application server layer. But standalone solutions may require separate solutions like TREX or Livecache.

  • Create a sizing and configuration document detailing the Azure VM types as well as storage configuration along with Backup/Restore and DBMS configuration like memory settings, Max Degree of Parallelism as well as traceflags.  

  • Develop a network design document including VNet, Subnet, NSG and UDR configuration.

  • Document as well as implement the security and hardening concept. Remove Internet Explorer while creating an Active Directory Container for SAP Service Accounts and Servers as well as implement a Firewall Policy blocking all except a limited number of required ports.

  • Make an OS/DB Migration Design document which gives an account of the Package & Table splitting concept, number of R3loads, SQL Server traceflags, Sorted/Unsorted, Oracle RowID setting, SMIGR_CREATE_DDL setting, Perfmon counters, RSS settings, Accelerated Networking settings, Log File configuration, BPE settings, TDE configuration.

  • Develop a "Flight Plan" graph detailing the progress of R3load export/import on each test cycle which enables the migration team to validate if tunings and changes improve r3load export or import performance.

  • Make a performance testing plan and compare to the runtime on Azure. If there are any performance differences, run SAT, ST05, and the other SAP tools to recognize inefficient statements. 

  • Audit deployment and configuration, and ensure that cluster timeouts, kernels, network settings, NTFS format size are all inclined with the design documents. Also verify that the SAP Servers are in a separate AD Container which has a group policy applied to it with Firewall configuration.

  • Ensure that the lead OS/DB migration consultant is licensed while requesting the consultant name, s-user, and certification date. 

  • Ensure that the whole project team associated with the VLDB migration project within one physical location instead of geographically dispersed across several continents and time zones if possible.

  • Ensure that there is a proper fallback plan which is also the part of the overall schedule.

  • Select fast thread count Intel CPU models for the R3load export servers while not using the"Energy Saver" CPU models as they have much lower performance and do not use 4-socket servers.

Don't:

  • Don't subcontract one consulting organization to do the Export while another one to do the Import as due to a very tight coupling between Export/Import tuning and configuration, it is not always possible if we assign these tasks to different organizations will produce good results.

  • Don't economize on Azure hardware resources during the migration and go live as the VMs are charged per minute and can be very easily reduced in size. Instead you should leverage the most powerful VM during a VLDB migration. Customers have successfully gone live 200-250% oversized systems and then stabilized while running significantly oversized systems. After monitoring utilization  for about 4-6 weeks, VMs can be reduced in size or shutdown to lower costs.  






Comments

Post a Comment

Popular posts from this blog

Deployment (Part 3)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 2, please click  here Microsoft Monitoring Agent (MMA) MMA is used across multiple Microsoft solutions and has undergone a few name changes as its features and functionality have evolved. If it is being considered to be used, then following areas should be covered during a Microsoft Sentinel deployment project: Consider the timelines and the strategy for a migration to AMA (Azure Monitor Agent) before the MMA end-of-life.  Consider the central deployment and management methodology for MMA. Determine which endpoints are in scope for deployment as these affect the log ingestion volume significantly.  Azure Monitor Agent (AMA) Organizations with MMA as the main Microsoft Sentinel agent should start planning the migration to AMA as soon as possible to avoid a rushed migration. One of the main advantages of AMA is the ability to control the log collection policy centrally and apply different policies against different...
Read more

Deployment (Part 1)

By Ashwin Venugopal -
Image
  To read part 2, please click  here To read part 3, please click  here Azure Resources Microsoft Sentinel needs following resources to be created: Subscription (if a dedicated subscription(s) will be used) Resource group(s) Log Analytics workspace(s) Automation rules/playbook Alert rules Workbooks Microsoft Sentinel offers hundreds of alert rules, workbooks, and automation playbook templates along with hunting scripts. The templates can be used to activate/deploy schedule alerts, create customized dashboards, create automation playbooks and perform threat-hunting activities. Generally, once deployed, the resources created have to be adjusted to match the existing environment, configure local credentials, etc. Methods of deployment: Manual- Administrator can manually configures the Microsoft Sentinel resources with the help of Azure portal. Any manual process has the inherent risks of human operator error, lack of compliance with potential change control procedures, and u...
Read more

Deployment (Part 2)

By Ashwin Venugopal -
Image
  To read part 1, please click  here To read part 3, please click  here Microsoft Sentinel Content Hub Content in Microsoft Sentinel includes any of the following types: Data connectors offer log ingestion from different sources into Microsoft Sentinel. Parsers give log formatting/transformation into ASIM formats, supporting usage across various Microsoft Sentinel content types and scenarios. Workbooks provide monitoring, visualization, and interactively with data in Microsoft Sentinel, highlighting meaningful insights for users.  Analytics rules give alerts that point to relevant SOC actions via incidents.  Hunting Queries are used by SOC teams to proactively hunt for threats in Microsoft Sentinel. Notebooks help SOC teams in using advanced hunting features in Jupyter and Azure Notebooks. Watchlists support the ingestion of specific data for enhanced threat detection and reduced alert fatigue.  Playbooks and Azure Logic Apps Custom Connectors provide featu...
Read more