videoIntegration & APIsofficial - Microsoftreviewed - checked by Opus

Introduction to Business Central Cloud Migration Tooling (2022)

Business Central cloud migration tooling from on-premises to Business Central online, using Azure Data Factory and a self-hosted integration runtime. It is evidence for preparation (table alignment, data cleanup), setup and permissions, troubleshooting, table mapping, the new upgrade flow and migration APIs.

  1. 0:00 Introduction and Session Overview
  2. 1:56 Cloud Migration Journey and Version History
  3. 4:31 Support Timeline and Migration Statistics
  4. 6:41 Pre-Migration Preparation: Table Definitions and Alignment
  5. 9:13 Data Cleanup and Sanitization
  6. 11:34 Cloud Migration Architecture and Pipeline Creation
  7. 16:28 Common Setup Errors and Troubleshooting
  8. 20:42 SQL Connection Issues and Migration Execution
  9. 24:17 Replication Optimizations and Troubleshooting
  10. 27:48 Metadata Limitations and Extension Management
  11. 31:02 Data Migration Errors and Performance Optimization
  12. 33:35 Data Mapping and Table Migration Strategies
  13. 35:57 Upgrade Process and Tenant Management
  14. 39:32 Upgrade Failure Recovery and Data Transfer
  15. 43:00 API Automation and Performance Improvements
video Microsoft Dynamics 365 Business Central (YouTube) · 2023-11-21Watch on YouTube

Overview

The session covers the cloud migration journey from on-premises Business Central, including the one-step upgrade from version 14 to version 21 and the option to replicate directly to the cloud. It explains the architecture: a wizard in the destination environment, a preparation pipeline and a replication pipeline in Azure Data Factory, and a self-hosted integration runtime that copies data through regional blob storage to the tenant database.

The presenters also cover common errors and fixes. Topics include connection strings, permissions, the 4 MB metadata limit, out-of-memory and row delimiter errors, and database replacement during replication. The session then covers table mapping, the new upgrade flow with automatic point-in-time restore, the data transfer feature for upgrades, and cloud migration APIs for bulk automation.

Key points

  • From version 14 you can do a single-step upgrade to version 21 on-premises and then replicate, or replicate straight from 14 to the cloud and let the cloud run the upgrade. Microsoft says it will support the 14 path as long as it technically can.
  • Table definitions must be aligned, including primary keys, field names and data types. Custom fields added to standard tables are ignored during migration unless they are refactored into extensions. Extra on-premises fields are silently ignored with no warnings. A prototype to split them into table extensions automatically was being tested.
  • Run the Invoke-NavSanitizeField commandlet before production migration to fix corrupted code fields. It does not cover company names, which are text fields. Those must be detected with a query and renamed before migration.
  • The self-hosted integration runtime needs .NET 4.72 or higher and must stay online during migration. A long offline period can trigger automated cleanup of Azure resources and force setup to be repeated. No NST is needed.
  • Setup needs the super role in BC, and customer consent if the user is a delegated admin. The SQL user needs the sysadmin server role (captioned as ‘CIS admin’) and the db owner role. Connection strings must match the exact documented format.
  • Metadata errors come from a 4 MB Azure Data Factory limit. Reduce the companies migrated at once (about 10 to 15 is suggested, depending on data and extensions) and uninstall unneeded extensions in the cloud tenant before migrating.
  • The first run is a full replication, and later runs are incremental through SQL change tracking. Do not replace or restore the on-premises database during migration. It causes invalid object name errors and invalidates change tracking, which forces a full replication again.

Features

Feature Status At
Cloud Migration Tooling status not stated, demoed 0:55
One-Step Upgrade to Version 21 status not stated 3:17
Direct Cloud Migration from Version 14 status not stated 3:57
Table Extension Auto-Refactoring status not stated 8:13
Invoke-NavSanitizeField Commandlet status not stated 9:24
Table Mapping for Cloud Migration status not stated, demoed 7:33
Azure Data Factory Integration for Migration status not stated, demoed 11:53
Self-Hosted Integration Runtime status not stated, demoed 12:54
Cloud Migration Wizard status not stated 14:26
Preparation Pipeline status not stated 15:17
Replication Pipeline status not stated 16:07
User Permissions Validation status not stated 17:03
SQL Connection String Validation status not stated 18:01
Database Query Performance Optimization status not stated 19:10
Version Matching Validation status not stated 19:52
SQL Connection Verification status not stated 20:42
Migration Run Execution status not stated 22:21
Data Compression and Two-Step Transfer status not stated 23:01
Large Table Replication status not stated 23:43
Small Table Optimization status not stated 24:17
Full and Incremental Replication status not stated 25:14
Integration Runtime Offline Handling status not stated 26:35
Database Replacement Impact status not stated 27:15
Metadata size limitations status not stated 27:48
Multi-company migration optimization status not stated, demoed 28:40
Extension cleanup for migration status not stated 29:20
Table-level replication exclusion status not stated 29:40
Tenant media bulk copy handling status not stated 30:20
Table-to-table full copy error handling status not stated 31:22
On-premises infrastructure monitoring status not stated 32:18
Database statistics and index optimization status not stated 32:38
Default data mapping for table replication status not stated 33:35
Missing table warning during replication status not stated 34:16
Table mapping for different table names status not stated, demoed 35:16
Improved upgrade logic for cloud migration status not stated 36:31
Single-session upgrade with full app inclusion status not stated 37:11
Upgrade status tracking in Admin Center status not stated, demoed 37:52
Automatic point-in-time restore on upgrade failure status not stated, demoed 39:32
Legacy upgrade flow support status not stated 40:12
Data transfer feature status not stated 40:46
Long-running methods optimization status not stated 43:13
Cloud migration APIs status not stated 44:14

AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

  • other “Invoke-NavSanitizeField” at 10:22
  • page “data archive page” at 11:14
  • table “cloud migration management” at 29:00
  • table “tenant media” at 30:41
  • other “is full copy” at 31:22

Not found in BC28-30: page “data archive page”, table “cloud migration management”, table “tenant media”.

Quotes

  • 2:16 “version 14 and this is a rather important version because this is the last version which is supporting CR”
  • 3:17 “you can do a single step upgrad to the version 21 skipping all of the versions in between”
  • 4:51 “we are going to support this part as long as we technically can because we want to make you as efficient when you’re running”
  • 6:08 “in the last 30 days we had 564 unique customers migrating”
  • 7:13 “these properties are must for both Cloud migration and for operate so you must align primary keys”
  • 8:13 “the cloud migration is going to ignore these fields currently so we will replicate the Full Table ignoring the fields that you have added”
  • 8:13 “we have a prototype that we are currently testing so we believe that during the cloud migration we will be able to split the”
  • 13:14 “self hosted integration runtime and this is installed within the network of the customer on prem and then registered with our azure data”
  • 13:48 “there is no NST needed no NST running needed um we really connect directly to the database”
  • 17:20 “the user that is doing the setup in BC has to have the super role”
  • 25:34 “we won’t move all the data but only those that are Delta change so incremental uh that would make it faster”
  • 25:34 “it’s important that you do not replace or you don’t change your on premise database otherwise the change tracking becomes invalid”
  • 25:55 “the replication is done by Asia data Factory and it’s mostly very stable we don’t see so many errors”
  • 26:55 “you keep the machine that hosts your integration run time online all the time while you’re doing Cloud migration”
  • 27:28 “these are the most difficult errors that you face they’re not very common there are like maybe 10 errors per months”
  • 28:08 “Asia data Factory has a limitation for this text description to be less than four megabytes”
  • 29:00 “we suggest that you kind of uh reduce the number of companies for migration you can do it on cloud migration management page”
  • 33:35 “an optimal size is about 10 15 companies it could be more it depends on your data”
  • 33:55 “we just copy them but we copy them because the SQL names match so we look at the SQL names and that usually the”
  • 36:31 “we are leveraging the upgrade Logic for the up if your Cloud migrating from previous versions”
  • 37:31 “we are triggering the full upgrade code all apps are going to be included and you will have a single upgrate Logic”
  • 38:12 “you can track the status in the admin Center so if you say yes you will get the message that the upgrade was scheduled”
  • 39:32 “you’ll get a full stack Trace we are looking into providing you with a nice error message message as well so you can see”
  • 40:12 “so now you can fix the errors replicate additional data or try to run the upgrade again so with this new flow you’re also”
  • 40:25 “because we are just rolling this one out we are still supporting the old upgrade flow so if you took any dependencies to how”
  • 41:27 “if you want to update 1 million entries it’s going to take anywhere between 2 hours or 24 hours and more the reason why”
  • 42:07 “with data transfer one to million entries is approximately taking two to 5 minutes so with this feature now we are able to update”
  • 43:00 “currently it’s marked as Target on Prem which means that you will unfortunately not be able to use it for SAS solution but you”
  • 43:13 “for this release we have shrinked the number of long running methods by 11 because we use this new data transfer feature to optimize”

Disclaimers in the video

  • 4:31 subject-to-change: we are going to support this part as long as we technically can
  • 8:33 preview: we have a prototype that we are currently testing so we believe that during the cloud migration we will be able to split the main table with table extensions into main table and table extension without the need for you to write any code and if we are successful we are going to announce it so just keep your eyes open there might be an announcement coming in a few weeks
  • 36:31 preview: we are just rolling this one out we are still supporting the old upgrade flow
  • 40:46 other: I will not be able to talk about it much I will just briefly cover it
  • 42:39 other: it has some limitations because it’s a very powerful feature
  • 43:00 preview: currently it’s marked as Target on Prem

Presenters (as heard): Nicola, Roman, Julia, Yia.