What's New: Cloud Migration and Upgrade (2023)

Overview

The session covers backend and UI changes to the cloud migration replication engine. Backend changes include storing replication metadata in a database table to bypass the 4 MB Data Factory limit, removing the limit on tables and companies per run, lowering the table-to-table copy threshold from 500 MB to 250 MB, disabling indexes during copy, smaller bulk sizes, longer change tracking retention, and direct linking of Azure SQL databases. Reported gains are 8 to 22 percent from index handling and 60 to 80 percent from bulk copy changes.

It then demos a new migration UI with failed-table drill-down, an unblock table action, company and table filtering, a complete-migration sanity check, and a sanitize tables action. It also shows moving customized fields to table extensions, telemetry and Power BI monitoring, and plans: stopping the direct upgrade path from version 14 after version 26, removing obsolete objects, and jump builds every 5 versions. Several items are described as not yet implemented and subject to change.

Key points

• Replication metadata now goes through a database table instead of Data Factory parameters, so the number of tables and companies per run is no longer limited. Business Central still supports at most 300 companies per environment.
• The large-table threshold for optimized table-to-table copy dropped from 500 MB to 250 MB. Indexes are disabled during the copy and rebuilt afterward, which gave 8 to 22 percent in the speaker’s tests.
• Smaller bulk sizes made bulk copy 60 to 80 percent faster. Timeouts can still happen on very weak or busy SQL servers.
• With Azure SQL as the source, the engine can link source and destination databases directly. This also allows SQL variant type fields to be replicated.
• The new UI shows failed tables, has an unblock table action, and blocks completion while failed tables exist. The old UI is still supported for existing customizations.
• Customized fields on Microsoft tables can be moved to table extensions through table mappings. Fields must be renamed to match the on-premises names. Partners can also supply mappings in code by subscribing to an event in codeunit 4001.
• Microsoft plans to stop the direct upgrade and cloud migration path from version 14 after version 26, about two years out. A jump build every 5 versions is announced and may change.

Features

AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

• other “SQL change tracking” at 4:58
• other “AL number sequences” at 7:24
• table 18 “Customer” at 25:29
• table “G entry” at 27:46
• table 405 “Change Log Entry” at 21:46
• table 9650 “Custom Report Layout” at 20:16
• table 77 “Report Selections” at 20:16
• table “upgrade TXS” at 20:57
• codeunit “4001” at 32:36

Not found in BC28-30: table “G entry”, table “upgrade TXS”, codeunit “4001”.

Quotes

• 1:28 “the number of uh business Central tenants migrating their data from on premise has almost doubled last year comparing to the previous year and”
• 1:48 “back then we removed the hard limitation on the amount of the UN premis data that can be replicated to the cloud and that”
• 6:22 “it has become possible to replicate an unlimited number of tables in a single migration run and we remove the limitation on the number”
• 7:24 “the AL number sequences data type is supported by Cloud migration and the numbering can continue in the BC tenant after migration instead of”
• 8:39 “our test measurement show 8 to 22% performance benefit for the new process comparing to the old way of inserting the data with all”
• 9:30 “the whole process of B copy became 60 to 80% faster”
• 11:08 “if uh Asia SQL is used a source it is possible to link the source and the destination Asia SQL databases directly without the”
• 13:09 “the more powerful service tier you get the faster and smoother your Cloud migrations will be”
• 13:47 “it is extremely hard to get an overview which data has been moved and if there has been any problems with the cloud migration”
• 14:47 “we have decided to support both for the time being because we believe that some of you have done the customizations to the old”
• 17:28 “now you’re able to see which tables have failed the replication and which tables are requiring your attention immediately”
• 18:08 “to enable you to be more efficient we have added this action which is called unblock the table”
• 20:16 “tenant media if it was copied successfully it needs to have more records than there are in the on Prem database because we never”
• 25:13 “we have shipped it with the version 21 actually because we backported the code to make it more efficient”
• 26:06 “we are going to insert the default mapping by name and we are going to search for all of the fields that we can”
• 26:21 “you are able to do the these simple migration table mappings and this will copy the custom fields to the Target table if the”
• 26:41 “you need to rename the fields so you will need to prefix them so they are called the same as in on Prem”
• 29:29 “we have added a dedicated UI for that and you can access it through this section which is called add table mappings”
• 32:36 “if you do this through code then table mappings are going to be inserted each time when you complete the setup wizard or if”
• 33:56 “approximately few weeks ago we have added support for the Telemetry and we have added a lot of events to the partner Telemetry”
• 37:45 “we are stopping the direct upgrate and cloud migration po from version 14”
• 38:39 “we have 7 years of obsolete removed objects and these objects are taking around 10% of the database”
• 39:20 “we have received the request from the partners to ship the ability to remove the obsolete objects earlier before version 26 and we are”
• 39:20 “we have received the request from the partners to ship the ability to remove the obsolete objects earlier before version 26”
• 40:41 “the only thing that you need to do to remove the table and the field from schema is to have a version in which”
• 42:03 “in order to get to 26 you need to go through 25 so all of these on Prem versions they will need to upgrade”
• 42:24 “we are planning to do these updates every five years so in every five versions We are planning to introduce a jumper build”
• 43:42 “currently there are no plans of stopping the on Prem support and if there are any PLs they are going to be announced in”
• 45:13 “if you would like to override this decision you need to do workarounds either to delete the content from the table or to use”

Disclaimers in the video

• 6:22 other: business Central supports not more than 300 companies in single environment
• 9:30 other: Timeouts may still happen for the extremely weak or busy SQL service
• 14:29 other: we have decided to support both for the time being because we believe that some of you have done the customizations to the old UI
• 20:16 not-in-this-release: we do not have the warning system in place yet
• 27:21 coming-later: we are planning to introduce name spaces and the goal of the development team is to eliminate prefixes and suffixes in all of the objects
• 37:45 coming-later: we will need to stop this upgrade but this is in approximately two years
• 38:59 subject-to-change: we are looking into this we have not started the implementation of this feature yet however the current thinking is
• 39:40 preview: we have not started the implementation of this feature yet however the current thinking is that it is going to work like this
• 39:40 coming-later: we have not started the implementation of this feature yet however the current thinking is that it is going to work like this
• 43:21 subject-to-change: this thing that you have just seen is our current thinking and these are the things that we are planning to do with the current knowledge
• 44:02 subject-to-change: this current schedule that you have seen it can change and in case of any changes we are also going to announce it to you

Presenters (as heard): Nicola, Julia.