Business Central Under the Hood episode 1: Developing AI in Business Central

Overview

Episode 1 of the Under the Hood series, published 2024-02-13, explains how the Business Central team builds AI features. It mentions marketing text generation and bank reconciliation matching as shipped features, and describes a chat Copilot that focuses on guidance and navigation and is still improving in quality.

The talk then covers practical prompt engineering: few-shot learning, Chain of Thought, function calling, feeding the LLM current data to limit hallucinations, and meta-prompting against jailbreaks. It also describes the Copilot service architecture with a chat pane, orchestrator and plugin service, the difficulty of testing fuzzy inputs and outputs, and Azure OpenAI Studio as a tool for building prompts.

Key points

• Marketing text generation (generally available) creates item marketing text from keywords; bank reconciliation already uses LLMs to match bank records.
• Few-shot learning shows the LLM examples of input and expected output, and makes it easy to specify formats such as JSON or XML.
• Chain of Thought tells the LLM to reason step by step; it outputs intermediate steps and takes longer to respond.
• Function calling gives the LLM function definitions and parameters; the LLM does not execute code, it only says what to call and with which parameters.
• To limit hallucinations, always feed the LLM current data and instruct it in the prompt to answer only from the provided data.
• Chat architecture: a chat pane in the client talks to a cloud Copilot service with an orchestrator for multi-turn history and context and a plugin service for skills.
• Meta-prompting and Azure OpenAI content filters reduce jailbreaking, but new jailbreak techniques keep appearing; the user must stay in the loop.

Features

AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

• other “bank reconciliation” at 4:47
• other “Chat pane” at 15:56
• other “Orchestrator” at 16:16
• other “Plugin service” at 16:37

Quotes

• 0:10 “we are going to tell you how we develop these features how we design them how we architect them”
• 3:33 “we have released at least marketing text where we can kind of generate the marketing text based on kind of keywords”
• 3:53 “we are starting slowly we’re also learning a lot by building these features turns out this is this is probably harder than we thought”
• 4:47 “we have already shipped uh bank reconciliation right try to match use llms to match bank records behind the product”
• 8:10 “prompting is not an exact science right because it’s no one let’s put it that way no one can really predict exactly what the”
• 10:52 “because this may not be really predictable right you actually need another prompt to to validate in many cases at least right”
• 12:14 “one of them and kind of a very successful technique is what kind of called few shot learning right and this basically that in”
• 13:35 “CH for is really just like what humans does right you kind of break break down the problem you you solve small sub parts”
• 14:15 “let’s think about it step by step it’s really breaking down the you write that in the prompt you simply just put that in”
• 16:16 “a chat is like it’s it’s a multi-turn of or turn of questions and answers right and the user may clarify or change subject”
• 19:18 “it tells you how to execute it doesn’t actually execute the code right that’s up to you but it guides you”
• 19:38 “Hallucination is basically when you don’t give it up to-date information right and the way this whole thing is trained it really wants to”
• 20:18 “you always feed the data and you can instruct in the prompt that you really only use this data”
• 20:59 “jailbreak the prompt in getting it to do something else then you know the developer or whoever made this prompt was intended it to”
• 23:40 “maybe you should not start with the absolutely mind-blowing stuff unless that mind-blowing stuff is really simple I think it’s good to develop a”
• 25:02 “on Azure there’s also the Azure um open a studio yeah right which is also a it’s very useful tool to kind of construct”

Disclaimers in the video

• 2:53 not-in-this-release: we don’t talk about it yet we don’t really talk about it yet right but but I I hope we will be able to productize this somewhat soon
• 23:20 other: we are releasing this tool kit uh for partners and developers - describes toolkit being released for external use

Presenters (as heard): Vincent Nicholas, Michael.