Business Central Under the Hood episode 13: Inside the Agent Runtime

Overview

Episode 13 of Business Central Under the Hood explains how the agent runtime works. The runtime holds the data model for agent tasks, the task execution engine and supporting functions such as timeline data. Agents interact with Business Central through the Logical Client API, a JSON API originally built to render pages on Windows, web and mobile clients. Because an LLM is stateless, the runtime keeps partial memory of visited pages, search results and agent notes and includes it in later prompts.

The talk then covers the supporting prompts: input content analysis, output validation, grounding with a second LLM call, loop detection with recovery nudges, and timeline grouping. It ends with model selection (mini vs full models, reasoning models tested and found too slow), scenario-based accuracy and challenge tests, prompt tuning when models change, and planned work such as MCP tools, cross-task memory and knowledge base access.

Key points

• The Logical Client API gives agents page fields, actions and tooltips as JSON. It existed long before AI to render the Windows, web and mobile clients, and is reused for agents.
• LLM calls are stateless, so the runtime keeps memory (visited pages, search results, notes the agent writes) and includes it in later prompts. Today that memory lasts only within the current task.
• Grounding uses a second LLM prompt as an evaluator to check that generated content with data matches data the agent has seen. It costs two LLM calls instead of one.
• Loop detection separates expected repetition from true loops. When it finds one, the runtime speaks as the user and tells the agent to get out of it. If that fails, a user must intervene.
• The runtime uses many prompts (the host guessed 11, Esteban said nowadays more like 20) and more than one model. Simpler prompts like summarization and grounding can often use mini models, while execute task, the most complex prompt, uses a full model.
• Reasoning models (o1, GPT-5 with reasoning) added significant latency for execute task with no clear benefit for this use case so far. The team keeps re-evaluating.
• Tests check that the goal is reached, not exact steps. They are split into accuracy tests and challenge tests, and every model upgrade so far has needed some prompt tweaks.

Features

AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

• other “Agent Runtime” at 0:52
• other “Agent SDK” at 2:13
• other “Logical Client API” at 3:07
• other “execute task” at 27:29
• other “validation prompt” at 27:29

Quotes

• 0:11 “we we have a few agents already in Business Central, the sales order agent and and the the payable agent”
• 1:16 “all the tables for storing your agents tasks, keeping track of what agents are doing”
• 3:07 “it it has this JSON based API uh that we call the the logical client API that provides all the information necessary”
• 8:18 “when if you do a pure call to LLM LLM it’s it’s it’s completely stateless. Like every time you do a call to an”
• 9:35 “the agent runtime remembers for example where you have been, which page you come from, right?”
• 13:26 “when when an llm generates content um you you want that content uh especially if it contains data to to be grounded”
• 13:26 “grounding basically um when when an llm generates content um you you want that content uh especially if it contains data to to”
• 14:22 “instead of running one call to the LLM, you actually do two, right? So, you do first you do generate that content”
• 15:02 “Yeah. So, this is what’s called grounding because you make sure it’s grounded in the in the data. Okay. Yeah, that’s a very common”
• 19:53 “I believe we have all in all, correct me if I’m wrong, but the I think we have something like 11 prompts. So, is”
• 20:50 “Is that fair to say that the execute task prompt we talked about before, that’s the more complex That’s the most complex of them”
• 21:24 “We We are using We’re not using only one model. Uh, the runtime is using more than one Um, yes, so we some of”
• 23:48 “what we found, uh, back then was there was a significant increase in in latency. Yeah, so it’s a lot slower. Reasoning models, especially”
• 24:44 “Um, so far we have not found, uh, any immediate benefit. benefit for our use case, but we are constantly evaluating and thinking, um,”
• 27:03 “The more important thing, first of all, is is making sure that the tests don’t break when we switch to a new model, which,”
• 27:18 “Sometimes you you have to make some tweaks, and otherwise things that were working don’t work anymore with again with our specific prompt and”
• 29:14 “if we only measured on the the one which have the higher accuracy, we would we wouldn’t we wouldn’t know if it the model”
• 31:30 “it seems to have been so far it actually seems to indicate there is there is a dependency and correlation between your prompt and”
• 31:42 “it was when we trying gpt 5 reasoning some of our tests uh for example had a test email saying”
• 32:44 “it’s not necessarily about us testing that it has performed the exact same steps in the exact same order. It’s more that it has”
• 35:34 “we are looking into is sort of this this we call it cross-task memory. You could consider it teaching or something as well, where”
• 36:24 “the mechanism for when to remember, whether it will ultimately be that you have a checkbox and say remember this when you’re providing the”
• 39:26 “every tool we give, every prompt change we make might lead to some other scenarios now the LLM making a different decision”
• 39:48 “we have also built two agents that have lots of test cases that we can always be running and making sure that are still”
• 40:12 “usually changes that only impact accuracy is are are changes that are not necessarily visible”

Disclaimers in the video

• 25:44 coming-later: we might have a combination eventually, you know, where every once in a while we reason, um, and then we continue with a smaller model, and then we reason again.
• 34:40 coming-later: the ability to give the agent some tools like as as an agent developer, for example, using MCP
• 36:44 subject-to-change: the mechanism for when to remember, whether it will ultimately be that you have a checkbox and say remember this when you’re providing the feedback or we just decide for for ourselves like some other systems sometimes do, we it’s it’s still I would say TBD
• 37:11 coming-later: the agent being able to consult some knowledge base like Microsoft Learn. Yeah, outside BC.

Presenters (as heard): Vincent, Esteban.