--- id: video/aNRzlbxVPWE type: video title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" summary: "Writing agent evals in AL for Business Central agents: YAML-driven, data-driven tests that follow the agent task flow (message arrival, agent run, output verification, multi-turn), using the agent test library and its built-in keywords. Covers suite setup, placeholders, logging and the library agent's public methods." tier: official language: en tags: - agent testing - evals - yaml-driven tests - data-driven testing - placeholder formulas - agent automation - test structure - suite setup - multi-turn conversation - al test library - agent evaluation - logging and validation system: development review: state: reviewed by: opus at: "2026-10-07T22:47:27.388Z" flags: [] generated: at: "2026-10-07T22:47:27.436Z" pipeline: 0.2.0 prompts: extract-video: 1 summarize-video: 2 input_hash: d22f670125777231278b57810187999161131c19bc2b29ec499a3b938660bbe7 evidence: - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=41s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 41 quote: going to end up with one of the following it is either going to create a message for you to review it - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=97s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 97 quote: ERP is quite busy and there is a lot of users and code that is changing the data, data might be changed between the - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=223s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 223 quote: in our testing we discovered that it wasn't searching for the customer 10 000 it knew that - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=270s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 270 quote: are going to be actual data-driven tests specified in YAML. The next component is library agent, which is going to help you to automate - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=648s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 648 quote: Unfortunately, currently it's not possible to check on the message if the message of the intervention request as as you expecting it. But we - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=840s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 840 quote: now you can replace the dates with placeholders like this. And this is going to calculate dynamically the work date based on the current - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=840s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 840 quote: the problem is that in the tests, I would need to change this one quite often. And the solution that we did is that - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=853s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 853 quote: this is going to calculate dynamically the work date based on the current work date and the formula that you provide here - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=985s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 985 quote: you can test You can run the same tests on different agents, change their instructions, maybe change the model, and see how that the - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1108s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 1108 quote: we are going to default the other fields like posting groups units of measure automatically by using test libraries - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1346s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 1346 quote: you're not using the asserts to check the data. We are actually using the text field, which is called error reason. - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1524s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 1524 quote: we have added the library agent. It has 27 public methods which will allow you to automate every single interaction that you can do - kind: video url: https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1562s title: "Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2)" date: "2026-06-29T13:46:29.000Z" commit: null t: 1562 quote: this one is quite good if you are going to invoke the agent through the action or AL code. So, you would like to links: learn: [] objects: [] features: [] topics: [] localizations: [] videos: [] posts: [] guidelines: [] video_id: aNRzlbxVPWE channel: yt-microsoft source_name: Microsoft Dynamics 365 Business Central (YouTube) url: https://www.youtube.com/watch?v=aNRzlbxVPWE published_at: "2026-06-29T13:46:29.000Z" duration_s: 1598 captions: full audience: - developer - functional consultant chapters: - t: 0 title: Mapping agent task flow and test structure - t: 136 title: Evals framework and suite setup strategy - t: 252 title: "Components: data sets, library agent, and extensibility" - t: 347 title: YAML-driven test definition and structure - t: 467 title: "Test sections: setup, message, and agent execution" - t: 625 title: Output verification and multi-turn testing - t: 763 title: Placeholders for dynamic date and datetime formulas - t: 917 title: AL test structure and agent initialization - t: 1043 title: Suite setup with YAML data and test libraries - t: 1208 title: Turn setup and executing agent interactions - t: 1323 title: Validating turns and error logging - t: 1454 title: Downloaded logs and LLM judge analysis - t: 1510 title: Library agent customization and public methods features: - name: Agent task flow mapping status: unclear t: 15 verified: false status_source: video - name: Suite setup strategy status: unclear t: 196 verified: false status_source: video - name: Data sets and YAML-driven tests status: unclear t: 252 verified: false status_source: video - name: Extensible framework with atomic functions status: unclear t: 309 verified: false status_source: video - name: YAML test definition syntax status: unclear t: 347 verified: false status_source: video - name: Turn setup section status: unclear t: 467 verified: false status_source: video - name: Message arrival with attachments status: unclear t: 529 verified: false status_source: video - name: Expected data verification status: unclear t: 625 verified: false status_source: video - name: Multi-turn conversation testing status: unclear t: 669 verified: false status_source: video - name: Date and datetime placeholders in data-driven tests status: unclear t: 804 verified: false status_source: video - name: Agent picker in test execution status: unclear t: 966 verified: false status_source: video - name: Get eval suit setup data input keyword status: unclear t: 1043 verified: false status_source: video - name: Test library data creation and field defaulting status: unclear t: 1095 verified: false status_source: video - name: Suite setup completion caching status: unclear t: 1172 verified: false status_source: video - name: Run turn and wait keyword status: unclear t: 1254 verified: false status_source: video - name: Get expected data keyword for validation status: unclear t: 1323 verified: false status_source: video - name: Error reason field for test failure logging status: unclear t: 1346 verified: false status_source: video - name: Finalize turn keyword with history logging status: unclear t: 1379 verified: false status_source: video - name: Log download with serialized task output status: unclear t: 1454 verified: false status_source: video - name: Library agent public methods for customization status: unclear t: 1510 verified: false status_source: video objects_mentioned: - other animal data sets - other library agent - other Get eval suit setup data input - other Run turn and wait - other Get expected data - other Finalize turn - other Wait for the task to complete quotes: - t: 41 text: going to end up with one of the following it is either going to create a message for you to review it check: fuzzy - t: 97 text: ERP is quite busy and there is a lot of users and code that is changing the data, data might be changed between the check: exact - t: 223 text: in our testing we discovered that it wasn't searching for the customer 10 000 it knew that check: fuzzy - t: 270 text: are going to be actual data-driven tests specified in YAML. The next component is library agent, which is going to help you to automate check: exact - t: 648 text: Unfortunately, currently it's not possible to check on the message if the message of the intervention request as as you expecting it. But we check: exact - t: 840 text: now you can replace the dates with placeholders like this. And this is going to calculate dynamically the work date based on the current check: snapped - t: 840 text: the problem is that in the tests, I would need to change this one quite often. And the solution that we did is that check: exact - t: 853 text: this is going to calculate dynamically the work date based on the current work date and the formula that you provide here check: exact - t: 985 text: you can test You can run the same tests on different agents, change their instructions, maybe change the model, and see how that the check: exact - t: 1108 text: we are going to default the other fields like posting groups units of measure automatically by using test libraries check: exact - t: 1346 text: you're not using the asserts to check the data. We are actually using the text field, which is called error reason. check: exact - t: 1524 text: we have added the library agent. It has 27 public methods which will allow you to automate every single interaction that you can do check: exact - t: 1562 text: this one is quite good if you are going to invoke the agent through the action or AL code. So, you would like to check: exact --- # Getting Started With Agent Testing: How to Write Agent Evals in AL (Part 2) > Writing agent evals in AL for Business Central agents: YAML-driven, data-driven tests that follow the agent task flow (message arrival, agent run, output verification, multi-turn), using the agent test library and its built-in keywords. Covers suite setup, placeholders, logging and the library agent's public methods. [Watch on YouTube](https://www.youtube.com/watch?v=aNRzlbxVPWE) · Microsoft Dynamics 365 Business Central (YouTube) · 2026-06-29 · 26:38 · tier official · reviewed (checked by Opus) ## Overview Part 2 of the agent testing series shows how to structure an agent eval. The test follows the agent's flow: a message arrives, the agent acts, and the output is verified as a message, a review request or an intervention request. Multi-turn conversations are covered by mocking user interventions in YAML. The presenter notes that data can change between turns in a busy ERP. The video then walks through the YAML test definition (suite setup, test name, description, turns) and the AL test code that reads it. It covers suite setup with custom data, turn setup, running a turn, validating results, error logging, downloaded logs for LLM judge analysis, and the library agent's public methods. ## Key points - Create custom master data for the suite setup instead of relying on demo data. In the team's testing, the agent recognized customer 10000 from public demo data, probably from model training. - Tests are driven by YAML with predefined sections (turn setup, message arrival with sender, title, body and attachments, expected data) plus free-format sections that you implement in AL code. - Attachments can come from resources or from AL-generated reports converted to attachments. Dynamic attachment generation needs custom AL code. - Date and datetime placeholders in YAML calculate dates from the current work date and a formula, so hardcoded dates need no manual updates. They also work in data-driven tests that are not agent tests. - Instead of asserts, tests check data and accumulate issues in an error reason field, returning true or false. The finalize turn keyword stores results in the history log, and a fact box shows why a test failed (recently added). - Downloaded logs contain serialized tasks, steps, inputs, expected data and error messages, and can be uploaded to Azure AI Foundry for LLM judge analysis. - The library agent has 27 public methods, with asynchronous ones ending in 'and wait'. Checking the text of an intervention request message is currently not possible, and the presenter says it is planned. ## Chapters - [0:00](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=0s) Mapping agent task flow and test structure - [2:16](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=136s) Evals framework and suite setup strategy - [4:12](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=252s) Components: data sets, library agent, and extensibility - [5:47](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=347s) YAML-driven test definition and structure - [7:47](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=467s) Test sections: setup, message, and agent execution - [10:25](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=625s) Output verification and multi-turn testing - [12:43](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=763s) Placeholders for dynamic date and datetime formulas - [15:17](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=917s) AL test structure and agent initialization - [17:23](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1043s) Suite setup with YAML data and test libraries - [20:08](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1208s) Turn setup and executing agent interactions - [22:03](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1323s) Validating turns and error logging - [24:14](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1454s) Downloaded logs and LLM judge analysis - [25:10](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1510s) Library agent customization and public methods ## Features | Feature | Status | At | |---|---|---| | Agent task flow mapping | status not stated, demoed | [0:15](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=15s) | | Suite setup strategy | status not stated, demoed | [3:16](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=196s) | | Data sets and YAML-driven tests | status not stated, demoed | [4:12](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=252s) | | Extensible framework with atomic functions | status not stated | [5:09](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=309s) | | YAML test definition syntax | status not stated, demoed | [5:47](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=347s) | | Turn setup section | status not stated, demoed | [7:47](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=467s) | | Message arrival with attachments | status not stated, demoed | [8:49](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=529s) | | Expected data verification | status not stated, demoed | [10:25](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=625s) | | Multi-turn conversation testing | status not stated, demoed | [11:09](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=669s) | | Date and datetime placeholders in data-driven tests | status not stated | [13:24](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=804s) | | Agent picker in test execution | status not stated, demoed | [16:06](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=966s) | | Get eval suit setup data input keyword | status not stated | [17:23](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1043s) | | Test library data creation and field defaulting | status not stated | [18:15](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1095s) | | Suite setup completion caching | status not stated | [19:32](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1172s) | | Run turn and wait keyword | status not stated | [20:54](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1254s) | | Get expected data keyword for validation | status not stated | [22:03](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1323s) | | Error reason field for test failure logging | status not stated | [22:26](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1346s) | | Finalize turn keyword with history logging | status not stated | [22:59](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1379s) | | Log download with serialized task output | status not stated | [24:14](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1454s) | | Library agent public methods for customization | status not stated | [25:10](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1510s) | ## AL objects mentioned As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named. - other "animal data sets" at [4:12](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=252s) - other "library agent" at [4:30](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=270s) - other "Get eval suit setup data input" at [17:34](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1054s) - other "Run turn and wait" at [20:54](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1254s) - other "Get expected data" at [22:03](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1323s) - other "Finalize turn" at [22:59](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1379s) - other "Wait for the task to complete" at [25:49](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1549s) ## Quotes - [0:41](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=41s) "going to end up with one of the following it is either going to create a message for you to review it" - [1:37](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=97s) "ERP is quite busy and there is a lot of users and code that is changing the data, data might be changed between the" - [3:43](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=223s) "in our testing we discovered that it wasn't searching for the customer 10 000 it knew that" - [4:30](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=270s) "are going to be actual data-driven tests specified in YAML. The next component is library agent, which is going to help you to automate" - [10:48](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=648s) "Unfortunately, currently it's not possible to check on the message if the message of the intervention request as as you expecting it. But we" - [14:00](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=840s) "now you can replace the dates with placeholders like this. And this is going to calculate dynamically the work date based on the current" - [14:00](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=840s) "the problem is that in the tests, I would need to change this one quite often. And the solution that we did is that" - [14:13](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=853s) "this is going to calculate dynamically the work date based on the current work date and the formula that you provide here" - [16:25](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=985s) "you can test You can run the same tests on different agents, change their instructions, maybe change the model, and see how that the" - [18:28](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1108s) "we are going to default the other fields like posting groups units of measure automatically by using test libraries" - [22:26](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1346s) "you're not using the asserts to check the data. We are actually using the text field, which is called error reason." - [25:24](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1524s) "we have added the library agent. It has 27 public methods which will allow you to automate every single interaction that you can do" - [26:02](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1562s) "this one is quite good if you are going to invoke the agent through the action or AL code. So, you would like to" ## Disclaimers in the video - [10:48](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=648s) coming-later: it's not possible to check on the message if the message of the intervention request as as you expecting it. But we are planning that. - [23:51](https://www.youtube.com/watch?v=aNRzlbxVPWE&t=1431s) preview: this fact box was not visible. We have added it recently Presenters (as heard): Quentin.