Test Copilot capability in AL
Testing Copilot capabilities and agents in AL for Business Central: the Evaluation framework (formerly AI Test Toolkit), JSONL and YAML datasets, writing AI tests and agent tests, and best practices for non-determinism, safety, languages and model changes.
- 6Learn pages
- 1videos
Path: Integration with AI > Integrate AI using developer tools for Copilot > Extend Copilot in Business Central > Test Copilot capability in AL · tier official · system copilot · narrative reviewed (checked by Opus)
Overview
This section explains how to test LLM-based Copilot features and agents built in AL. Evaluation (formerly AI Test Toolkit) is the data-driven test automation framework. It organizes test suites, runs them across languages, and tracks Copilot credit consumption, including environment and company credit limits for evaluation runs.
The pages fit together in layers. The overview page introduces why to test (accuracy, consistency, safety) and the AI TEST TOOLKIT permission set. Datasets define input and expected output pairs in JSONL or YAML. “Write AI tests” covers prompt-based Copilot features that run dataset scenarios and compare results. “Write agent tests” covers end-to-end agent scenarios using the AI Test Suite and the Library - Agent codeunit. The best practices page covers strategy.
Start with the overview, then read Evaluation and Datasets for the framework and data format. Move to Write AI tests or Write agent tests depending on what you build, and use the best practices page to design realistic test cases.
Key points
- Evaluation is the new name for AI Test Toolkit; it is a data-driven framework for validating Copilot features and agents.
- The overview page explains why to test (accuracy, consistency, safety) and references the AI TEST TOOLKIT permission set.
- Datasets hold test inputs and expected outputs in JSONL or YAML, with support for suite-level setup, per-turn setup, intervention handling and date placeholders.
- AI tests in AL run dataset scenarios and compare outputs to expected results, with internal or external evaluation and test output export.
- Agent tests use the AI Test Suite and Library - Agent codeunit, with a turn loop, intervention declarations, suite-level setup and manual task management.
- Evaluation tracks Copilot credit consumption and supports environment-level and company-level credit limits for evaluations.
- Evaluation can be run across languages, and suites carry suite-level metadata.
- Best practices cover non-determinism, scaling with Evaluation, realistic test cases, safety, tone, bias and stereotype checks, cross-language compatibility and model version regression testing.
Learn pages
- Best Practices for Testing the Copilot Capability in AL: Learn about best practices for testing the Copilot capability in AL.
- Datasets: Learn how to create datasets for AI tests and agent tests in Business Central.
- Evaluation: Learn how to use Evaluation to test Copilot features and agents in AL.
- Test the Copilot Capability in AL: Explore testing strategies for Copilot features in AL. Ensure AI-generated outputs are accurate, safe, and user-friendly.
- Write agent tests: Learn how to write AL tests for Business Central agents using the Library - Agent helpers to verify end-to-end agent behavior.
- Write AI tests: Learn how to write AL tests for Copilot features using datasets, assertions, and external evaluation.
Videos and posts
Linked by a Haiku matcher with a grounding quote from the item’s summary (link/topics.ts); machine-generated.
- What’s New: Testing Copilot in Business Central (2024 release wave 2) (video): “Copilot Test Toolkit; Data-Driven Testing for Copilot; Accuracy Testing for Copilot”
Source: Microsoft Learn (CC BY 4.0). Descriptions are Learn’s own.