BC-Bench evaluation framework

A reproducible open-source framework evaluates AI coding agents on real AL tasks from real PRs and tests. It measures the impact of model, tool, instruction and agent changes.

Area: Developer tools. Status: GA. Developer relevance: high. Airtime: 3 min across 1 video.

Status evidence

• Nothing said about status; launch event convention: generally available unless stated otherwise.

Where they talk about it

• What's new in BC-Bench: 0:06 to 2:46

Quotes

• 1:13 "the general coding benchmarks like SWEBench don't uh reflect the AL development realities" - States the core reason BC-Bench exists as a separate benchmark.
• 1:35 "there's only 400 um that are written in AL. Well, there's 3.3 million in Python and 650,000 in C." - Gives the numbers showing how small the open-source AL ecosystem is.

Caveats mentioned

• Only 400 open-source repositories are written in AL, compared with 3.3 million in Python, so general benchmarks like SWE-Bench do not reflect AL work

Documented features match

• No documented item matched. BC-Bench not documented
• Product documentation: Development in AL, checked 2026-10-02. BC-Bench is a benchmarking framework for evaluating agent performance on real-world Business Central AL coding tasks. Note: Short dev-itpro section plus the release plan page 'Evaluate AL coding agents with BC-Bench' (https://learn.microsoft.com/dynamics365/release-plan/2026wave1/smb/dynamics365-business-central/evaluate-al-coding-agents-bc-bench), which documents 2026 release wave 1 (GA Apr 1, 2026), not wave 2.

Docs checked on 2026-10-02: the match is against Microsoft's what's new pages for the wave as they were on that date. Microsoft keeps filling the documentation, so "no documented item" can go stale.

Tags: testing, agents, benchmark, al

Generated by pipeline step 06. Source: YouTube auto-captions of the launch event videos, see the deep links. Not official.