---
id: video/Hdp4KbbGpQA
type: video
title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
summary: BC-Bench is Microsoft's offline benchmark for measuring how coding agents perform on real AL bug-fixing tasks in Business Central. The episode covers its method, the 66-task dataset, early results for the AL2 MCP server, model ranking (Claude Opus 4.5 best) and planned categories.
tier: official
language: en
tags:
  - bc bench
  - al benchmarking
  - coding agents
  - ai evaluation
  - mcp servers
  - bug fixing automation
  - developer experience
  - vs code extension
  - github copilot
  - al2 mcp server
  - tool evaluation
  - software engineering tasks
system: development
review:
  state: reviewed
  by: opus
  at: "2026-10-07T22:48:27.770Z"
  flags: []
generated:
  at: "2026-10-13T00:32:41.360Z"
  pipeline: 0.2.0
  prompts:
    extract-video: 1
    summarize-video: 2
  input_hash: d9f9d81075f3ea4386dd696cd8054ff966293ad74a60ff990fcc7171134e9f10
evidence:
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=41s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 41
    quote: BC Bench is a offline evaluation framework that measures how coding agents performs on real world engineering tasks in Business Central or AL programming
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=70s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 70
    quote: if a model perform better than others in AL tasks or if we can provide custom instructions or different tooling to improve the coding
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=109s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 109
    quote: this is about measuring um how good the deep the energetic experience is when you do AL development
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=264s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 264
    quote: what they did is they go to GitHub and find some popular Python repositories. Uh they find previous pull requests done by human engineers
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=553s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 553
    quote: last time I checked is 66 tasks in the data set. Well obviously the number is growing every day
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=588s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 588
    quote: one of the criteria is it's a pull request must be resolving exactly one bug. So so we know this PR is fixing this
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=796s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 796
    quote: we are currently focusing on tool that built by our developer experience team that's called AL2 MCP server
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=841s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 841
    quote: having BC bench because we'll know that we we we are going to invest into adding more tools to to the Visual Studio code
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=881s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 881
    quote: the short answer is not conclusively. We we run them we actually did a rigorous analysis on on this AL2 MCP server. We run
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=892s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 892
    quote: We did see an increase on the average accuracy or what we call resolution rate. But there is a overlap on the confidence interval.
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=931s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 931
    quote: We provided some feedback to the developer experience team and now we have a new version of AL2 MCP server.
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=968s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 968
    quote: If we figure out this tool doesn't make any difference then we don't need to support it anymore and then we can focus our
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1076s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1076
    quote: No, it is a well, it's similar to GitHub copilot in Visual Studio code but designed for to be running in terminal and in
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1260s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1260
    quote: the next on the list is to run tests uh, through the MCP server to ask to allow coding agent to iterate on the
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1619s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1619
    quote: Cloud Opus 45 is the best model we have access so far and it outperforms than any other models we have access to through
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1665s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1665
    quote: the ranking we see for for PC bench is more or less the same, isn't it? Yes. Yes. And that's So that's that's comforting
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1665s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1665
    quote: the the ranking we see for for PC bench is more or less the same, isn't it?
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1697s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1697
    quote: there is more or less you know, equivalence between you know, handling writing code in Python and writing code in AL
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1799s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1799
    quote: we aim to make BC Bench public. Um well, first of all a bench benchmark is not uh useful unless people can reproduce the
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1831s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1831
    quote: Um now with BC Bench they can verify if it helps in solving what bugs in AL.
  - kind: video
    url: https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1873s
    title: "Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks"
    date: "2026-05-07T14:00:23.000Z"
    commit: null
    t: 1873
    quote: where partners can simply plug in this own uh their own code base in in
links:
  learn: []
  objects: []
  features: []
  topics: []
  localizations: []
  videos: []
  posts: []
  guidelines: []
video_id: Hdp4KbbGpQA
channel: yt-microsoft
source_name: Microsoft Dynamics 365 Business Central (YouTube)
url: https://www.youtube.com/watch?v=Hdp4KbbGpQA
published_at: "2026-05-07T14:00:23.000Z"
duration_s: 1918
captions: full
audience:
  - developer
  - partner
  - decision maker
chapters:
  - t: 0
    title: Introduction to BC Bench and AI benchmarking
  - t: 130
    title: Sweep Bench methodology and open-source inspiration
  - t: 248
    title: "How Sweep Bench works: GitHub pull requests and unit tests"
  - t: 328
    title: Adapting Sweep Bench methodology for AL and Azure DevOps
  - t: 424
    title: "BC Bench methodology: recreating bug scenarios from history"
  - t: 541
    title: "Dataset criteria: single bugs with full context"
  - t: 680
    title: Test independence and fairness in benchmarking
  - t: 796
    title: Tools and MCP servers for measuring agent performance
  - t: 943
    title: Scientific validation, agent harness variations, and prioritization
  - t: 1091
    title: Custom instructions, skills, and future test generation
  - t: 1260
    title: Agent feedback loops, test execution, and BC Bench roadmap categories
  - t: 1509
    title: BC-Bench results, model ranking, and consistency with SWE Bench
  - t: 1748
    title: AL versus Python parity, public release plans, and partner dataset roadmap
features:
  - name: BC Bench
    status: unclear
    t: 11
    verified: false
    status_source: video
  - name: AL2 MCP Server
    status: unclear
    t: 796
    verified: false
    status_source: video
  - name: Mini BC Agent
    status: unclear
    t: 1025
    verified: false
    status_source: video
  - name: GitHub Copilot Agent Terminal Variant
    status: unclear
    t: 1062
    verified: false
    status_source: video
  - name: Test Generation Category
    status: unclear
    t: 1214
    verified: false
    status_source: video
  - name: Code Review Category
    status: unclear
    t: 1226
    verified: false
    status_source: video
  - name: Test Execution via MCP
    status: unclear
    t: 1260
    verified: false
    status_source: video
  - name: Code Refactoring Category
    status: unclear
    t: 1509
    verified: false
    status_source: video
  - name: Feature Addition Category
    status: unclear
    t: 1509
    verified: false
    status_source: video
  - name: Claude Opus 4.5 Performance
    status: unclear
    t: 1619
    verified: false
    status_source: video
  - name: AL code parity with Python
    status: unclear
    t: 1697
    verified: false
    status_source: video
  - name: Public BC-Bench dataset
    status: unclear
    t: 1799
    verified: false
    status_source: video
  - name: Partner dataset for BC-Bench
    status: unclear
    t: 1873
    verified: false
    status_source: video
objects_mentioned:
  - other AL2 MCP Server
  - other BC Bench
  - other mini BC agent
quotes:
  - t: 41
    text: BC Bench is a offline evaluation framework that measures how coding agents performs on real world engineering tasks in Business Central or AL programming
    check: exact
  - t: 70
    text: if a model perform better than others in AL tasks or if we can provide custom instructions or different tooling to improve the coding
    check: exact
  - t: 109
    text: this is about measuring um how good the deep the energetic experience is when you do AL development
    check: exact
  - t: 264
    text: what they did is they go to GitHub and find some popular Python repositories. Uh they find previous pull requests done by human engineers
    check: exact
  - t: 553
    text: last time I checked is 66 tasks in the data set. Well obviously the number is growing every day
    check: exact
  - t: 588
    text: one of the criteria is it's a pull request must be resolving exactly one bug. So so we know this PR is fixing this
    check: exact
  - t: 796
    text: we are currently focusing on tool that built by our developer experience team that's called AL2 MCP server
    check: exact
  - t: 841
    text: having BC bench because we'll know that we we we are going to invest into adding more tools to to the Visual Studio code
    check: exact
  - t: 881
    text: the short answer is not conclusively. We we run them we actually did a rigorous analysis on on this AL2 MCP server. We run
    check: exact
  - t: 892
    text: We did see an increase on the average accuracy or what we call resolution rate. But there is a overlap on the confidence interval.
    check: exact
  - t: 931
    text: We provided some feedback to the developer experience team and now we have a new version of AL2 MCP server.
    check: exact
  - t: 968
    text: If we figure out this tool doesn't make any difference then we don't need to support it anymore and then we can focus our
    check: exact
  - t: 1076
    text: No, it is a well, it's similar to GitHub copilot in Visual Studio code but designed for to be running in terminal and in
    check: exact
  - t: 1260
    text: the next on the list is to run tests uh, through the MCP server to ask to allow coding agent to iterate on the
    check: exact
  - t: 1619
    text: Cloud Opus 45 is the best model we have access so far and it outperforms than any other models we have access to through
    check: exact
  - t: 1665
    text: the ranking we see for for PC bench is more or less the same, isn't it? Yes. Yes. And that's So that's that's comforting
    check: exact
  - t: 1665
    text: the the ranking we see for for PC bench is more or less the same, isn't it?
    check: exact
  - t: 1697
    text: there is more or less you know, equivalence between you know, handling writing code in Python and writing code in AL
    check: exact
  - t: 1799
    text: we aim to make BC Bench public. Um well, first of all a bench benchmark is not uh useful unless people can reproduce the
    check: exact
  - t: 1831
    text: Um now with BC Bench they can verify if it helps in solving what bugs in AL.
    check: exact
  - t: 1873
    text: where partners can simply plug in this own uh their own code base in in
    check: fuzzy
---

# Business Central Under the Hood episode 15: BC-Bench: How we evaluate AI on AL tasks

> BC-Bench is Microsoft's offline benchmark for measuring how coding agents perform on real AL bug-fixing tasks in Business Central. The episode covers its method, the 66-task dataset, early results for the AL2 MCP server, model ranking (Claude Opus 4.5 best) and planned categories.

[Watch on YouTube](https://www.youtube.com/watch?v=Hdp4KbbGpQA) · Microsoft Dynamics 365 Business Central (YouTube) · 2026-05-07 · 31:58 · tier official · reviewed (checked by Opus)

## Overview

Episode 15 of Business Central Under the Hood explains how BC-Bench evaluates AI on AL tasks. The method is inspired by Sweep Bench: bugs are collected from internal repositories (Azure DevOps rather than GitHub), the code is restored to the state when the bug occurred, an agent tries to fix it, and the original unit tests validate the result. A task qualifies only if it resolves exactly one bug, has full context, and its tests are independent of the fix.

The team uses the benchmark to compare models and to decide whether tooling such as the AL2 MCP server, built by the developer experience team, is worth supporting. The speakers report an increase in resolution rate with the MCP server, but the confidence intervals overlap, so the effect is not conclusive. They also describe plans for more categories (test generation, code review, refactoring, possibly feature addition), running tests through the MCP server, a public release and a partner dataset.

## Key points

- BC-Bench is an offline evaluation framework for coding agents on real AL engineering tasks. It is an internal developer tool, not a product feature. It currently covers bug fixing only, with 66 tasks at the time of recording and a dataset that keeps growing.
- Method (adapted from SWE Bench): take a historical bug, recreate the code state at that time from a base commit, let the agent fix it, and validate with the original unit tests. Tests run through AL-Go and BC Container Helper.
- Task criteria: a pull request must resolve exactly one bug, the bug must include full context, and the tests must not depend on the fix (for example, exact error messages).
- The AL2 MCP server brings AL language extension tools such as compile and symbol finding to agents. It showed a higher average resolution rate, but the confidence intervals overlap, so the improvement is not conclusive. A new version has been built based on BC-Bench feedback.
- Comparisons used a mini BC agent (about 200 lines of Python, with only PowerShell and Microsoft Foundry endpoint access) and a terminal/pipeline variant of the GitHub Copilot agent. Custom instructions and skills can also be varied.
- Claude Opus 4.5 is the best-performing model among those available through GitHub Copilot. The ranking is close to SWE Bench Python results, and AL performance is roughly equal to Python, even though there are over 2 million public Python repositories and only about 300 AL ones.
- Planned items: test generation, code review and refactoring categories, possibly feature addition, running tests via the MCP server, a public BC-Bench and a partner dataset. No timelines were given.

## Chapters

- [0:00](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=0s) Introduction to BC Bench and AI benchmarking
- [2:10](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=130s) Sweep Bench methodology and open-source inspiration
- [4:08](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=248s) How Sweep Bench works: GitHub pull requests and unit tests
- [5:28](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=328s) Adapting Sweep Bench methodology for AL and Azure DevOps
- [7:04](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=424s) BC Bench methodology: recreating bug scenarios from history
- [9:01](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=541s) Dataset criteria: single bugs with full context
- [11:20](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=680s) Test independence and fairness in benchmarking
- [13:16](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=796s) Tools and MCP servers for measuring agent performance
- [15:43](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=943s) Scientific validation, agent harness variations, and prioritization
- [18:11](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1091s) Custom instructions, skills, and future test generation
- [21:00](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1260s) Agent feedback loops, test execution, and BC Bench roadmap categories
- [25:09](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1509s) BC-Bench results, model ranking, and consistency with SWE Bench
- [29:08](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1748s) AL versus Python parity, public release plans, and partner dataset roadmap

## Features

Every feature below: status not stated.

| Feature | At |
|---|---|
| BC Bench | [0:11](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=11s) |
| AL2 MCP Server | [13:16](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=796s) |
| Mini BC Agent | [17:05](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1025s) |
| GitHub Copilot Agent Terminal Variant | [17:42](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1062s) |
| Test Generation Category | [20:14](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1214s) |
| Code Review Category | [20:26](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1226s) |
| Test Execution via MCP | [21:00](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1260s) |
| Code Refactoring Category | [25:09](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1509s) |
| Feature Addition Category | [25:09](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1509s) |
| Claude Opus 4.5 Performance | [26:59](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1619s) |
| AL code parity with Python | [28:17](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1697s) |
| Public BC-Bench dataset | [29:59](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1799s) |
| Partner dataset for BC-Bench | [31:13](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1873s) |

## AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

- other "AL2 MCP Server" at [13:33](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=813s)
- other "BC Bench" at [14:01](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=841s)
- other "mini BC agent" at [17:05](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1025s)

## Quotes

- [0:41](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=41s) "BC Bench is a offline evaluation framework that measures how coding agents performs on real world engineering tasks in Business Central or AL programming"
- [1:10](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=70s) "if a model perform better than others in AL tasks or if we can provide custom instructions or different tooling to improve the coding"
- [1:49](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=109s) "this is about measuring um how good the deep the energetic experience is when you do AL development"
- [4:24](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=264s) "what they did is they go to GitHub and find some popular Python repositories. Uh they find previous pull requests done by human engineers"
- [9:13](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=553s) "last time I checked is 66 tasks in the data set. Well obviously the number is growing every day"
- [9:48](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=588s) "one of the criteria is it's a pull request must be resolving exactly one bug. So so we know this PR is fixing this"
- [13:16](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=796s) "we are currently focusing on tool that built by our developer experience team that's called AL2 MCP server"
- [14:01](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=841s) "having BC bench because we'll know that we we we are going to invest into adding more tools to to the Visual Studio code"
- [14:41](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=881s) "the short answer is not conclusively. We we run them we actually did a rigorous analysis on on this AL2 MCP server. We run"
- [14:52](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=892s) "We did see an increase on the average accuracy or what we call resolution rate. But there is a overlap on the confidence interval."
- [15:31](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=931s) "We provided some feedback to the developer experience team and now we have a new version of AL2 MCP server."
- [16:08](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=968s) "If we figure out this tool doesn't make any difference then we don't need to support it anymore and then we can focus our"
- [17:56](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1076s) "No, it is a well, it's similar to GitHub copilot in Visual Studio code but designed for to be running in terminal and in"
- [21:00](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1260s) "the next on the list is to run tests uh, through the MCP server to ask to allow coding agent to iterate on the"
- [26:59](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1619s) "Cloud Opus 45 is the best model we have access so far and it outperforms than any other models we have access to through"
- [27:45](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1665s) "the ranking we see for for PC bench is more or less the same, isn't it? Yes. Yes. And that's So that's that's comforting"
- [27:45](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1665s) "the the ranking we see for for PC bench is more or less the same, isn't it?"
- [28:17](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1697s) "there is more or less you know, equivalence between you know, handling writing code in Python and writing code in AL"
- [29:59](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1799s) "we aim to make BC Bench public. Um well, first of all a bench benchmark is not uh useful unless people can reproduce the"
- [30:31](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1831s) "Um now with BC Bench they can verify if it helps in solving what bugs in AL."
- [31:13](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1873s) "where partners can simply plug in this own uh their own code base in in"

## Disclaimers in the video

- [6:51](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=411s) coming-later: So one day hopefully we'll make it public
- [14:41](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=881s) subject-to-change: We did see an increase on the average accuracy or what we call resolution rate but not conclusively
- [20:14](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1214s) coming-later: hopefully we have much more work as a software engineer. We want to bring test generation
- [20:26](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1226s) coming-later: we also want to build a a category called code review
- [21:00](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1260s) coming-later: the next on the list is to run tests uh, through the MCP server
- [25:09](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1509s) coming-later: we also have code refactoring on the agenda and potentially feature addition
- [26:19](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1579s) coming-later: that's a bit uh, probably in the future. We haven't really thought about that yet
- [26:47](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1607s) coming-later: agents looking at pages is more in the future
- [29:59](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1799s) coming-later: we aim to make BC Bench public
- [31:13](https://www.youtube.com/watch?v=Hdp4KbbGpQA&t=1873s) coming-later: create a partner data set for BC-Bench is part of the plan

Presenters (as heard): Håkon, Erhard.
