---
id: video/RfHVHa_Qiqk
type: video
title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
summary: Small language models and fine-tuning for Business Central AI features, shown with the sales order agent. Compares GPT-4, GPT-4 mini and a fine-tuned GPT-4 mini on accuracy and cost, using preliminary results from test cases.
tier: official
language: en
tags:
  - small language models
  - fine-tuning
  - model distillation
  - phi models
  - parameters
  - cost optimization
  - supervised fine-tuning
  - prompt engineering
  - ai testing
  - sales order agent
  - accuracy measurement
  - data generation
system: copilot
review:
  state: reviewed
  by: opus
  at: "2026-10-07T23:02:03.312Z"
  flags: []
generated:
  at: "2026-10-07T23:02:03.360Z"
  pipeline: 0.2.0
  prompts:
    extract-video: 1
    summarize-video: 2
  input_hash: 09368b29f48c5c828ebbee938dc795fe98a20f1919bee523d04e90c83403d1c9
evidence:
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1173s
    title: "Sales order agent: preview"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1173
    quote: it's a feature which is in preview uh you we actually generate we have the LM generating a reply to the customer
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=50s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 50
    quote: i started working on ai and machine learning in 2005 so it's 20 years ago so before
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=293s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 293
    quote: um this is 4 years ago that that had 175 billion parameters
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=455s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 455
    quote: was released um this december with i think 14 billion parameters and that uh outperforms it clearly outperforms something like
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=496s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 496
    quote: these 14 billion that that's small enough that you can run it efficiently on a single GPU so it means that it's so you
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=637s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 637
    quote: we can do the same as open AI does on this pre-trained LM and we can also do it in a more efficient way
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=738s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 738
    quote: you would do that only if you find out that the model is not doing a good job to start with
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=819s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 819
    quote: you take a big model and uh which performs for your particular application well yes and you take the output of that and then
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=859s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 859
    quote: the difference between in pricing of dg4 for example in tpg4 mini is is a factor 10 15
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=900s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 900
    quote: you can deploy them also from AI Foundry and then you can consume them just like any other model
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1019s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1019
    quote: for um the sales order agent that we are releasing now we have a measure of accuracy that is um you know given an
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1229s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1229
    quote: we also have support in the AI test to kit for using an llm to to test features
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1331s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1331
    quote: we have test cases so we have hundreds of test cases that run mhm and if these test cases pass that's actually a good
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1473s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1473
    quote: uh um here on the left we ran through 10 emails and in approximately 85% of the cases gbgo did the right thing so
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1513s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1513
    quote: tt4 mini achieves around 30% accuracy so that's a quite a significant drop and so useless
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1533s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1533
    quote: if we do that we get more than 90% accuracy on on these execute task steps which is pretty good and in this case
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1614s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1614
    quote: risk that when we have been training with this data then small the small language model that get fine tune is very good at
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1654s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1654
    quote: gbgo uh achieved around 85 85% accuracy but it cost um around 22 cents to run these these 10 emails in this case whereas
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1675s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1675
    quote: dg4 mini then the fun version then hits kind of a sweet spot because it's it's a little bit more expensive than dg4 mini
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1715s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1715
    quote: the first thing to do is always prom prompt engineering you know few shot prompting Chain of Thought and and all those things
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1735s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1735
    quote: if there is a feature uh that uses gg4 um that has high accuracy but is expensive yeah then that's that's and you have
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1775s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1775
    quote: if you have a scenario where you need speed MH then it's also Factor 10 or so faster so yeah so from from you
  - kind: video
    url: https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1836s
    title: "Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning"
    date: "2025-02-18T16:09:42.000Z"
    commit: null
    t: 1836
    quote: you have to you know have good data you have to make sure that it's representative that it's accurate you have to make sure
links:
  learn: []
  objects: []
  features: []
  topics: []
  localizations: []
  videos: []
  posts: []
  guidelines: []
video_id: RfHVHa_Qiqk
channel: yt-microsoft
source_name: Microsoft Dynamics 365 Business Central (YouTube)
url: https://www.youtube.com/watch?v=RfHVHa_Qiqk
published_at: "2025-02-18T16:09:42.000Z"
duration_s: 1875
captions: full
audience:
  - developer
  - partner
  - functional consultant
  - decision maker
chapters:
  - t: 0
    title: Introduction and AI Journey
  - t: 110
    title: "Claus's Background: From Business Central to Power BI"
  - t: 232
    title: "Large Language Models: Size, Parameters, and Cost"
  - t: 355
    title: Small Language Models and Phi Series
  - t: 476
    title: SLM Capabilities and Home Deployment
  - t: 557
    title: "Fine-tuning: Pre-training, Supervised Fine-tuning, and Specialization"
  - t: 697
    title: "Fine-tuning Use Cases: Specialization and Model Distillation"
  - t: 819
    title: Cost Savings and Small Language Models in Business Central
  - t: 960
    title: Testing AI Features and Non-determinism
  - t: 1099
    title: Data Generation and Seed Data for Testing
  - t: 1173
    title: Testing the Sales Order Agent Output
  - t: 1290
    title: Fine-tuning with Test Case Data
  - t: 1391
    title: "Accuracy Comparison: GPT-4 vs GPT-4 Mini"
  - t: 1513
    title: Fine-tuned GPT-4 Mini Results and Cost Analysis
  - t: 1675
    title: Advice for Partners on Small Language Models
  - t: 1755
    title: Speed Advantages and Best Practices for Fine-tuning
features:
  - name: Small Language Models
    status: unclear
    t: 232
    verified: false
    status_source: video
  - name: Fine-tuning for Business Central
    status: unclear
    t: 536
    verified: false
    status_source: video
  - name: Model Distillation
    status: unclear
    t: 778
    verified: false
    status_source: video
  - name: Phi Model Series
    status: unclear
    t: 435
    verified: false
    status_source: video
  - name: AI test toolkit
    status: unclear
    t: 960
    verified: false
    status_source: video
  - name: Data generation for AI testing
    status: unclear
    t: 1099
    verified: false
    status_source: video
  - name: Sales order agent
    status: preview
    t: 1173
    verified: true
    status_source: video
  - name: Execute task steps for agent control
    status: unclear
    t: 1371
    verified: false
    status_source: video
  - name: GPT-4 Mini fine-tuning optimization
    status: unclear
    t: 1290
    verified: false
    status_source: video
  - name: Cost-accuracy tradeoff analysis
    status: unclear
    t: 1654
    verified: false
    status_source: video
  - name: Prompt engineering for cost reduction
    status: unclear
    t: 1715
    verified: false
    status_source: video
  - name: Data quality requirements for fine-tuning
    status: unclear
    t: 1816
    verified: false
    status_source: video
objects_mentioned:
  - other Phi models
  - other AI Foundry
  - other AI toolkit
  - other AI test toolkit
quotes:
  - t: 50
    text: i started working on ai and machine learning in 2005 so it's 20 years ago so before
    check: fuzzy
  - t: 293
    text: um this is 4 years ago that that had 175 billion parameters
    check: fuzzy
  - t: 455
    text: was released um this december with i think 14 billion parameters and that uh outperforms it clearly outperforms something like
    check: fuzzy
  - t: 496
    text: these 14 billion that that's small enough that you can run it efficiently on a single GPU so it means that it's so you
    check: snapped
  - t: 637
    text: we can do the same as open AI does on this pre-trained LM and we can also do it in a more efficient way
    check: snapped
  - t: 738
    text: you would do that only if you find out that the model is not doing a good job to start with
    check: exact
  - t: 819
    text: you take a big model and uh which performs for your particular application well yes and you take the output of that and then
    check: exact
  - t: 859
    text: the difference between in pricing of dg4 for example in tpg4 mini is is a factor 10 15
    check: exact
  - t: 900
    text: you can deploy them also from AI Foundry and then you can consume them just like any other model
    check: exact
  - t: 1019
    text: for um the sales order agent that we are releasing now we have a measure of accuracy that is um you know given an
    check: exact
  - t: 1229
    text: we also have support in the AI test to kit for using an llm to to test features
    check: exact
  - t: 1331
    text: we have test cases so we have hundreds of test cases that run mhm and if these test cases pass that's actually a good
    check: exact
  - t: 1473
    text: uh um here on the left we ran through 10 emails and in approximately 85% of the cases gbgo did the right thing so
    check: exact
  - t: 1513
    text: tt4 mini achieves around 30% accuracy so that's a quite a significant drop and so useless
    check: exact
  - t: 1533
    text: if we do that we get more than 90% accuracy on on these execute task steps which is pretty good and in this case
    check: exact
  - t: 1614
    text: risk that when we have been training with this data then small the small language model that get fine tune is very good at
    check: exact
  - t: 1654
    text: gbgo uh achieved around 85 85% accuracy but it cost um around 22 cents to run these these 10 emails in this case whereas
    check: exact
  - t: 1675
    text: dg4 mini then the fun version then hits kind of a sweet spot because it's it's a little bit more expensive than dg4 mini
    check: exact
  - t: 1715
    text: the first thing to do is always prom prompt engineering you know few shot prompting Chain of Thought and and all those things
    check: exact
  - t: 1735
    text: if there is a feature uh that uses gg4 um that has high accuracy but is expensive yeah then that's that's and you have
    check: exact
  - t: 1775
    text: if you have a scenario where you need speed MH then it's also Factor 10 or so faster so yeah so from from you
    check: snapped
  - t: 1836
    text: you have to you know have good data you have to make sure that it's representative that it's accurate you have to make sure
    check: snapped
---

# Business Central Under the Hood episode 8: AI Small Language Models and Fine-tuning

> Small language models and fine-tuning for Business Central AI features, shown with the sales order agent. Compares GPT-4, GPT-4 mini and a fine-tuned GPT-4 mini on accuracy and cost, using preliminary results from test cases.

[Watch on YouTube](https://www.youtube.com/watch?v=RfHVHa_Qiqk) · Microsoft Dynamics 365 Business Central (YouTube) · 2025-02-18 · 31:15 · tier official · reviewed (checked by Opus)

## Overview

Episode 8 of Business Central Under the Hood explains what small language models are, how the Phi series relates to larger models, and how fine-tuning and model distillation can lower the cost of AI features. It then applies this to the sales order agent, which is in preview.

The demo covers generating test data from seed examples, running test cases with the AI test toolkit, and using successful runs as fine-tuning data for GPT-4 mini. Results are preliminary and the speaker warns about overfitting to the toy data. The closing advice for partners is to try prompt engineering first and to treat data quality as the main work.

## Key points

- Small language models have fewer parameters (billions). Phi 4 has 14 billion, runs on a single GPU, and is described as outperforming GPT-3. They are weaker at reasoning and world knowledge, and are about 10 times faster.
- Model distillation uses a larger model such as GPT-4 as teacher to train a smaller one such as GPT-4 mini. The price difference between them is a factor of 10-15. The larger model must already work well on the use case.
- OpenAI models such as GPT-4o and GPT-4 mini can be fine-tuned in Azure AI Foundry, deployed from there, and consumed in Business Central through the AI toolkit like any other model.
- Sales order agent test on 10 emails: GPT-4 reached about 85% accuracy at about 22 cents, and GPT-4 mini reached about 30% at about 1 cent. Fine-tuned GPT-4 mini reached over 90% on execute task steps at about twice the cost of plain GPT-4 mini.
- Fine-tuning data came from successful test case runs, taking input-output pairs from the execute task steps. Hundreds of test cases measure accuracy. The AI test toolkit runs data-driven AL tests and also supports LLM-based checks of natural-language output such as reply emails.
- The fine-tuned results are preliminary and trained on a limited data set. The model may overfit to the toy customer and item names and not generalize to other customers' master data.
- Try prompt engineering first (few-shot prompting, Chain of Thought). Fine-tune only if the model does not do a good job. Training data must be representative, accurate and kept separate from test data.

## Chapters

- [0:00](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=0s) Introduction and AI Journey
- [1:50](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=110s) Claus's Background: From Business Central to Power BI
- [3:52](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=232s) Large Language Models: Size, Parameters, and Cost
- [5:55](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=355s) Small Language Models and Phi Series
- [7:56](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=476s) SLM Capabilities and Home Deployment
- [9:17](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=557s) Fine-tuning: Pre-training, Supervised Fine-tuning, and Specialization
- [11:37](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=697s) Fine-tuning Use Cases: Specialization and Model Distillation
- [13:39](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=819s) Cost Savings and Small Language Models in Business Central
- [16:00](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=960s) Testing AI Features and Non-determinism
- [18:19](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1099s) Data Generation and Seed Data for Testing
- [19:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1173s) Testing the Sales Order Agent Output
- [21:30](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1290s) Fine-tuning with Test Case Data
- [23:11](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1391s) Accuracy Comparison: GPT-4 vs GPT-4 Mini
- [25:13](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1513s) Fine-tuned GPT-4 Mini Results and Cost Analysis
- [27:55](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1675s) Advice for Partners on Small Language Models
- [29:15](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1755s) Speed Advantages and Best Practices for Fine-tuning

## Features

| Feature | Status | At | Evidence |
|---|---|---|---|
| Small Language Models | status not stated | [3:52](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=232s) |  |
| Fine-tuning for Business Central | status not stated, demoed | [8:56](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=536s) |  |
| Model Distillation | status not stated | [12:58](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=778s) |  |
| Phi Model Series | status not stated | [7:15](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=435s) |  |
| AI test toolkit | status not stated, demoed | [16:00](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=960s) |  |
| Data generation for AI testing | status not stated, demoed | [18:19](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1099s) |  |
| Sales order agent | preview, demoed | [19:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1173s) | "it's a feature which is in preview uh you we actually generate we have the LM generating a reply to the customer" ([19:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1173s)) |
| Execute task steps for agent control | status not stated, demoed | [22:51](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1371s) |  |
| GPT-4 Mini fine-tuning optimization | status not stated, demoed | [21:30](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1290s) |  |
| Cost-accuracy tradeoff analysis | status not stated, demoed | [27:34](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1654s) |  |
| Prompt engineering for cost reduction | status not stated | [28:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1715s) |  |
| Data quality requirements for fine-tuning | status not stated | [30:16](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1816s) |  |

## AL objects mentioned

As heard in the captions. A name that matches one object page by exact type and name links to it; the others stay as named.

- other "Phi models" at [7:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=455s)
- other "AI Foundry" at [15:00](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=900s)
- other "AI toolkit" at [15:20](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=920s)
- other "AI test toolkit" at [17:39](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1059s)

## Quotes

- [0:50](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=50s) "i started working on ai and machine learning in 2005 so it's 20 years ago so before"
- [4:53](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=293s) "um this is 4 years ago that that had 175 billion parameters"
- [7:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=455s) "was released um this december with i think 14 billion parameters and that uh outperforms it clearly outperforms something like"
- [8:16](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=496s) "these 14 billion that that's small enough that you can run it efficiently on a single GPU so it means that it's so you"
- [10:37](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=637s) "we can do the same as open AI does on this pre-trained LM and we can also do it in a more efficient way"
- [12:18](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=738s) "you would do that only if you find out that the model is not doing a good job to start with"
- [13:39](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=819s) "you take a big model and uh which performs for your particular application well yes and you take the output of that and then"
- [14:19](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=859s) "the difference between in pricing of dg4 for example in tpg4 mini is is a factor 10 15"
- [15:00](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=900s) "you can deploy them also from AI Foundry and then you can consume them just like any other model"
- [16:59](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1019s) "for um the sales order agent that we are releasing now we have a measure of accuracy that is um you know given an"
- [20:29](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1229s) "we also have support in the AI test to kit for using an llm to to test features"
- [22:11](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1331s) "we have test cases so we have hundreds of test cases that run mhm and if these test cases pass that's actually a good"
- [24:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1473s) "uh um here on the left we ran through 10 emails and in approximately 85% of the cases gbgo did the right thing so"
- [25:13](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1513s) "tt4 mini achieves around 30% accuracy so that's a quite a significant drop and so useless"
- [25:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1533s) "if we do that we get more than 90% accuracy on on these execute task steps which is pretty good and in this case"
- [26:54](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1614s) "risk that when we have been training with this data then small the small language model that get fine tune is very good at"
- [27:34](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1654s) "gbgo uh achieved around 85 85% accuracy but it cost um around 22 cents to run these these 10 emails in this case whereas"
- [27:55](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1675s) "dg4 mini then the fun version then hits kind of a sweet spot because it's it's a little bit more expensive than dg4 mini"
- [28:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1715s) "the first thing to do is always prom prompt engineering you know few shot prompting Chain of Thought and and all those things"
- [28:55](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1735s) "if there is a feature uh that uses gg4 um that has high accuracy but is expensive yeah then that's that's and you have"
- [29:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1775s) "if you have a scenario where you need speed MH then it's also Factor 10 or so faster so yeah so from from you"
- [30:36](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1836s) "you have to you know have good data you have to make sure that it's representative that it's accurate you have to make sure"

## Disclaimers in the video

- [7:35](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=455s) other: phi4 was released this December with I think 14 billion parameters
- [19:33](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1173s) preview: it's a feature which is in preview
- [25:53](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1553s) subject-to-change: these are preliminary uh results um so I think in order to fully you know bake this and and and ship it we we need to train this dt4 mini on more data
- [26:13](https://www.youtube.com/watch?v=RfHVHa_Qiqk&t=1573s) subject-to-change: there's a risk that because we have these data so it's you know we have toys and and and so the toys have certain names and that's the items we have and the customers have certain names so there's a risk that because we have been training with this data then small the small language model

Presenters (as heard): Claus, Claus Marius Hansen.