Live demo with a CPU playground: chirak.fleetq.net. This page is a static snapshot of Run 5 (
merchant-triage-j1zjro), generated 15 Oct 2026 from its stored results. Runs: Run 6 · Run 5b · Run 5 · Run 4 · Run 7 (store reviews) · Run 7b (store reviews + real) · Run 8e (store reviews, best). Every number can be re-scored with make eval; see the run log.Merchant ticket triage (BG/EN), guidelines v2
not worth itTriage incoming customer-support tickets of an online shop. Public, read-only report built from the stored results.
Fine-tuning was skipped: baseline already meets target.
Baseline macro-f1
93.5%
Final macro-f1
—
Change
—
Baseline vs iterations
| Model | Accuracy | Macro-F1 | JSON validity | Exact match |
|---|---|---|---|---|
| Baseline (base model) | 93.5% | 93.5% | 100.0% | 69.8% |
Target: macro-f1 at least 92%. Stopped: baseline already meets target.
Per-class results, baseline
| Class | Support | Base P | Base R | Base F1 |
|---|---|---|---|---|
| orders | 31 | 97% | 94% | 95% |
| shipping | 31 | 100% | 94% | 97% |
| returns | 31 | 100% | 97% | 98% |
| payments | 31 | 89% | 77% | 83% |
| product_questions | 30 | 81% | 100% | 90% |
| complaints | 30 | 100% | 100% | 100% |
| other | 31 | 91% | 94% | 92% |
Confusion matrix (Baseline, rows = expected, columns = predicted)
| orders | shipping | returns | payments | product_qu... | complaints | other | __invalid_... | |
|---|---|---|---|---|---|---|---|---|
| orders | 29 | 0 | 0 | 0 | 2 | 0 | 0 | 0 |
| shipping | 0 | 29 | 0 | 0 | 2 | 0 | 0 | 0 |
| returns | 0 | 0 | 30 | 1 | 0 | 0 | 0 | 0 |
| payments | 1 | 0 | 0 | 24 | 3 | 0 | 3 | 0 |
| product_questions | 0 | 0 | 0 | 0 | 30 | 0 | 0 | 0 |
| complaints | 0 | 0 | 0 | 0 | 0 | 30 | 0 | 0 |
| other | 0 | 0 | 0 | 2 | 0 | 0 | 29 | 0 |
| __invalid__ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Leakage guard and frozen eval set
- Eval items
- 215 (200 synthetic + 15 seed)
- Frozen at
- 2026-10-14 10:26:50 UTC
- Blind label check
- 429 of 661 eval candidates dropped because the generator, labelling the text blind with the student's prompt, disagreed with the target label. The kept items are the unambiguous ones, so this eval is cleaner and easier than an unfiltered one.
- Eval SHA-256
- 78f3529dc3037330e68de05cca9ffccca2c59273391bde5f484fcb13ea1952b0
Decisions log
No decision yet. The first one comes after the first adapter is evaluated.
Training loss
No training has finished yet.