Distil a small local model for delivery-note extraction

Define the narrow business task

The task is to extract supplier, order reference, delivery date, and item rows from synthetic delivery-note text. Missing values must remain missing. The experiment excludes OCR, production documents, and live ERP writes so the extraction and training behaviour can be examined separately.

A small specialist model may be useful for repeated bounded work, but adaptation adds data preparation, validation, training, and maintenance. The right question is whether it improves the accepted-task result over the original prompted student. The answer must come from evaluation.

Distillation and fine-tuning are different parts

This lab uses response distillation: a larger teacher generates task outputs, and those outputs become supervised examples for a smaller student. LoRA fine-tuning is the adaptation method. Because the base is quantised, this run is a parameter-efficient adaptation over a four-bit model.

It does not match teacher logits, transfer the teacher’s general reasoning capability, or train a foundation model from scratch. Fine-tuning only on human-provided labels would be adaptation without this teacher–student response-distillation step.

Use an explicit model manifest

The teacher is a quantised MLX conversion of DeepSeek-R1-Distill-Qwen-7B. The student is Qwen3-0.6B, also in four-bit MLX format. The downloadable kit includes repository IDs and exact snapshot revisions used for the experiment.

DeepSeek’s model card explicitly permits distillation, with the applicable upstream obligations for its Qwen-derived weights. Preserve licences and notices. Model access, conversion format, licence, and output-use terms are separate checks; do not assume that “open-weight” means every model has identical commercial conditions.

Validate the teacher against independent truth

The dataset generator starts from known records and renders them into disjoint training, validation, and test template families. Supplier patterns also differ across splits. Some documents omit the order reference or contain distracting instructions.

Teacher output must parse into the exact schema and equal the independent record before it enters training. Incorrect quantities, unsupported values, and invalid types are rejected rather than silently repaired. Only final JSON is retained as the training target, not reasoning text.

The held-out test labels come directly from the original records. They are not copied from teacher predictions. This avoids scoring the student merely for agreeing with a teacher that may be wrong.

Run a bounded local adaptation

Use the Delivery-Note Model Lab kit and its pinned Mac environment. The canonical path uses MLX on Apple silicon, a batch size of one, four adapted layers, a short sequence limit, prompt masking, gradient checkpointing, and a bounded iteration count.

python -m labs.delivery_model.pipeline generate
python -m labs.delivery_model.pipeline label
python -m labs.delivery_model.pipeline train --iterations 30
python -m labs.delivery_model.pipeline evaluate --name student-base
python -m labs.delivery_model.pipeline evaluate   --name student-adapted   --adapter labs/delivery_model/artifacts/adapters

Download the model snapshots using the supplied script before these steps. Model weights and intermediate artefacts remain outside Git and website downloads. The kit contains a separate fusion and local-serving recipe; it does not promise Qwen-to-Ollama GGUF conversion.

Compare the result without moving the goalposts

Evaluate the original student, adapted student, and teacher on the same held-out set. Record schema validity, exact-record match, four-top-level-field accuracy, unsupported fields, missing-reference handling, latency, and MLX peak allocation. Preserve individual failed predictions.

The measured experiment report is included with the lab download. Its environment and sample size are explicit. Mac Studio and NVIDIA-PC configurations remain reference setups, not measurements inferred from this MacBook run. A small synthetic experiment cannot establish production-document accuracy or a general hardware ranking.

An adapted model that fails to improve the baseline is still an informative result: retain the simpler model or improve the task/data before spending more. Training loss alone is not a business acceptance test.

Measured local result

The initial bounded run used an Apple M2 Pro with 16 GB unified memory. Of 32 training and eight validation documents, 21 and three respectively passed exact teacher-label validation. The student was adapted for 30 iterations. All three candidates were then evaluated on the same 12 held-out synthetic documents.

Candidate	Exact records	Schema-valid output	Mean task time	MLX allocation peak
teacher	4/12	41.7%	3.605 s	4.277 GiB
student-base	5/12	50.0%	0.378 s	0.643 GiB
student-adapted	9/12	83.3%	0.470 s	0.631 GiB

The adapted student matched more held-out records than its original baseline, but still failed three of 12. The teacher also failed several cases; it is not an oracle. These observations support further testing of this narrow configuration, not a production accuracy promise or a general model ranking.

Timings come from a single local run per candidate, including first-generation overhead. Peak memory is the MLX allocator measurement. No electricity, reviewer-time, or monetary saving was measured. The downloadable report preserves individual predictions and additional limitations.

Serve and maintain the adapted model

Fuse or load the adapter through MLX and expose it only within the intended local boundary. Keep the same prompt/template, schema validator, model revision, and evaluation set. Preserve the original student as a fallback.

Budget for future document formats, label review, regression tests, and model updates. A narrow adapted model can reduce inference requirements only when its quality and maintenance cost support that decision.

References

• DeepSeek model card and licence
• Qwen3-0.6B model card
• MLX LoRA, QLoRA, and fusion
• MLX-LM