Can a 494-million-parameter model become a genuine expert in one narrow domain — and can we prove it learned the skill from us, not from the internet?
The AI models direction at SmartInfer Research asks a simple-sounding question with big practical consequences: can we take a small AI model — small enough to run on a laptop or a phone — and make it a genuine expert in one narrow domain?
Frontier models are astonishing generalists, but they live in data centers, cost real money per call, and require sending your data to someone else's servers. Many real jobs don't need a generalist. A company with an internal query language, a hospital system that normalizes medication records, an engineering team with its own configuration format — each needs deep skill in one narrow area, available privately, cheaply, and on its own hardware. Our bet is that the right small model, taught the right way, beats a giant model rented over an API for jobs like these.
There's a classic problem with testing whether you've taught a model anything: big models have read most of the internet. If we train a small model on, say, algebra and it improves, a skeptic can always argue the skill was half-there already. So we removed that possibility by construction.
We had a powerful model invent a brand-new tiny programming language — we call it velk — with made-up keywords and deliberately strange semantics that no existing language uses. Velk was invented and frozen on July 5, 2026, so it cannot appear in any training corpus assembled before that date. A taste:
zon 6 # set the accumulator: a = 6 gorp 5 # parity-steered: a even, so a = 6 + 5 = 11 murn 4 # reflect about 4: a = 2·4 − 11 = −3 skiv # twisted swap: (a, b) = (b, −a) = (0, 3) plif # a = a + b = 3 → output: #### 3
The student is Qwen2.5-0.5B-Instruct, a 494-million-parameter open-weight model. Before training, it scored 0/36 on an independently authored probe suite of velk tasks (21 usage probes, 15 transfer probes) — exact zeros, as designed. On the frozen standard test it scored 10% (10/100), all of it on the two easiest problem types, where the output of a short program is sometimes guessable from surface patterns with no knowledge of the language (the probe suite excludes such items by construction, hence the exact zero there). Read together: the skill measurably did not exist.
The teaching method is the interesting part. The powerful model doesn't hand over text to memorize — it compiles the entire training environment: a compact textbook for the skill (560 tokens under the student's own tokenizer; the whole language fits on a page), a generator of practice problems, and, crucially, a checker. Because velk is a programming language, the checker is just its interpreter. Formally, for a problem \(x\) and a candidate answer \(y\), the verifier \(V(x,y)\in\{0,1\}\) is exact — no human grading, no learned reward model, no opinions.
Then the small model teaches itself. In each round it samples attempts with the textbook \(B\) in its context, keeps only what the checker approves, and is then trained on those successes with the textbook removed:
$$y \sim \pi_\theta(\,\cdot \mid x, B\,)$$ $$\mathcal{D}_r = \{(x,y) : V(x,y)=1\}$$ $$\theta \leftarrow \arg\min_\theta \; \sum_{(x,y)\in \mathcal{D}_{1..r}} -\log \pi_\theta(y \mid x)$$Note the asymmetry: reward is earned with the book, but the gradient is applied without it. Over rounds, the book migrates from the prompt into the weights. And the loop bootstraps itself even from a near-zero start: if a single sampled attempt succeeds with probability \(p\), then \(k\) attempts find at least one success with probability \(1-(1-p)^k\). We measured \(p = 0.0625\) per sample with the manual in context; eight attempts per problem then yielded at least one verified success on 3 of 10 problems in a held-out slice. That thin seam is enough ore to mine: skills learned from easy exercises unlock harder ones the model had never solved, which feed the next round.
Two points of discipline about what "self-teaching" means here. First, each round's weight update is plain supervised fine-tuning on the cumulative verified corpus, with a fresh adapter trained from the base model each round — the loop's contribution is generating the data, not a new learning rule. Second, the corpus is small and fully auditable: three rounds produced 38, then 79, then 127 deduplicated, machine-verified traces (at most two kept per problem).
The adapter is a rank-16 LoRA over all seven projection matrices of every transformer block: 8.8 million trainable parameters, or 1.75% of the 503 million total with the adapter attached, trained for three epochs per round. Everything — curriculum synthesis, training, evaluation — ran on Boson, SmartInfer's PyTorch-based model-development platform, on a single Apple M1 Pro laptop with 16 GB of memory. No GPU server was involved in anything reported here.
Before training began, the two test files, all four probe files, and both interpreter implementations were hashed (SHA-256) into a manifest and committed to version control. Every evaluation run verifies every digest before and after scoring and aborts on any mismatch; the tooling refuses in-place re-freezing. Every training and evaluation command appends a record to an append-only run ledger with content hashes of what it read and wrote.
The grader itself is cross-checked: the training-side interpreter and the examiner-side interpreter were written independently, and a 2,000-program cross-fuzz found zero disagreements between them. The trainer never writes the exam, never sees the exam, and cannot alter the exam.
We also pre-registered a target before any training run: at least 40% on the standard test and 20% on the harder test — smoke-scale bars, locked in the engagement's requirements ledger and committed on July 5, before training began. We have not met them yet; the results below say exactly where we are.
| Condition | n | Standard test | Harder test |
|---|---|---|---|
| Untrained, plain | 100 | 10% (10/100) | — |
| Untrained, few-shot | 25 | 4% (1/25) | — |
| Untrained, full manual in the prompt | 100 | 10% (10/100) | 0% (0/100) |
| Self-trained (round 3), no manual anywhere | 100 / 25 | 23% (23/100) | 12% (3/25) |
A finding we did not anticipate: at full scale, the manual in the prompt adds nothing at all over plain prompting — 10% either way, both entirely from the guessable easy types. Under greedy decoding, reading the book contributes exactly zero solved problems; its only measured value is under repeated sampling (the \(p=0.0625\) bootstrap above), which is precisely the seam the training loop mines.
The score built over rounds: 4% after round 1 (whose evaluation slice was dominated by task types that round had not yet trained), 20% after round 2, 23% after round 3. By task type, round 3 scored 28% on the shortest programs, 12% on mid-length programs, 8% on loop-and-nesting problems — and 44% (11/25) on the program-writing task, where the model composes a new velk program to meet a specification. That's no accident: when many different programs are all correct and every one of them is machine-checkable, the learning signal is densest.
What these numbers do — and don't — establish. A skill that measurably did not exist (0/36 on the probe suite) now lives inside a 494M-parameter model with no manual in its context: 10% → 23% on the frozen standard test, against a manual-in-prompt baseline of 10% — a 13-point gap at roughly 2.5 standard errors on the full 100-problem sets. More telling than the totals: the entire trained gain sits in problem types that cannot be guessed. On program-writing, the untrained model scores 0% with or without the manual; the trained model scores 44%. On the harder held-out set, the prompted baseline solves 0 of 100 while the trained model solves 12% of its slice. Memorization is ruled out by construction: training and test problems never overlap.
Alongside the expert-iteration loop, we ran a deliberately short reinforcement-learning experiment on the same workload. Boson includes a from-scratch GRPO trainer — group-relative advantages, a KL anchor to a frozen reference policy, and the verifier invoked exactly once per episode as the terminal reward — validated by twelve independently authored tests. We applied a single GRPO iteration to the round-3 model and scored the checkpoint against the same frozen sets.
The result: the checkpoint's scores were unchanged from its round-3 starting point — 23/100 on the standard test, identical per task type, and within one item on the harder slice (2/25 versus 3/25). To be clear about what that means: this is not two training methods arriving at the same score. GRPO started from the round-3 model, and a single KL-anchored update left its greedy outputs where they were — no collapse, no drift, no added capability — which is exactly what one conservative, KL-constrained step should do. These runs were kept short by design: their purpose was to verify that the training signal behaves — that accuracy holds or improves under it — not to maximize the score. No multi-iteration GRPO checkpoint has been scored yet, and none of the headline numbers in this report come from GRPO; they all come from the expert-iteration loop above. Longer GRPO runs are queued with the GPU-scale protocol.
Equally important is what we cannot yet claim. This is a single run — one sampling seed per round, deterministic evaluation, no seed replication. And the ablation that would isolate the loop's contribution has not been run: a model fine-tuned directly on the 480 teacher-written gold solutions, rather than on its own verified attempts, might do as well. That arm is scripted but unscored. Until it runs, we credit the result to self-generated, machine-verified data — not to iteration being better than plain fine-tuning.
Twenty-three percent is also short of our pre-registered 40%/20% bar, and we say so plainly. We have a specific suspect: the adapter is 1.75% of the model's weights — a capacity limit we imposed on ourselves, not an observed limit of the method. Raising it is the first experiment in the queue, alongside the three-seed replication and verdict-blind control protocol already scripted for GPU hardware.
Velk itself is useless — that's the point; it's a clean measuring instrument. What it calibrates is a pipeline: give our agent system a topic, and it compiles a textbook, builds an honest examiner, and trains a small model until the skill demonstrably lives in the weights. Swap velk for a company's internal query language, a compliance rulebook, or a niche calculation domain, and the same machinery produces something genuinely useful: a private, inexpensive, on-device expert.
Our next experiments push in two directions. First, closing the gap to the pre-registered bar, starting with adapter capacity. Second, moving from toy languages to domains people care about — including an invented proof language, where the same self-teaching loop would train a small model to write algebraic proofs that a 300-line checker can verify with certainty, with real competition mathematics as the final exam.
Small models won't out-think the giants any time soon. But for one domain at a time, with a perfect grader in the loop, they may not need to.
— SmartInfer Lab
Provenance: every number above traces to an append-only run ledger; the test sets, probe sets, and both graders were SHA-256-frozen and committed to version control before training began. Language invented and frozen 2026-07-05; headline training runs 2026-07-05/06; full-set baseline re-runs and the GRPO check 2026-07-23.