What fits on 16GB: choosing a Swahili speech recognition baseline

This project started with a hardware limit rather than a research question. One GPU, 16GB of video memory, and no budget for data-centre time. Before deciding anything about datasets or fine-tuning method, we needed to know what would physically train on that machine, and how badly an untrained model handles Swahili in the first place.

Those two questions turn out to answer each other.

What the memory limit actually decides

Whisper large-v3 has 1,550,490,560 parameters, a count documented alongside its architecture in a recent Swiss German fine-tuning study. On paper that sounds like a model for a data centre. In practice it depends entirely on what you are asking the GPU to hold.

Running it for inference in FP16 takes around 3.4GB, counting weights, activations and the key-value cache. Adding LoRA adapters for fine-tuning pushes that to roughly 5GB, which leaves room for a usable batch size. Full fine-tuning is a different proposition: gradients and optimiser state take roughly four times the inference footprint, and 16GB will not hold it. Those memory figures come from GPU-requirement estimators, infrastructure guides and practitioner write-ups rather than from a paper, so treat them as indicative and check them against your own hardware.

So the constraint did not force us down to a smaller model. It forced us into a different training method. Quantise the base model to 4-bit, attach LoRA adapters, switch on gradient checkpointing, and large-v3 trains inside 16GB with headroom.

Two published fine-tuning recipes arrived at almost the same configuration independently. The Indic ASR flywheel study settled on rank 16 with alpha 32, targeting the attention projections. The Swiss German study explored rank and alpha across twelve runs before landing in the same region, and added the feed-forward layers. So: rank between 16 and 32, alpha at twice the rank, attention projections first and feed-forward layers if memory allows. When two teams working on unrelated languages converge on the same numbers, that is a reasonable place to start rather than a hyperparameter search.

What Swahili costs before you train anything

Whisper’s headline accuracy comes from English. Benchmark summaries put it at roughly 2.7% word error rate on the clean LibriSpeech test set and 8 to 12% on real-world English audio, and record that 20 of the 99 languages Whisper supports have no training data behind them and are flagged as experimental. Those particular figures come from 2026 vendor benchmarking write-ups rather than from the original paper, which is worth knowing before quoting them.

Swahili is not in the no-data group, and it is still expensive. Nahabwe and colleagues at Makerere University and Marconi Lab benchmarked ASR models across African languages for the 2025 Deep Learning Indaba research track. They report Swahili word error rate at roughly 51.3% with one hour of adaptation data, improving to roughly 16.9% at 400 hours. The gains flatten after about 100 hours.

That curve changed our plan more than any of the memory arithmetic. It says the difference between a demo and a usable system is around 100 hours of adaptation data, and that going from 100 to 400 hours buys much less than the first jump. It also says nothing at all about which 100 hours.

The numbers we have decided not to trust

Whisper was pretrained on roughly 680,000 hours of multilingual and multitask audio, and that corpus has never been itemised. Any public test set may already sit inside it, which means a good-looking score can be memorisation rather than recognition.

The Swiss German study handled this well enough to be worth copying. The team checked their evaluation data for overlap with Whisper’s pretraining set and published 25.6% word error rate, and 13.8% character error rate, as a contamination-checked baseline, rather than the better number they could have reported without the check. They put it in the title. We will hold ourselves to the same standard: disclose the overlap risk, and report on a held-out set we can show is absent from Whisper’s published training description.

Three more reporting decisions follow from the same discipline.

Report character error rate alongside word error rate. Swahili is agglutinative, so a single word-boundary error can inflate WER without much affecting whether a human understands the transcript.

Report raw and normalised figures separately, and name the normaliser. Whisper ships its own text normaliser and applying it changes the number materially, which makes cross-study comparison misleading unless everyone says what they did.

Stop treating “Swahili” as one language for benchmarking purposes. It spans Kenyan, Tanzanian coastal, Zanzibari, Congolese and Ugandan variants with real phonological differences. Almost every published figure, including the 51.3% above, is a blend. We will tag by variant wherever the dataset metadata allows it.

Small model to decide, large model to report

The last open question was whether to run method experiments on a small model or a large one. Cheap iteration argues for small. Production representativeness argues for large.

The literature suggests this is not a single choice. Sehar, Khalid, Adeeba and Hussain’s n-shot evaluation of Whisper on Pashto, Punjabi and Urdu found that few-shot fine-tuning reduces error rates on the smaller variants too, which means conclusions about method transfer directionally even when the absolute numbers do not. So the plan is to run data cleaning, augmentation and hyperparameter work on Whisper small or medium, then validate the winning configuration once on large-v3 for the figures we publish.

On cost: the Indic ASR study reports around seven GPU-hours and roughly $13 per language on a single A10G. We are quoting that with low confidence. It is one job on one architecture, cloud pricing moves, and we have not reproduced it.

What we are actually testing

The engineering question was which model fits on the hardware we have. That one is now settled, and it was the easy one.

The question underneath it is whether a Swahili model that runs on a single consumer GPU can be accurate enough to matter in a real workflow. Not accurate in the abstract, but accurate enough that a business owner who has never completed a written application can be assessed on what they said, in the variety of Swahili they actually speak, by an institution that would otherwise have never heard from them at all.

We will publish the zero-shot numbers first, contamination check included, before any fine-tuned figure.

Sources