7B Pharo models outperform larger LLMs on low-resource language code completion
Researchers trained Pharo-specific code models from open checkpoints, proving small fine-tuned LLMs can outperform larger general models on low-resource languages with scarce training data.
A team from TU Wien, Inria, and the University of Victoria released a pipeline and benchmark suite for bringing LLM-based code completion to Pharo, a Smalltalk-inspired language whose community has relied on single-token completion until now. Published on arXiv on July 9, the paper describes an end-to-end workflow that curates Pharo-specific data, continues pre-training on open code models, and fine-tunes for completion tasks. The resulting specialized models—small enough to run in real-time inside an IDE—substantially outperform their original base checkpoints and exceed the accuracy of much larger general-purpose code LLMs on Pharo completion.
Pharo is a low-resource language: training corpora are scarce, and mainstream code models treat it as out-of-distribution. The authors built two benchmarks to measure whether models learn Pharo's syntax and whether they accurately complete masked code from real-world GitHub repositories. Empirical results show that continued pre-training on Pharo data followed by task-specific fine-tuning closes the gap, with 7B-parameter specialized models outscoring larger general models that have never seen Pharo in training. The paper frames Pharo as a case study for any language community facing the same data scarcity problem.
The authors released the benchmarks and the training pipeline on HuggingFace, giving other low-resource language communities a template for replicating the approach. The paper does not name specific model checkpoints or publish weights, leaving open questions about which base models were used and whether the Pharo-specialized weights will be shared publicly. The next step for the Pharo community is integrating one of these models into the IDE and measuring real-world completion acceptance rates—benchmarks measure syntax correctness, but developers care about whether suggestions match their intent. If the integration lands and acceptance rates hold, the pipeline could become a blueprint for dozens of other languages stuck in the single-token completion era.
