Developer Paul Toth has released MiniLLM, an experimental language model implemented entirely in Delphi without neural networks, GPUs, external dependencies, or machine-learning frameworks.
Instead of using a transformer architecture, MiniLLM combines PMI-based word embeddings, a weighted Markov transition graph, a recurrent context vector, and similarity-based scoring. It generates multiple candidate responses and reranks them using semantic similarity, transition coherence, repetition penalties, and anti-loop heuristics.
The result is a small, deterministic, and fully inspectable language model that can learn from a compact text corpus, understand simple questions, and generate coherent responses. Its deliberately straightforward Object Pascal code also makes the project useful for exploring how tokenization, embeddings, contextual scoring, and text generation work under the hood.
MiniLLM is designed to be hacked and extended, with room for larger corpora, multilingual support, syntactic tagging, custom scoring functions, improved reranking, and model import or export.
See how far a transparent, neural-network-free language model can go with pure Delphi.