Eric Spencer
projects
A small transformer trained on ASCII art, reading it rather than generating it. The model is given a chunk of pyfiglet's blocky text rendering of a random word and has to recover the word, which makes it less an LLM than a text-only OCR experiment with a transformer encoder attached. The repo name comes from the looser description.
The pipeline is four scripts. ascii_generator.py picks a random lowercase word between 3 and 12 characters long, runs it through pyfiglet in a chosen FIGlet font (default standard, or sampled from a list including slant, 3-d, doh, isometric1, and bubble when --multi-font is passed), and writes the (word, art, font) triples to a JSONL file. data_prep.py builds two small vocabularies, one for the characters that appear in the rendered art and one for the 26 lowercase letters plus a <pad> token, then converts everything into padded numpy arrays. train.py runs the transformer. evaluate.py loads a checkpoint and prints predictions.
The model is small: PyTorch's nn.TransformerEncoder over the flattened ASCII art token sequence, default 4 layers, d_model of 192 to 256, 8 heads, 512-dim feedforward. The flattened sequence runs to roughly 1200 tokens, because ASCII art is wide. Encoder outputs are mean-pooled into one vector, projected to target_vocab_size * MAX_WORD_LEN and reshaped, so all 12 character positions are predicted at once instead of decoded autoregressively. The loss is cross-entropy with ignore_index=PAD on the target side. The flat head keeps the model simple and avoids a decoder loop for what should be an easy task.
self.classifier = nn.Linear(d_model, target_vocab_size * max_word_len)
# ...
pooled = enc_out.mean(dim=1)
logits = self.classifier(pooled).view(-1, self.max_word_len, self.target_vocab_size)
On single-font data in the standard font this works well. The model converges in a handful of epochs and reaches high per-character accuracy on held-out words. That follows from the font being a deterministic letter-by-letter rendering: the model learns to segment the art into per-letter columns and classify each column. It is template matching, not language modeling.
Multi-font data is where it falls apart. Once doh, isometric1, and banner3-D are mixed into the same dataset, the model has to learn each font's quirks, including variable letter width, characters that bleed across positions, and fonts that draw the same letter in a completely different way. A 4-layer encoder with a few thousand samples is not enough, and exact-match accuracy drops hard. The candidate fixes are a proper sequence-to-sequence decoder, CTC loss so the model does not need to know where each letter starts and ends, or simply more data and a bigger model. All of them sit unimplemented in the README's "Future Ideas" section.
This is a one-weekend curiosity project. It does not produce a usable model, and it does not generate ASCII art, the inverse direction being both harder and more interesting. What it is good for is a small transformer written from scratch with PyTorch, and a refresher on how TransformerEncoderLayer works.