X/Twitter LinkedIn Reddit Facebook Listen on Spotify

Training Our First SVG Icon Generation Model

June 17, 2026 LLM Training Hands-On
"The best way to learn how LLMs work is to train one yourself — even if it's not the model you'd ship to production."
🎧 Listen to this article

Why I'm Learning to Train Models

It's July 2026, and I've set myself a goal that feels both timely and necessary: seriously level up my LLM engineering skills. Not just prompting, not just fine-tuning — actually understanding how these models work from the ground up, how they're trained, and what happens when you push real data through them.

The catalyst is Sebastian Raschka's book "Build a Large Language Model (From Scratch)", published by Manning. It doesn't just explain transformers — it walks you through building a GPT-style LLM in PyTorch, step by step, from text embedding and attention through pretraining and fine-tuning. The companion code repo on GitHub is excellent.

But reading about training isn't the same as doing it. So I decided to pair the book with a real project: train a model that generates SVG icons.

The Project: IconShop

I teamed up with a good friend of mine in California — someone who shares the belief that the people who understand model training will have an unfair advantage in the years ahead. We found an open-source project called IconShop, forked it, and got to work.

IconShop is based on the paper "IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers" by Ronghuan Wu, Wanchao Su, Kede Ma, and Jing Liao (arXiv:2304.14400, 2023). The idea is elegant — feed the model thousands of SVG icon definitions, train it to understand the structure and patterns of vector graphics, then prompt it to generate new icons. Instead of calling an API or picking from a library, you get a model that creates icons on demand.

The original repo is kingnobro/IconShop on GitHub. My fork is at [GitHub link coming — will provide].

The Model Architecture

Here's what we trained:

Model architecture specifications
Architecture: 16-layer transformer decoder, 1024 hidden dimensions, 8 attention heads, 512 embedding dimension. Trained on the FIGR-SVG dataset (icon-style SVGs).
16
Transformer Layers
1024
Hidden Dims
8
Attention Heads
512
Embedding Dim

A 16-layer transformer decoder with 1024 hidden dimensions, 8 attention heads, and 512 embedding dimensions. Trained on the FIGR-SVG dataset — a collection of icon-style SVGs specifically designed for this kind of autoregressive generation task.

Training Run #1

We booked a GPU rig and let it run for two days. Here's what we put through it:

~6
Epochs
~200K
Steps
2
Days
1st
Model Checkpoint

Six epochs, roughly 200,000 training steps, over the course of two full days on rented GPU hardware. Not a massive run by modern standards, but more than enough to see if the approach works.

First Results

The model works. It can generate SVG icons from text prompts. Is it perfect? Absolutely not. There are artifacts, inconsistencies, and the occasional hallucination where the output is valid SVG but not quite the icon you asked for.

But it does produce recognizable, usable icons. And that's the point — this isn't about shipping a product. This is about learning what model training actually feels like: watching loss curves, adjusting hyperparameters, waiting through epochs, and seeing your first outputs come back from a model you trained yourself.

Sample Outputs

Below are some of the icons our first model generated. These are raw outputs — no post-processing, no human editing.

Prompt: "bottle"

The IconShop UI running on our GPU rig at checkpoint step_126000, generating 3 variants with top-p=0.50, temperature=0.40, and max sequence length of 2048.

IconShop generating bottle icons - 3 variants shown
Prompt: "bottle" — Variant 1 (thick outline), Variant 2 (line drawing with cap and bubbles), Variant 3 (abstract segmented shape). Generation time: 12.1s.

Prompt: "star" and "trash"

Two prompts showing the model's ability to generate multiple icon types. The star icons show good shape recognition, while the trash icons demonstrate consistency in icon style.

Generated star and trash can icons in grid view
Star icons (top row) and trash can icons (bottom row) — each with 4 variants showing different styles from solid fills to outlined versions.

Prompt: "clock"

Same generation settings as the bottle prompt. The clock results show the model's struggle with circular shapes and internal details.

IconShop generating clock icons - 3 variants shown
Prompt: "clock" — Variant 1 (simple L-shape in circle), Variant 2 (thermometer-like), Variant 3 (stylized airplane/A shape). Generation time: 7.6s.

What Went Wrong (and What We Learned)

Every first training run teaches you more from its failures than its successes. Here's what we've observed so far:

What's Next

This is just checkpoint one. The plan moving forward:

I'll keep updating this post as we iterate. The goal isn't just a better model — it's a better understanding of the craft.

Key Takeaways

References