Software is changing.

Grow with it.

Learn how LLMs work by building one.

Write the Transformer yourself, from an empty file, and at the end published open weights run on your code. Stage 1 is open now.

(ssh) try.modelmathics.com

The road to your own Transformer

You start from an empty Python file, and every lesson adds code that the lessons after it build on.

An empty Python file

A tokenizer

A bigram model

Attention

A Transformer of your own

Published open weights run on your code

I am a software developer who wanted to understand how language models really work. So I am building Modelmathics and taking every lesson myself, as its first learner. It is teaching me to write a Transformer, so it can teach you too!

Lucas

Join the beta

Stage 1, the bigram model, has 11 lessons and its first build live today.

The lessons of stage 1 go on in mm, the terminal app, with your code in files on your own machine.

mm opens by invitation during the private beta, and there is nothing to pay. The invitation comes by email.

Request access

Once your invitation arrives, create your account at app.modelmathics.com/signup. Then install mm and run mm login:

curl -fsSL https://modelmathics.com/install.sh | sh
What this installs

This installs mm with uv, Astral's Python tool manager, which it installs first if it is missing. It downloads a Python if none at 3.11 or newer is there, and adds uv's tool folder to the PATH of new terminals. It all goes in your home folder, without sudo. Read the script first: modelmathics.com/install.sh.