LLMs do not think, reason, or understand physics. They are continuous high-dimensional vector engines that transform human text into numbers, push those numbers through billions of static matrix calculations, and roll a weighted dice to turn numbers back into text.
Words → Numbers
Text split into sub-words, assigned unique dictionary IDs, and converted into continuous vector coordinates.
Numbers × Billions of Weights
Vectors pass through 32 to 80 Transformer layers where attention connects words and memory cells fire.
Numbers → Words (Dice Roll)
Output scores become probabilities. A stochastic dice filter rolls to pick the next word, looping autoregressively.
Signals originate as raw prompt strings on the far left, convert to numbers, ripple through 32 to 80 Transformer layers, pass through a stochastic probability filter, and emerge as output words on the right.
Ingestion: Translating String Text into Continuous Vector Coordinates
Computers cannot process string letters directly. First, a tokenizer slices words into sub-word chunks called tokens using algorithms like Byte-Pair Encoding (BPE).
Every token maps directly to a unique integer index in the model's fixed vocabulary dictionary (containing 32,000 to 128,000 cataloged words).
Integer IDs cannot undergo math operations. They are converted into dense streams of 4,096 numbers (embeddings) blended with sinusoidal wave signals (positional encodings) to track word order.
Deep Transformer Layers: Calculating Word Connections & Associative Memory
Click on any word below to see how the model calculates attention weights connecting it to other prompt words (e.g., how "blue" directs strongest attention toward "sky" and "Why").
While Attention connects active prompt words, Feed-Forward layers act as a massive static dictionary encyclopedia. The vector expands 4x in size to fire stored pre-training knowledge.
Triggers static knowledge memory cells learned during web training
Each layer refines the numbers. Early layers handle basic syntax; middle layers handle semantics; deep layers form high-level abstract predictions.
Output Logits, Probability Distributions, and the Temperature Dice Sampler
After 80 layers, the LLM produces raw score numbers (logits) for every word in its vocabulary. The Temperature Knob controls how flat or sharp the probabilities become before rolling a weighted dice to pick the winner!
At Temperature 0.7, highest scoring token ("Sunlight") has a 68% chance of winning, but lower candidates still have a valid dice roll probability.
To demonstrate why LLMs lack comprehension, consider what happens when a model generates a response to "Why is the sky blue?":
Sunlight contains all color wavelengths. When light enters Earth's atmosphere, high-frequency short blue light waves collide with Nitrogen ($N_2$) and Oxygen ($O_2$) gas molecules and scatter in every direction.
The LLM has no concept of light waves, gas, or skies. If it generates: "The sky is blue because water vapor scatters light", it outputs this simply because words like "water", "vapor", and "blue" co-occur near each other on millions of web pages.
Language models synthesize syntactic coherence via mathematical probabilities, not semantic comprehension.