#transformer architecture

Code screenshot of Andrej Karpathy's microGPT Python implementation

microGPT: Karpathy Shreds GPT's Hype in 200 Lines

Karpathy built GPT from scratch. In 200 lines. Turns out, the emperor's got no clothes.

Hand-drawn diagram tracing Transformer forward pass with matrices and calculations

Pencil-Tracing Transformers: Retro or Rescue?

Remember when we understood our models? This guy's dragging Transformers back to the drafting table, matrix multiplications and all. Brutal, brilliant, or both?

3 min read 1 week, 5 days ago

#transformer architecture

microGPT: Karpathy Shreds GPT's Hype in 200 Lines

Pencil-Tracing Transformers: Retro or Rescue?

Stay in the loop