What are n-grams in language models?

Tiny word sequences — uni (1), bi (2), tri (3) — used to predict what's next by raw counts from text data.

How do n-grams relate to ChatGPT?

They're the primitive ancestor; LLMs upscale the next-token prediction game with massive context n-grams couldn't touch.

On edge devices, yes — cheap, private, no cloud needed. Transformers too fat for phones.

🔬 AI Research

Peanut butter and... engine? Your gut recoils. Machines learned that trick with n-grams — the forgotten stats hack kickstarting language AI.

theAIcatchup Apr 10, 2026 3 min read

N-grams pioneered next-word prediction via simple counts, birthing sequential language modeling. 𝕏
Markov assumption limited them to local context, exposing early why language defies stats alone. 𝕏
Modern LLMs evolved this idea; n-grams could revive for efficient, on-device AI. 𝕏

Published by

AI news that actually matters.

#Markov models #NLP history #language prediction #n-grams

Get the best AI stories of the week in your inbox — no noise, no spam.

Originally reported by Towards AI