Have questions?
Learn more about the Center
LLMs are neural network models (often using a transformer architecture) that have been trained on text datasets to statistically predict text. Training an LLM involves feeding it text (for example, all of Wikipedia, millions of books, articles, and web pages) and having it repeatedly guess the next word in a sentence. Over time, the model adjusts its internal parameters to improve these predictions. Training on large text datasets allows LLMs to learn the patterns of grammar and nuances of usage. The model builds up a complex internal representation of the patterns in language. When you sumbit a prompt, it uses those learned patterns to generate a continuation that statistically fits.
Importantly, the LLM does not know facts the way a person does. It doesn’t have an explicit database of verified information. Instead, it generates text that sounds plausible based on its training. This can make LLM outputs impressively coherent while simultaneously being unreliable and factualy incorrect.
To illustrate, imagine asking an LLM: “What are the main causes of the French Revolution?” The model will draw on patterns in texts it saw about French Revolution causes and generate a paragraph about economic hardship, inequality, Enlightenment ideas, etc. The output can be very detailed and written in scholarly tone because the model has effectively “learned” from countless history texts. However, the model isn’t truly reasoning or recalling real facts; it’s echoing and combining things from its training data. This has profound implications for how we critically evaluate LLM outputs.
This dual nature, fluent generation without true comprehension, is why critical literacy is crucial when using generative AI.