Learning in the Age of Large Language Models

Since large language models (LLMs) became part of my daily work, I have noticed my relationship with knowledge quietly changing too. This post is a list of the tangible things I found in that change: both a user’s guide and a warning.

The illusion of fluency

Why can the fluent texts of language models be misleading?

In its simplest definition, a language model is a system that predicts the likely continuation of a text. During training it learns the statistical texture of language from billions of texts, and that texture is not only grammatical but also cultural and conceptual. That is why the text it produces is fluent. Research in psychology also shows that statements that are easy to read and process are more likely to be judged true.

But fluency is not the same as accuracy. A model writes in the same confident tone whether it “knows” a fact or is making something up. The silence between the two is the most dangerous part: false information speaks in the same voice as true information.

History writing has an equivalent: witnesses can be confident too. That is why the old method does not question the witness’s story but where the story comes from.

Verification is the new literacy

How should the output of artificial intelligence be verified?

Before language models, “checking sources” was the specialty of certain professions. Not anymore. Opening a URL the model gives you, calculating a number yourself, actually running a piece of code: these now have to be everyone’s basic skills.

My own rough rule is this: model output is a draft by default. No unverified line becomes permanent for me before I publish, send or decide anything. This is not a rule that slows the work down; on the contrary, it is the only way to use the speed of a draft safely.

Where models really shine

What are language models really good at?

In my experience, LLMs are consistently fast at:

• Producing a first draft: they remove the fear of the blank page; a bad draft is better than a perfect void.
• Tedious transformations: format changes, filling templates, skeleton code; anything mechanical.
• Translation: a quick first translation between technical texts.
• Getting started with the unknown: sketching the map of a new concept; but the map is not the territory,¹ so it must always be walked with primary sources.

Where models mislead

What do language models get wrong?

• Citing sources: a model can confidently cite a book, article or API function that does not exist. A 2023 study found that more than half of the citations produced by GPT-3.5, and about a fifth of those produced by GPT-4, were fabricated. So every citation must be opened and checked one by one.
• Numerical reasoning: for a model, arithmetic is a language problem; for a calculator it is not.
• Long chains of logic: an argument in which every step looks right can be wrong as a whole.
• Fields that demand precision: law, medicine, money. Here the model is a drafting assistant, never the decision maker.

The learning problem: the price of ease

Does artificial intelligence make learning harder?

This is the issue that makes me think the most. Learning is, by definition, a little effortful. In 1994 the cognitive psychologist Robert Bjork called this “desirable difficulties”. Some difficulties, such as spaced repetition or testing yourself, slow learning down in the moment but make it last in the long run. Ready-made answers can remove exactly these difficulties.

A learning process that consists of taking every answer ready-made from a model looks fast, but it stays on the surface. The balance I try to keep is this: when I want to learn something, I first work with my own notes and sources, and only then use the model to fill the gaps where I struggled. When I want to produce something, the order is reversed.

A small manifesto

Which principles should we follow when working with language models?

To cut a long story short, these are the principles I keep for living with language models:

1. Fluency is not evidence.
2. Every citation is verified; every number is calculated; every piece of code is run.
3. The model is a drafting partner, not the one who signs.
4. When learning, your own effort first, the model’s help second.
5. Getting a wrong answer quickly can cost more than finding a right one slowly.

These are really new costumes for old principles that apply everywhere, from software to history writing. Perhaps the greatest lesson of the age of artificial intelligence will be to teach the oldest lesson again: do not believe what you see; look at things as they are, not as they are presented.

Notes

1. The phrase goes back to the thinker Alfred Korzybski’s dictum “The map is not the territory”. ↩

References

1. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). ACM. https://doi.org/10.1145/3442188.3445922
2. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. P. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185–205). MIT Press.
3. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 248. https://doi.org/10.1145/3571730
4. Reber, R., & Schwarz, N. (1999). Effects of perceptual fluency on judgments of truth. Consciousness and Cognition, 8(3), 338–342. https://doi.org/10.1006/ccog.1999.0386
5. Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. https://doi.org/10.1038/s41598-023-41032-5