When the model is certain: library versions that never existed

An agent once explained to me that a library was now available in a newer major version, and that this version solved exactly my problem. The description was coherent, the proposed usage fitted the rest of the code. The version did not exist. When I asked again, what came back was not a correction but a restatement.

That is the failure mode that occupies me with third-party libraries. Not the code that fails to compile, but the code that is wrong and sounds assured while being it.

Why this differs from your own code

Inside your own project an agent has the truth in front of it. It can open the file the function lives in and see which parameters it takes. Being wrong about that surfaces at compile time.

With a third-party library that feedback is missing. What the model knows about it comes from the point its training ended, and from everything written about it since, undated and unranked. A method that was standard two years ago and has since been removed is just as present in that view as its successor.

Both are attested, just not at the same time.

On top of that, errors in this class surface late. A wrong method name breaks immediately. A call with the right signature but the semantics of the previous version runs through, returns something plausible, and only becomes conspicuous when somebody checks the number.

The most unpleasant are silent assumptions: that a library carries time zones along, that a call is idempotent, that a default is what it was three versions ago. Assumptions like that are written down nowhere. They surface when a number is wrong.

What I built out of that

The conclusion is not to distrust answers and look each one up individually. It consists of two rules, both of which take effect before the work starts.

Documentation comes from outside, not from memory. When a third-party library is involved, the agent is handed the current documentation: either as a reference I look up myself, or through an access path that fetches the documentation at runtime. The difference is not convenience. Remembered knowledge carries no date; retrieved knowledge does.

Versions are outside the agent's authority. That sits as a rule in the project's guardrails: it does not change dependencies or their versions. If a task needs a newer version, that is a finding it reports, not a change it makes.

The second rule matters more, for a reason that only shows on second look: it makes the invented version harmless. A model that hallucinates a version but may not touch the dependency file produces, at worst, a proposal somebody decides on. A model that writes the same hallucination in itself produces an install failure, in the good case, or a package with a similar name, in the worse one.

That moves the check to a place where it is cheap. Instead of reading calls against my own memory, I answer a single question: where does this claim come from. Is it in the documentation that was provided, or did it emerge between the lines.

How this differs from checking intent

On superfluous abstractions I wrote that tools do not help and that I have to look myself. Here it is the other way around, and that is why the two belong apart.

Whether an abstraction is appropriate cannot be answered by any tool, because the answer depends on the undertaking. Whether a method exists can be answered by any tool that reads the documentation. Where an external authority can be right, my judgement is the weaker instrument, not the stronger one. That applies to me as much as to the model: I too remember APIs as they were two years ago.

What I pay for this

The first cost is carried by the version rule. An agent that may not touch dependencies also does not raise one when raising it would be right. Updates accumulate and get done by hand. I traded wrong changes for deferred ones, and deferred updates are a debt that grows quietly.

The second cost sits in the flow of work. Providing documentation means knowing in advance which library will be needed, or stopping mid-task to go and fetch it. It is not a large item in hours, but it interrupts, and interruptions feel more expensive than they measure.

Where the problem stays small

With established libraries whose interface has barely moved in years, the model is almost always right. There is plenty written about what holds, and what holds has not changed. The whole effort is aimed at a different case: young projects, fast release cycles, libraries that rebuilt their interface between two major versions.

The calculation is also different when a wrong assumption breaks immediately. Where the compiler finds the error, no upstream check is needed. It gets interesting where a stale assumption runs through and hides as a number in the result.

Remembered or looked up

The most useful question with third-party libraries is not whether the code looks right. It is: where do its claims come from. Remembered knowledge and looked-up knowledge are indistinguishable in the output, and only one of them has a date.

Everything I built here serves that single distinction. Not because the model is unreliable, but because its reliability cannot be read off from the outside.