Baidu's ERNIE Evolve Beats Public Records on NanoChat and NanoGPT Speedrun, and It Is Already Optimizing the Next ERNIE Model

Baidu's ERNIE team has disclosed an automated research system called ERNIE Evolve that it says can propose research directions, write and modify the underlying code, run the experiments and keep only the changes that survive independent evaluation and review. The company positions it as a practical step toward recursive self-improvement: AI participating directly in the development of AI, with numbers attached rather than a manifesto.

The architecture runs two nested loops. An outer loop combines an Experience Bank of past experiments with a Knowledge Base of papers and code, and its Director agent selects the next research direction and starting version for each of several parallel research lanes. Inside each lane, an inner loop iterates: a Selector picks the candidate and hypothesis, an Executor modifies the code, and an Evaluator, kept as a fixed program outside the repository so no agent can see it, scores the result against a noise threshold. A Reviewer then checks whether the improvement follows the intended direction or merely exploits an evaluation loophole. Failed experiments are logged so later rounds avoid repeating them.

On public benchmarks, Baidu claims verifiable bests. On NanoChat AutoResearch, the system reached 0.8763 validation bits per byte within a five-minute single-GPU budget, beating the public best of 0.8878, with an architecture that added an n-gram branch and a document-retrieval branch beside the network, mixed by a learned gate. On NanoGPT Speedrun Track 1 it hit the target loss in 38.92 seconds averaged over 18 runs on eight H100s, against the 39.9-second community record; Track 3 fell from 2,690 steps to 2,590. On SOL-ExecBench, a suite of 235 kernel problems on B200s, it scored 0.819 against a leading 0.827, the one gap the company discloses. For comparison, Tencent's Hyra-1.0 and Recursive posted 76.4 and 77.5 seconds on Speedrun runs set against the older record.

The production results matter more than the leaderboards. Baidu says ERNIE Evolve has been applied to inference optimization for the upcoming ERNIE 5.5 Flash, and that every accepted change passed the same stress and correctness checks as human submissions. On Kunlunxin P800 chips in low-latency settings, the system spent roughly 168 hours across four rounds of exploration, adding fused operators and speculative decoding before optimizing host scheduling and communication paths, and raised generation speed to 623.6 percent of baseline. On already well-tuned GPUs, deterministic inference throughput rose 33.7 percent in about three days and extreme decoding speed improved 14.9 percent, with accuracy unchanged.

All figures are Baidu's own measurements and have not been independently verified, though the company has released code and run logs in a public GitHub repository, ernie-research/ERNIE-Evolve-Results, which makes the benchmark claims checkable by anyone willing to rerun them. That is a meaningful step beyond the usual self-reported slide.

The significance is less the records than the pattern. Anthropic's internal research acceleration, METR's measurement of AI-driven R&D speedups, and Tencent's Hyra all point the same direction: the loop of proposing, coding, testing and reviewing is being automated, first for inference efficiency and next, by Baidu's own stated roadmap, for training systems, model capabilities and evaluation. Chinese labs have largely been consumers of this narrative; ERNIE Evolve is the first Chinese system to put verifiable public-benchmark numbers behind it.