MIT Shows GPT-6 Astra Rewiring Another Model: Editing 0.016 Percent of Qwen3.5-0.8B Weights Lifts MATH-500 From 7.9 to 48.1

Researchers at MIT CSAIL say frontier models have picked up a strange new skill: reading and rewriting the weights of other neural networks directly, no gradient descent, no training loop. In a blog post titled "Sparks of Weight-Space Programming" published on November 2, doctoral students Rikiya Takehi and Jayden Teoh, together with Alex Zhang and assistant professor Omar Khattab, describe eight experiments in which GPT-6 Astra inspected small networks, diagnosed their failures, and patched them by editing weights the way a programmer edits code.

The headline experiment targets Qwen3.5-0.8B, an 800-million-parameter model whose thinking mode had a known failure: it gets stuck repeating itself, and 93 percent of its answers ran into the 32,768-token limit, leaving MATH-500 accuracy at just 7.9 percent. Astra was given a Python sandbox, access to the weights, practice problems, and a budget of at most 50 forward passes. It traced the failure to a direction written by the MLPs in layers 15 and 19 and subtracted that direction from 64 output columns in each layer, touching 131,072 weights, about 0.016 percent of the model. MATH-500 jumped to 48.1 percent, GSM8K rose from 13.8 to 67.1 percent, and instruction following on IFEval went from 21.0 to 62.3 percent, while the share of answers hitting the token cap fell to 33.1 percent.

The paper itself is more careful than the viral framing. The authors note the gains come from removing an obstacle, not adding capability: their control experiment ran the same procedure on Gemma 3 270M, a model where only 8.5 percent of answers hit the cap, and could not move MATH-500 beyond roughly a point, from 6.8 to at most 7.7. Editing weights, they conclude, can remove hidden blockers but cannot give a model abilities it never had. The full 48.1 percent result also needed all 50 forward passes; budgets of 10 or 25 passes produced far smaller gains.

The other experiments sketch what the models can do with weights alone. Astra named which game a 17,253-weight policy network had been trained on among Breakout, Pong, Pac-Man and Freeway, identified what three small diffusion models drew, and located induction heads in a tiny transformer that matched the published mechanistic-interpretability literature. In one repair task it fixed an official Kinetix agent that scored zero out of 256 episodes by editing 3.5 percent of its weights, after which the agent passed all 1,024 test episodes on the first attempt. Writing from scratch was more mixed: a zero-shot network for Breakout scored 517.4 against 25.3 for PPO trained 500 million steps, but on Seaquest and Asterix the PPO baselines still won.

Not everything worked. Asked to identify sycophantic models from weights alone, Astra failed all four attempts; it only succeeded once allowed to inspect attention patterns, where it flagged a mid-layer head that attended to the user answer token 85.7 percent of the time versus 0.75 percent in the honest model. Community reaction has been sharp on both sides: the announcement drew millions of views within a day, while skeptics note that the prompts guided the search direction, that weight-editing methods like ROME date back to 2022, and that the capability, once edited in, may not survive on cleaner models.

The MIT group is upfront that the work is early and that results are limited by context length and harness efficiency rather than any deep ceiling. Logs, code, the edited model and the evaluation details are public. Whether weight-space programming becomes a real tool or stays a demo, the framing has already shifted: model weights are becoming something models can read and write, not just something humans train.