Microsoft has switched on RDMA over USB4 in Windows 11, a feature that lets several ordinary PCs connected with standard USB4 cables pool their memory and run a single AI model that would not fit in any one of them, with no dedicated network adapters, no switches and no third-party drivers.
The company's published tests show what that means in practice. Three AMD Ryzen AI Max+ 395 systems, each with 128GB of unified memory, ran the open-weight Qwen3-235B-A22B model in 4-bit quantization at 18.5 tokens per second in Microsoft's lab measurements, with AMD reporting 21.98 tokens per second in its own testing. A 235-billion-parameter model does not fit in 128GB; across three machines it does.
Intel's reference configuration points the same direction at smaller scale. Three Intel Core Ultra Series 3 systems with 64GB of memory each ran GPT-OSS-120B at 49.4 tokens per second, and a four-machine setup reached 54.2. Again, the point is not raw speed but feasibility: a 120-billion-parameter model on machines whose individual memory tops out at 64GB.
The enabling numbers are the latency ones. Microsoft measured round-trip RDMA latency of 28 to 50 microseconds between two machines over USB4, against 116 to 194 microseconds for conventional UDP-based Ethernet on the same physical link, roughly a quarter of the delay. Tensor-parallel inference swaps activations between machines many times per token, which is why the company argues that a low-latency direct memory path riding a link rated up to 80Gbps is what makes multi-PC inference usable at all.
Deployment is currently for enthusiasts rather than consumers. The feature sits in the Windows Insider Experimental channel, requires registry edits and, per Chinese-language write-ups of the preview, disabling Secure Boot. Microsoft recommends a fully-meshed topology, a direct cable between every pair of machines, over daisy-chaining, which visibly hurts performance. Inference frameworks that support the NetworkDirect interface work without code changes, and Microsoft has open-sourced a llama.cpp-based test harness.
All performance figures are vendor-reported and have not been independently verified. But the direction is notable: the memory wall that has pushed local AI toward expensive unified-memory machines and cloud rental is now being attacked with USB cables and spare laptops. It will not outrank a real GPU cluster, yet for hobbyists and small teams it moves running the big open model at home from a hardware purchasing decision to a cabling one.
Comments (0)
Log in to join the discussion
Log InNo comments yet