Nvidia, AMD, Intel, Google and Meta Stand Behind One Open-Source Inference Engine: The SGLang Consortium Launches

The companies that make AI chips, the companies that train the models, and the companies that serve them to users have agreed to steer a single piece of open-source software together. On November 12, at the opening keynote of the first SGLang Summit in San Francisco, the nonprofit LMSYS Org announced the SGLang Consortium, with founding members Nvidia, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk.

Those eight names sit on opposite sides of nearly every commercial battle in AI infrastructure — Nvidia and AMD sell competing accelerators, Google and Meta run rival clouds and model labs, Cloudflare and Modal compete in serving. The consortium exists because the software layer underneath all of them has become too important for any one company to own alone. SGLang, part of the PyTorch ecosystem, is the open-source inference engine behind some of the largest production deployments in the world, running on hundreds of thousands of GPUs for companies including Meta, Microsoft, Google, Amazon, Thinking Machines Lab, Cloudflare, Reflection and Modal.

The economics explain the urgency. Deloitte estimates that inference will account for roughly two-thirds of all AI compute in 2026, up from about half in 2025. Training builds the models; serving them is where the money now goes. As models grow more agentic — chaining dozens of reasoning steps and tool calls into a single answer — squeezing more work out of every GPU matters more than ever, and SGLang is best known for exactly that: its RadixAttention technique reuses cached computation across requests, and it is widely used for fast structured generation.

Under the consortium's structure, each founding member designates a technical representative and joins two roadmap meetings per year with SGLang's maintainers, contributing engineering expertise, deployment feedback, testing resources or joint development. Working groups may collaborate between meetings, with public updates on roadmap direction and progress. The shared priorities named in the announcement are frontier model support, reasoning and multimodal workloads, distributed inference, and hardware optimization — the last one telling, since SGLang already runs on Nvidia GPUs, AMD Instinct accelerators, Intel Xeon processors, Google TPUs, Huawei Ascend chips and Moore Threads hardware.

The governance terms matter as much as the membership list. SGLang's maintainers keep final responsibility for technical decisions and code acceptance, and the project remains open to the entire community — the consortium shapes the roadmap, it does not take over the project. "The teams running SGLang at the largest scale see what inference needs next before anyone else does," said Ying Sheng, co-author of the original SGLang paper and co-founder and CEO of RadixArk. "With the challenges in building new data centers, software approaches to improve AI efficiency will be incredibly important."

The vendors' quotes underline what each side wants. Ian Buck, Nvidia's vice president of hyperscale and high-performance computing, framed open inference software as a multiplier for Nvidia's own silicon: "Better software helps developers get more performance from NVIDIA compute... And lower inference costs make AI more accessible to organizations everywhere." AMD's Ramine Roane, corporate vice president of AI applications, emphasized the cross-hardware angle: "SGLang gives developers a fast, open inference engine that runs across hardware, including AMD Instinct GPUs."

The launch lands days after a security flaw in SGLang was exploited at NEAR AI, a reminder that infrastructure this widely deployed needs shared stewardship as well as shared roadmaps. No new feature shipped with the consortium announcement; the real test is whether eight rivals produce public cross-vendor benchmarks and faster serving in code rather than press releases. But the signal is hard to miss: even Nvidia, whose CUDA software has been the industry's deepest moat, is now sitting at a table built to make AI inference run well on everybody's hardware.