Is CUDA Still a Moat for NVIDIA?
Erik Bernhardsson says CUDA is a little overstated as NVIDIA's moat. He does not think CUDA itself is the moat. He calls it an annoying programming model.
The advantage he names is the NVIDIA ecosystem. Kernel work and Torch support NVIDIA much better than other chips, and open-source engines, including SGLang, do the same, so the stack stays NVIDIA-first. For 15 years he has said that next year would be the year of non-NVIDIA accelerators, and it has not happened. Trainium, TPUs, and AMD look quite good and are catching up. He still thinks they are quite behind.
He says he has been wrong about that call every year. NVIDIA's staying power is what has kept it dominating the market.
From the transcript
To what extent is CUDA the moat that everyone says it is for Nvidia? I actually, I think it's a little overstated. I don't think CUDA itself is demote. CUDA is actually kind of annoying to work with. It's like a pretty annoying programming model. But I do think all the NVIDIA, the ecosystem is. Like I think, you know, all the kernel optimizations and all this stuff that, you know, Torch always supports NVIDIA much better than anything else.
And the open source engines like VLM and SGLang, because of that, they always support it much better. So everything is NVIDIA first today. And that's probably going to continue. It's funny, for the last 15 years, I've been saying, next year is the year of non-NVIDIA accelerators. It never keeps happening. I'm always wrong. NVIDIA has just shown to be incredibly good at having that staying power and just dominating the market.