You are currently viewing SemiWiki as a guest which gives you limited access to the site. To view blog comments and experience other SemiWiki features you must be a registered member. Registration is fast, simple, and absolutely free so please, join our community today!
AFAIK Cerebras have not done this for the scaled-up AI case which will drive the entire industry in the next few years (as opposed to particular benchmarks that they chose) and neither has any independent test including anything on SemiAnalysis -- am I wrong?
I think you are correct - they have found a lucrative sub-market market for super-fast response times and token production. For instance, Opus-4.6 from Anthropic on Cerebras, is fast is but ~5x more expensive than the regular version on Cursor. Not sure if and when Cerebras will benchmark over a broader operating range outside of their sweet spot.
I find this guy's blogs to be interesting on the hardware/software challenges of serving coding agents. He explains why different compute paradigms / architectures are needed for different phases of inference for coding agents.
Opus-4.6 and GPT-5.3-Codex both use 𝗖𝗲𝗿𝗲𝗯𝗿𝗮𝘀 for fast inference options. Seems like companies are 𝗱𝗶𝘀𝗰𝗮𝗿𝗱𝗶𝗻𝗴 𝗡𝘃𝗶𝗱𝗶𝗮 for real-time agents. But is this the future trend? In my newest blog, I argue that Nvidia’s latest moves still demonstrate great potential for fast but economical agent...
www.linkedin.com
He's one of the guys who originally developed KV caching, while doing a post-Doc at University of Chicago. Now has a startup that is focused on making inference far more cost efficient.
We've been talking about inference here -- how about training? Or is this so much smaller as a fraction of the total AI market that it doesn't really matter?
We've been talking about inference here -- how about training? Or is this so much smaller as a fraction of the total AI market that it doesn't really matter?
Most analysis I've seen shows the TAMs between data center inference and training to be about 50/50 right now but tipping toward inference with a 35% CAGR / 20% CAGR differential. I think most new entrants also view training as the harder problem, with more legacy infrastructure, so they seem willing to cede that to NVIDIA and Google, in favor of the faster growing market.
www.linkedin.com